[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:03:09.020126 12725 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.109.126:42611
I20260709 13:03:09.021741 12725 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:03:09.022918 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:09.040139 12732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:09.040172 12734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:09.041787 12725 server_base.cc:1061] running on GCE node
W20260709 13:03:09.040582 12731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:10.229049 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:10.229336 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:10.229542 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602190229524 us; error 0 us; skew 500 ppm
I20260709 13:03:10.230438 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:10.235930 12725 webserver.cc:533] Webserver started at http://127.12.109.126:38515/ using document root <none> and password file <none>
I20260709 13:03:10.236804 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:10.237047 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:10.237501 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:10.243224 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/master-0-root/instance:
uuid: "976606eb42f54514b87d1653ce4a6bc8"
format_stamp: "Formatted at 2026-07-09 13:03:10 on dist-test-slave-zhdc"
I20260709 13:03:10.251003 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:03:10.256341 12740 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:10.257565 12725 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:03:10.257912 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/master-0-root
uuid: "976606eb42f54514b87d1653ce4a6bc8"
format_stamp: "Formatted at 2026-07-09 13:03:10 on dist-test-slave-zhdc"
I20260709 13:03:10.258450 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:10.322356 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:10.323933 12725 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:03:10.324412 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:10.396658 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.126:42611
I20260709 13:03:10.396731 12791 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.126:42611 every 8 connection(s)
I20260709 13:03:10.405647 12792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:10.429695 12792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8: Bootstrap starting.
I20260709 13:03:10.435504 12792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.437630 12792 log.cc:826] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8: Log is configured to *not* fsync() on all Append() calls
I20260709 13:03:10.442009 12792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8: No bootstrap required, opened a new log
I20260709 13:03:10.462251 12792 raft_consensus.cc:359] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "976606eb42f54514b87d1653ce4a6bc8" member_type: VOTER }
I20260709 13:03:10.462829 12792 raft_consensus.cc:385] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.463032 12792 raft_consensus.cc:740] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 976606eb42f54514b87d1653ce4a6bc8, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.463992 12792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "976606eb42f54514b87d1653ce4a6bc8" member_type: VOTER }
I20260709 13:03:10.464473 12792 raft_consensus.cc:399] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:03:10.464720 12792 raft_consensus.cc:493] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:03:10.465000 12792 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.469000 12792 raft_consensus.cc:515] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "976606eb42f54514b87d1653ce4a6bc8" member_type: VOTER }
I20260709 13:03:10.469683 12792 leader_election.cc:304] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 976606eb42f54514b87d1653ce4a6bc8; no voters: 
I20260709 13:03:10.471364 12792 leader_election.cc:290] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:03:10.471735 12795 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:10.474092 12795 raft_consensus.cc:697] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [term 1 LEADER]: Becoming Leader. State: Replica: 976606eb42f54514b87d1653ce4a6bc8, State: Running, Role: LEADER
I20260709 13:03:10.475062 12795 consensus_queue.cc:237] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "976606eb42f54514b87d1653ce4a6bc8" member_type: VOTER }
I20260709 13:03:10.476136 12792 sys_catalog.cc:565] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:03:10.487320 12797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "976606eb42f54514b87d1653ce4a6bc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "976606eb42f54514b87d1653ce4a6bc8" member_type: VOTER } }
I20260709 13:03:10.487381 12796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 976606eb42f54514b87d1653ce4a6bc8. Latest consensus state: current_term: 1 leader_uuid: "976606eb42f54514b87d1653ce4a6bc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "976606eb42f54514b87d1653ce4a6bc8" member_type: VOTER } }
I20260709 13:03:10.488080 12797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:10.488132 12796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:10.491531 12805 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:03:10.503159 12805 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:03:10.509845 12725 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:03:10.521358 12805 catalog_manager.cc:1383] Generated new cluster ID: c5ec82dd02d747d5b8640f6aff41e84b
I20260709 13:03:10.521657 12805 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:03:10.537401 12805 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:03:10.539028 12805 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:03:10.554742 12805 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8: Generated new TSK 0
I20260709 13:03:10.555809 12805 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:03:10.578001 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:10.587689 12814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:10.588536 12815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:10.593641 12725 server_base.cc:1061] running on GCE node
W20260709 13:03:10.593582 12817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:10.594820 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:10.595055 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:10.595263 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602190595246 us; error 0 us; skew 500 ppm
I20260709 13:03:10.595932 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:10.600468 12725 webserver.cc:533] Webserver started at http://127.12.109.65:44155/ using document root <none> and password file <none>
I20260709 13:03:10.601030 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:10.601243 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:10.601533 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:10.602725 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-0-root/instance:
uuid: "be35d11a361b4ef8b3442d16fa47dadc"
format_stamp: "Formatted at 2026-07-09 13:03:10 on dist-test-slave-zhdc"
I20260709 13:03:10.608596 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:03:10.613281 12822 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:10.614207 12725 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:03:10.614564 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-0-root
uuid: "be35d11a361b4ef8b3442d16fa47dadc"
format_stamp: "Formatted at 2026-07-09 13:03:10 on dist-test-slave-zhdc"
I20260709 13:03:10.614872 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:10.659638 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:10.661010 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:10.663905 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:10.668529 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:10.668749 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:10.669086 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:10.669250 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:10.776531 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.65:42425
I20260709 13:03:10.776602 12884 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.65:42425 every 8 connection(s)
I20260709 13:03:10.783844 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:10.798938 12889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:10.801026 12890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:10.809855 12892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:10.814869 12725 server_base.cc:1061] running on GCE node
I20260709 13:03:10.815961 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:10.816241 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:10.816383 12885 heartbeater.cc:344] Connected to a master server at 127.12.109.126:42611
I20260709 13:03:10.816458 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602190816437 us; error 0 us; skew 500 ppm
I20260709 13:03:10.816970 12885 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:10.817379 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:10.818207 12885 heartbeater.cc:507] Master 127.12.109.126:42611 requested a full tablet report, sending...
I20260709 13:03:10.820745 12725 webserver.cc:533] Webserver started at http://127.12.109.66:38775/ using document root <none> and password file <none>
I20260709 13:03:10.821352 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:10.821583 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:10.821882 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:10.821933 12757 ts_manager.cc:194] Registered new tserver with Master: be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425)
I20260709 13:03:10.823112 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-1-root/instance:
uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
format_stamp: "Formatted at 2026-07-09 13:03:10 on dist-test-slave-zhdc"
I20260709 13:03:10.824998 12757 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52936
I20260709 13:03:10.829111 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:03:10.834192 12897 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:10.835101 12725 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:03:10.835395 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-1-root
uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
format_stamp: "Formatted at 2026-07-09 13:03:10 on dist-test-slave-zhdc"
I20260709 13:03:10.835672 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:10.849143 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:10.850473 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:10.853051 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:10.855978 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:10.856154 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:10.856444 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:10.856602 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:10.958536 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.66:38627
I20260709 13:03:10.958782 12959 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.66:38627 every 8 connection(s)
I20260709 13:03:10.970786 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260709 13:03:10.978122 12960 heartbeater.cc:344] Connected to a master server at 127.12.109.126:42611
I20260709 13:03:10.978655 12960 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:10.979852 12960 heartbeater.cc:507] Master 127.12.109.126:42611 requested a full tablet report, sending...
W20260709 13:03:10.981823 12963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:10.984282 12757 ts_manager.cc:194] Registered new tserver with Master: 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:10.986593 12757 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52938
W20260709 13:03:10.992009 12964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:10.996848 12966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:10.996956 12725 server_base.cc:1061] running on GCE node
I20260709 13:03:10.998306 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:10.998564 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:10.998766 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602190998750 us; error 0 us; skew 500 ppm
I20260709 13:03:10.999500 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:11.002215 12725 webserver.cc:533] Webserver started at http://127.12.109.67:41443/ using document root <none> and password file <none>
I20260709 13:03:11.002825 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:11.003059 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:11.003383 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:11.004869 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-2-root/instance:
uuid: "3637c09fbfb3492dbfd64b820edb64d1"
format_stamp: "Formatted at 2026-07-09 13:03:10 on dist-test-slave-zhdc"
I20260709 13:03:11.010695 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.005s
I20260709 13:03:11.014616 12971 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:11.015419 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:03:11.015734 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-2-root
uuid: "3637c09fbfb3492dbfd64b820edb64d1"
format_stamp: "Formatted at 2026-07-09 13:03:10 on dist-test-slave-zhdc"
I20260709 13:03:11.016089 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602188851028-12725-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:11.045753 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:11.047320 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:11.049315 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:11.052982 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:11.053246 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:11.053527 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:11.053761 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:11.136744 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.67:33003
I20260709 13:03:11.136795 13033 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.67:33003 every 8 connection(s)
I20260709 13:03:11.153592 13034 heartbeater.cc:344] Connected to a master server at 127.12.109.126:42611
I20260709 13:03:11.154016 13034 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:11.154934 13034 heartbeater.cc:507] Master 127.12.109.126:42611 requested a full tablet report, sending...
I20260709 13:03:11.157022 12757 ts_manager.cc:194] Registered new tserver with Master: 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:11.157148 12725 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014822454s
I20260709 13:03:11.158804 12757 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52942
I20260709 13:03:11.217275 12757 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52950:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:03:11.221315 12757 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:03:11.311434 12848 tablet_service.cc:1511] Processing CreateTablet for tablet 5a675310e5d64451be0e2037ae21c12d (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:03:11.312892 12850 tablet_service.cc:1511] Processing CreateTablet for tablet 8f5a6eb6aa46488c915db40e5af29f28 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:03:11.311434 12849 tablet_service.cc:1511] Processing CreateTablet for tablet 377ae8d110aa4a12a45ebe1597cb1a44 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:03:11.320255 12848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a675310e5d64451be0e2037ae21c12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.321208 12850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f5a6eb6aa46488c915db40e5af29f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.327068 12847 tablet_service.cc:1511] Processing CreateTablet for tablet 2e6a1b8c03e44012867f7f5c7fa73cbb (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:03:11.328953 12847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e6a1b8c03e44012867f7f5c7fa73cbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.336197 12849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 377ae8d110aa4a12a45ebe1597cb1a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.340776 12999 tablet_service.cc:1511] Processing CreateTablet for tablet 8f5a6eb6aa46488c915db40e5af29f28 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:03:11.346367 12999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f5a6eb6aa46488c915db40e5af29f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.348493 12996 tablet_service.cc:1511] Processing CreateTablet for tablet 2e6a1b8c03e44012867f7f5c7fa73cbb (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:03:11.359704 12846 tablet_service.cc:1511] Processing CreateTablet for tablet 3a6fd117644347009d06ef121ef2ee5f (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:03:11.362335 12846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a6fd117644347009d06ef121ef2ee5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.344478 12998 tablet_service.cc:1511] Processing CreateTablet for tablet 377ae8d110aa4a12a45ebe1597cb1a44 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:03:11.364447 12997 tablet_service.cc:1511] Processing CreateTablet for tablet 5a675310e5d64451be0e2037ae21c12d (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:03:11.366180 12996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e6a1b8c03e44012867f7f5c7fa73cbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.367976 12998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 377ae8d110aa4a12a45ebe1597cb1a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.369654 12997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a675310e5d64451be0e2037ae21c12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.389760 12925 tablet_service.cc:1511] Processing CreateTablet for tablet 8f5a6eb6aa46488c915db40e5af29f28 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:03:11.391242 12925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f5a6eb6aa46488c915db40e5af29f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.393267 12924 tablet_service.cc:1511] Processing CreateTablet for tablet 377ae8d110aa4a12a45ebe1597cb1a44 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:03:11.395376 12923 tablet_service.cc:1511] Processing CreateTablet for tablet 5a675310e5d64451be0e2037ae21c12d (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:03:11.397554 12922 tablet_service.cc:1511] Processing CreateTablet for tablet 2e6a1b8c03e44012867f7f5c7fa73cbb (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:03:11.399133 12924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 377ae8d110aa4a12a45ebe1597cb1a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.403461 12923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a675310e5d64451be0e2037ae21c12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.408746 12921 tablet_service.cc:1511] Processing CreateTablet for tablet 3a6fd117644347009d06ef121ef2ee5f (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:03:11.409816 12922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e6a1b8c03e44012867f7f5c7fa73cbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.410775 12921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a6fd117644347009d06ef121ef2ee5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.411182 12995 tablet_service.cc:1511] Processing CreateTablet for tablet 3a6fd117644347009d06ef121ef2ee5f (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:03:11.412741 12995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a6fd117644347009d06ef121ef2ee5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.413251 12850 tablet_service.cc:1511] Processing CreateTablet for tablet edff4ec626ca49ccadc3e871a4a9f7b9 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:03:11.414541 12850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edff4ec626ca49ccadc3e871a4a9f7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.452946 12994 tablet_service.cc:1511] Processing CreateTablet for tablet edff4ec626ca49ccadc3e871a4a9f7b9 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:03:11.441010 12920 tablet_service.cc:1511] Processing CreateTablet for tablet edff4ec626ca49ccadc3e871a4a9f7b9 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:03:11.456506 12994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edff4ec626ca49ccadc3e871a4a9f7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.456449 12845 tablet_service.cc:1511] Processing CreateTablet for tablet ed86e84c2abc4214af3e60aa476051e3 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:03:11.458607 12920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edff4ec626ca49ccadc3e871a4a9f7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.462447 12845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed86e84c2abc4214af3e60aa476051e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.487627 13052 tablet_bootstrap.cc:492] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc: Bootstrap starting.
I20260709 13:03:11.492064 12919 tablet_service.cc:1511] Processing CreateTablet for tablet ed86e84c2abc4214af3e60aa476051e3 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:03:11.493429 12919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed86e84c2abc4214af3e60aa476051e3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:11.500200 12804 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:11.506893 12999 tablet_service.cc:1511] Processing CreateTablet for tablet ed86e84c2abc4214af3e60aa476051e3 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:03:11.507675 12848 tablet_service.cc:1511] Processing CreateTablet for tablet 014d24f18def4caabe406d752914e49a (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:03:11.508170 12999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed86e84c2abc4214af3e60aa476051e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.509161 12848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 014d24f18def4caabe406d752914e49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.520519 13053 tablet_bootstrap.cc:492] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1: Bootstrap starting.
I20260709 13:03:11.533643 12924 tablet_service.cc:1511] Processing CreateTablet for tablet 014d24f18def4caabe406d752914e49a (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:03:11.535038 12924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 014d24f18def4caabe406d752914e49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.548851 12995 tablet_service.cc:1511] Processing CreateTablet for tablet 014d24f18def4caabe406d752914e49a (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:03:11.550212 12995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 014d24f18def4caabe406d752914e49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.550727 12849 tablet_service.cc:1511] Processing CreateTablet for tablet 5be387e9a3cf48a4b518280c92643005 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:03:11.551911 12849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5be387e9a3cf48a4b518280c92643005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.552892 12923 tablet_service.cc:1511] Processing CreateTablet for tablet 5be387e9a3cf48a4b518280c92643005 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:03:11.554030 12923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5be387e9a3cf48a4b518280c92643005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.559943 12997 tablet_service.cc:1511] Processing CreateTablet for tablet 5be387e9a3cf48a4b518280c92643005 (DEFAULT_TABLE table=test-workload [id=59c2e4df0106444181f31c962e5514bd]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:03:11.561228 12997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5be387e9a3cf48a4b518280c92643005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:11.578173 13054 tablet_bootstrap.cc:492] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70: Bootstrap starting.
I20260709 13:03:11.589769 13052 tablet_bootstrap.cc:654] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.602309 13054 tablet_bootstrap.cc:654] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.614712 13053 tablet_bootstrap.cc:654] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.620833 13054 tablet_bootstrap.cc:492] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70: No bootstrap required, opened a new log
I20260709 13:03:11.621587 13054 ts_tablet_manager.cc:1403] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent bootstrapping tablet: real 0.099s	user 0.012s	sys 0.004s
I20260709 13:03:11.623544 13052 tablet_bootstrap.cc:492] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc: No bootstrap required, opened a new log
I20260709 13:03:11.624130 13052 ts_tablet_manager.cc:1403] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc: Time spent bootstrapping tablet: real 0.162s	user 0.013s	sys 0.009s
I20260709 13:03:11.624819 13054 raft_consensus.cc:359] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.625623 13054 raft_consensus.cc:385] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.625998 13054 raft_consensus.cc:740] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.626801 13054 consensus_queue.cc:260] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.627089 13052 raft_consensus.cc:359] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.627844 13052 raft_consensus.cc:385] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.628196 13052 raft_consensus.cc:740] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.631366 13053 tablet_bootstrap.cc:492] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1: No bootstrap required, opened a new log
I20260709 13:03:11.631860 13053 ts_tablet_manager.cc:1403] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1: Time spent bootstrapping tablet: real 0.137s	user 0.013s	sys 0.004s
I20260709 13:03:11.632047 13052 consensus_queue.cc:260] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.635845 13053 raft_consensus.cc:359] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.637171 13053 raft_consensus.cc:385] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.637621 13053 raft_consensus.cc:740] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.638659 13053 consensus_queue.cc:260] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.641835 13052 ts_tablet_manager.cc:1434] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc: Time spent starting tablet: real 0.017s	user 0.011s	sys 0.002s
I20260709 13:03:11.642118 12960 heartbeater.cc:499] Master 127.12.109.126:42611 was elected leader, sending a full tablet report...
I20260709 13:03:11.642454 13054 ts_tablet_manager.cc:1434] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent starting tablet: real 0.020s	user 0.012s	sys 0.008s
I20260709 13:03:11.642526 12885 heartbeater.cc:499] Master 127.12.109.126:42611 was elected leader, sending a full tablet report...
I20260709 13:03:11.643471 13054 tablet_bootstrap.cc:492] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70: Bootstrap starting.
I20260709 13:03:11.649991 13054 tablet_bootstrap.cc:654] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.655705 13034 heartbeater.cc:499] Master 127.12.109.126:42611 was elected leader, sending a full tablet report...
I20260709 13:03:11.656749 13053 ts_tablet_manager.cc:1434] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1: Time spent starting tablet: real 0.025s	user 0.000s	sys 0.011s
I20260709 13:03:11.657744 13053 tablet_bootstrap.cc:492] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1: Bootstrap starting.
I20260709 13:03:11.657954 13052 tablet_bootstrap.cc:492] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc: Bootstrap starting.
I20260709 13:03:11.668157 13053 tablet_bootstrap.cc:654] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.680822 13053 tablet_bootstrap.cc:492] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1: No bootstrap required, opened a new log
I20260709 13:03:11.681398 13053 ts_tablet_manager.cc:1403] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.005s
I20260709 13:03:11.689486 13053 raft_consensus.cc:359] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.690405 13053 raft_consensus.cc:385] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.690805 13053 raft_consensus.cc:740] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.691587 13053 consensus_queue.cc:260] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.694018 13053 ts_tablet_manager.cc:1434] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:03:11.695037 13053 tablet_bootstrap.cc:492] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1: Bootstrap starting.
I20260709 13:03:11.701543 13053 tablet_bootstrap.cc:654] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.709496 13059 raft_consensus.cc:493] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.710042 13059 raft_consensus.cc:515] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.711107 13053 tablet_bootstrap.cc:492] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1: No bootstrap required, opened a new log
I20260709 13:03:11.711611 13053 ts_tablet_manager.cc:1403] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:03:11.713179 13059 leader_election.cc:290] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:11.714680 13053 raft_consensus.cc:359] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:11.715516 13053 raft_consensus.cc:385] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.715895 13053 raft_consensus.cc:740] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.716764 13053 consensus_queue.cc:260] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:11.717569 13052 tablet_bootstrap.cc:654] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.719365 13053 ts_tablet_manager.cc:1434] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
W20260709 13:03:11.720177 12961 tablet.cc:2406] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:03:11.720381 13053 tablet_bootstrap.cc:492] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1: Bootstrap starting.
I20260709 13:03:11.727885 13053 tablet_bootstrap.cc:654] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.740654 13052 tablet_bootstrap.cc:492] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc: No bootstrap required, opened a new log
I20260709 13:03:11.741129 13052 ts_tablet_manager.cc:1403] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc: Time spent bootstrapping tablet: real 0.094s	user 0.009s	sys 0.004s
I20260709 13:03:11.744992 13053 tablet_bootstrap.cc:492] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1: No bootstrap required, opened a new log
I20260709 13:03:11.745013 13054 tablet_bootstrap.cc:492] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70: No bootstrap required, opened a new log
I20260709 13:03:11.745764 13053 ts_tablet_manager.cc:1403] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.008s
I20260709 13:03:11.745796 13054 ts_tablet_manager.cc:1403] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent bootstrapping tablet: real 0.103s	user 0.035s	sys 0.058s
I20260709 13:03:11.751637 13054 raft_consensus.cc:359] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.752187 13054 raft_consensus.cc:385] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.752456 13054 raft_consensus.cc:740] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.753810 13054 consensus_queue.cc:260] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.753973 13053 raft_consensus.cc:359] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.754946 13053 raft_consensus.cc:385] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.755300 13053 raft_consensus.cc:740] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.755306 13052 raft_consensus.cc:359] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:11.756086 13052 raft_consensus.cc:385] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.756398 13052 raft_consensus.cc:740] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.757246 13052 consensus_queue.cc:260] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:11.756441 13053 consensus_queue.cc:260] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.761026 13054 ts_tablet_manager.cc:1434] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.005s
I20260709 13:03:11.762779 12935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" is_pre_election: true
I20260709 13:03:11.763912 13054 tablet_bootstrap.cc:492] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70: Bootstrap starting.
W20260709 13:03:11.764801 12826 leader_election.cc:343] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.762910 13053 ts_tablet_manager.cc:1434] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:03:11.765980 13053 tablet_bootstrap.cc:492] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1: Bootstrap starting.
I20260709 13:03:11.771961 13054 tablet_bootstrap.cc:654] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.779577 13052 ts_tablet_manager.cc:1434] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.001s
I20260709 13:03:11.773067 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1" is_pre_election: true
I20260709 13:03:11.779028 13053 tablet_bootstrap.cc:654] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.782855 13009 raft_consensus.cc:2468] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 0.
I20260709 13:03:11.784287 12826 leader_election.cc:304] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:11.785090 13059 raft_consensus.cc:2804] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:11.785709 13059 raft_consensus.cc:493] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.786187 13059 raft_consensus.cc:3060] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.787889 13054 tablet_bootstrap.cc:492] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70: No bootstrap required, opened a new log
I20260709 13:03:11.788349 13054 ts_tablet_manager.cc:1403] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent bootstrapping tablet: real 0.025s	user 0.002s	sys 0.009s
I20260709 13:03:11.791263 13054 raft_consensus.cc:359] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:11.792018 13054 raft_consensus.cc:385] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.792341 13054 raft_consensus.cc:740] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.794934 13053 tablet_bootstrap.cc:492] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1: No bootstrap required, opened a new log
I20260709 13:03:11.795444 13052 tablet_bootstrap.cc:492] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc: Bootstrap starting.
I20260709 13:03:11.797315 13053 ts_tablet_manager.cc:1403] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.001s
I20260709 13:03:11.798599 13059 raft_consensus.cc:515] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.800611 13059 leader_election.cc:290] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Requested vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:11.796291 13054 consensus_queue.cc:260] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:11.801286 12935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:11.801627 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:11.802150 13009 raft_consensus.cc:3060] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:11.803046 12826 leader_election.cc:343] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.800352 13053 raft_consensus.cc:359] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.804629 13053 raft_consensus.cc:385] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.805037 13053 raft_consensus.cc:740] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.805856 13053 consensus_queue.cc:260] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.804313 13054 ts_tablet_manager.cc:1434] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.007s
I20260709 13:03:11.807339 13054 tablet_bootstrap.cc:492] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70: Bootstrap starting.
I20260709 13:03:11.807919 13052 tablet_bootstrap.cc:654] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.811822 13053 ts_tablet_manager.cc:1434] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.003s
I20260709 13:03:11.817047 13053 tablet_bootstrap.cc:492] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1: Bootstrap starting.
I20260709 13:03:11.816051 13054 tablet_bootstrap.cc:654] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.826157 13053 tablet_bootstrap.cc:654] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.830878 13009 raft_consensus.cc:2468] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 1.
I20260709 13:03:11.832315 12826 leader_election.cc:304] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:11.833663 13059 raft_consensus.cc:2804] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:11.836071 13059 raft_consensus.cc:697] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Becoming Leader. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:11.837224 13059 consensus_queue.cc:237] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.844918 13054 tablet_bootstrap.cc:492] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70: No bootstrap required, opened a new log
I20260709 13:03:11.845503 13054 ts_tablet_manager.cc:1403] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.009s
I20260709 13:03:11.847553 13052 tablet_bootstrap.cc:492] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc: No bootstrap required, opened a new log
I20260709 13:03:11.848060 13052 ts_tablet_manager.cc:1403] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc: Time spent bootstrapping tablet: real 0.066s	user 0.038s	sys 0.005s
I20260709 13:03:11.848227 13054 raft_consensus.cc:359] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.848855 13054 raft_consensus.cc:385] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.849099 13054 raft_consensus.cc:740] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.849738 13054 consensus_queue.cc:260] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.851699 13058 raft_consensus.cc:493] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.852135 13058 raft_consensus.cc:515] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.850826 13052 raft_consensus.cc:359] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.854884 13052 raft_consensus.cc:385] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.856675 13052 raft_consensus.cc:740] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.856735 13058 leader_election.cc:290] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:11.857390 13052 consensus_queue.cc:260] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.859772 13052 ts_tablet_manager.cc:1434] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.003s
I20260709 13:03:11.860657 13052 tablet_bootstrap.cc:492] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc: Bootstrap starting.
I20260709 13:03:11.861968 13054 ts_tablet_manager.cc:1434] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 13:03:11.862903 13054 tablet_bootstrap.cc:492] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70: Bootstrap starting.
I20260709 13:03:11.869292 12755 catalog_manager.cc:5697] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc reported cstate change: term changed from 0 to 1, leader changed from <none> to be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65). New cstate: current_term: 1 leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.872344 13054 tablet_bootstrap.cc:654] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.882630 13053 tablet_bootstrap.cc:492] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1: No bootstrap required, opened a new log
I20260709 13:03:11.883193 13053 ts_tablet_manager.cc:1403] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.009s
I20260709 13:03:11.884123 13052 tablet_bootstrap.cc:654] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.886193 13053 raft_consensus.cc:359] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.887049 13053 raft_consensus.cc:385] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.887478 13053 raft_consensus.cc:740] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.888738 13053 consensus_queue.cc:260] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.891819 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1" is_pre_election: true
I20260709 13:03:11.892452 13009 raft_consensus.cc:2468] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 0.
I20260709 13:03:11.893885 13054 tablet_bootstrap.cc:492] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70: No bootstrap required, opened a new log
I20260709 13:03:11.893800 12901 leader_election.cc:304] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, 3637c09fbfb3492dbfd64b820edb64d1; no voters: 
I20260709 13:03:11.894625 13058 raft_consensus.cc:2804] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:11.895011 13058 raft_consensus.cc:493] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.895359 13058 raft_consensus.cc:3060] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.896248 13053 ts_tablet_manager.cc:1434] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 13:03:11.897297 13053 tablet_bootstrap.cc:492] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1: Bootstrap starting.
I20260709 13:03:11.903825 13053 tablet_bootstrap.cc:654] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.912942 13060 raft_consensus.cc:493] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.913605 13060 raft_consensus.cc:515] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.919502 13060 leader_election.cc:290] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:11.894423 13054 ts_tablet_manager.cc:1403] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.002s
I20260709 13:03:11.926038 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc" is_pre_election: true
W20260709 13:03:11.927649 12898 leader_election.cc:343] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.925854 13058 raft_consensus.cc:515] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.931592 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:11.931926 13059 raft_consensus.cc:493] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.932384 13059 raft_consensus.cc:515] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
W20260709 13:03:11.933146 12898 leader_election.cc:343] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.934316 13059 leader_election.cc:290] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:11.935530 12935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5a6eb6aa46488c915db40e5af29f28" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" is_pre_election: true
I20260709 13:03:11.936323 12935 raft_consensus.cc:2468] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 0.
I20260709 13:03:11.937853 12826 leader_election.cc:304] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:11.938751 13059 raft_consensus.cc:2804] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:11.938908 13060 raft_consensus.cc:493] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.939117 13059 raft_consensus.cc:493] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.939517 13059 raft_consensus.cc:3060] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.939500 13060 raft_consensus.cc:515] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.942042 13060 leader_election.cc:290] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:11.944769 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:11.945317 13009 raft_consensus.cc:3060] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.945428 13054 raft_consensus.cc:359] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.946348 13054 raft_consensus.cc:385] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.946700 13054 raft_consensus.cc:740] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.947575 13054 consensus_queue.cc:260] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.950729 13058 leader_election.cc:290] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 1 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:11.952281 13009 raft_consensus.cc:2468] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 1.
I20260709 13:03:11.953444 12901 leader_election.cc:304] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, 3637c09fbfb3492dbfd64b820edb64d1; no voters: be35d11a361b4ef8b3442d16fa47dadc
I20260709 13:03:11.956343 13058 raft_consensus.cc:2804] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:11.956813 13058 raft_consensus.cc:697] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 LEADER]: Becoming Leader. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:11.956771 13059 raft_consensus.cc:515] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:11.958762 13059 leader_election.cc:290] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Requested vote from peers 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:11.959854 12935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5a6eb6aa46488c915db40e5af29f28" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:11.960528 12935 raft_consensus.cc:3060] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.968429 12935 raft_consensus.cc:2468] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 1.
I20260709 13:03:11.969550 12826 leader_election.cc:304] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:11.970345 13059 raft_consensus.cc:2804] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:11.970803 13059 raft_consensus.cc:697] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Becoming Leader. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:11.971621 13059 consensus_queue.cc:237] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:11.972604 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5a6eb6aa46488c915db40e5af29f28" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:11.973316 13008 raft_consensus.cc:3060] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.975778 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5a6eb6aa46488c915db40e5af29f28" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1" is_pre_election: true
I20260709 13:03:11.957744 13058 consensus_queue.cc:237] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:11.982591 13008 raft_consensus.cc:2468] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 1.
I20260709 13:03:11.988628 13054 ts_tablet_manager.cc:1434] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.001s
I20260709 13:03:11.989636 13054 tablet_bootstrap.cc:492] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70: Bootstrap starting.
I20260709 13:03:11.990500 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc" is_pre_election: true
I20260709 13:03:11.993734 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6fd117644347009d06ef121ef2ee5f" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc" is_pre_election: true
W20260709 13:03:11.995985 12972 leader_election.cc:343] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:11.997331 12972 leader_election.cc:343] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.005015 13053 tablet_bootstrap.cc:492] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1: No bootstrap required, opened a new log
I20260709 13:03:12.010380 12935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" is_pre_election: true
I20260709 13:03:12.005553 13053 ts_tablet_manager.cc:1403] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1: Time spent bootstrapping tablet: real 0.108s	user 0.015s	sys 0.011s
W20260709 13:03:12.012974 12975 leader_election.cc:343] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.013417 12975 leader_election.cc:304] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1; no voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc
I20260709 13:03:12.011585 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6fd117644347009d06ef121ef2ee5f" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" is_pre_election: true
I20260709 13:03:12.014582 12934 raft_consensus.cc:2468] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 0.
I20260709 13:03:12.015362 13053 raft_consensus.cc:359] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.015767 13060 raft_consensus.cc:2749] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:12.016069 13053 raft_consensus.cc:385] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.016518 13053 raft_consensus.cc:740] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.017038 12975 leader_election.cc:304] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, 3637c09fbfb3492dbfd64b820edb64d1; no voters: be35d11a361b4ef8b3442d16fa47dadc
I20260709 13:03:12.017798 13060 raft_consensus.cc:2804] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:12.017351 13053 consensus_queue.cc:260] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.018162 13060 raft_consensus.cc:493] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.018543 13060 raft_consensus.cc:3060] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.022221 13052 tablet_bootstrap.cc:492] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc: No bootstrap required, opened a new log
I20260709 13:03:12.022755 13052 ts_tablet_manager.cc:1403] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc: Time spent bootstrapping tablet: real 0.162s	user 0.027s	sys 0.041s
I20260709 13:03:12.023362 12755 catalog_manager.cc:5697] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc reported cstate change: term changed from 0 to 1, leader changed from <none> to be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65). New cstate: current_term: 1 leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.025799 13052 raft_consensus.cc:359] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.026559 13052 raft_consensus.cc:385] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.026854 13052 raft_consensus.cc:740] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.030527 13060 raft_consensus.cc:515] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.034523 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6fd117644347009d06ef121ef2ee5f" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:12.034850 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6fd117644347009d06ef121ef2ee5f" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:12.035454 12934 raft_consensus.cc:3060] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.035692 13053 ts_tablet_manager.cc:1434] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.004s
W20260709 13:03:12.035856 12972 leader_election.cc:343] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.036473 13060 leader_election.cc:290] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:12.032613 13052 consensus_queue.cc:260] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.039044 13058 raft_consensus.cc:493] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.039786 13054 tablet_bootstrap.cc:654] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.039487 13058 raft_consensus.cc:515] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.040490 12756 catalog_manager.cc:5697] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.039494 13052 ts_tablet_manager.cc:1434] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:03:12.043059 13053 tablet_bootstrap.cc:492] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1: Bootstrap starting.
I20260709 13:03:12.043942 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc" is_pre_election: true
I20260709 13:03:12.043689 13058 leader_election.cc:290] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.045770 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1" is_pre_election: true
I20260709 13:03:12.046602 13008 raft_consensus.cc:2393] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in current term 1: Already voted for candidate be35d11a361b4ef8b3442d16fa47dadc in this term.
I20260709 13:03:12.047111 13052 tablet_bootstrap.cc:492] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc: Bootstrap starting.
I20260709 13:03:12.049429 12901 leader_election.cc:304] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70; no voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc
I20260709 13:03:12.050374 13058 raft_consensus.cc:3060] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.055943 12934 raft_consensus.cc:2468] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 1.
I20260709 13:03:12.057178 12975 leader_election.cc:304] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, 3637c09fbfb3492dbfd64b820edb64d1; no voters: be35d11a361b4ef8b3442d16fa47dadc
I20260709 13:03:12.060758 13058 raft_consensus.cc:2749] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:12.061957 13090 raft_consensus.cc:2804] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:12.062602 13090 raft_consensus.cc:697] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 LEADER]: Becoming Leader. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Running, Role: LEADER
I20260709 13:03:12.064196 13090 consensus_queue.cc:237] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.073814 13052 tablet_bootstrap.cc:654] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.075771 13053 tablet_bootstrap.cc:654] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.096161 13054 tablet_bootstrap.cc:492] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70: No bootstrap required, opened a new log
I20260709 13:03:12.096813 13054 ts_tablet_manager.cc:1403] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent bootstrapping tablet: real 0.107s	user 0.017s	sys 0.003s
I20260709 13:03:12.097390 13053 tablet_bootstrap.cc:492] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1: No bootstrap required, opened a new log
I20260709 13:03:12.097983 13053 ts_tablet_manager.cc:1403] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1: Time spent bootstrapping tablet: real 0.056s	user 0.004s	sys 0.009s
I20260709 13:03:12.099368 13059 raft_consensus.cc:493] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.103986 13059 raft_consensus.cc:515] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.104241 13054 raft_consensus.cc:359] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.104326 13053 raft_consensus.cc:359] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.105054 13052 tablet_bootstrap.cc:492] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc: No bootstrap required, opened a new log
I20260709 13:03:12.105080 13054 raft_consensus.cc:385] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.105154 13053 raft_consensus.cc:385] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.105598 13054 raft_consensus.cc:740] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.105718 13053 raft_consensus.cc:740] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.106137 13059 leader_election.cc:290] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.106592 13052 ts_tablet_manager.cc:1403] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc: Time spent bootstrapping tablet: real 0.062s	user 0.016s	sys 0.018s
I20260709 13:03:12.106842 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a675310e5d64451be0e2037ae21c12d" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" is_pre_election: true
I20260709 13:03:12.106587 13053 consensus_queue.cc:260] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.107287 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a675310e5d64451be0e2037ae21c12d" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1" is_pre_election: true
I20260709 13:03:12.107457 12934 raft_consensus.cc:2468] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 0.
I20260709 13:03:12.107918 13008 raft_consensus.cc:2468] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 0.
I20260709 13:03:12.108724 12826 leader_election.cc:304] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:12.109416 13053 ts_tablet_manager.cc:1434] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:03:12.109571 13059 raft_consensus.cc:2804] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:12.109969 13059 raft_consensus.cc:493] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.109829 13052 raft_consensus.cc:359] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.110399 13059 raft_consensus.cc:3060] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.110695 13052 raft_consensus.cc:385] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.110558 13054 consensus_queue.cc:260] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.111155 13052 raft_consensus.cc:740] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.116649 13053 tablet_bootstrap.cc:492] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1: Bootstrap starting.
I20260709 13:03:12.118707 13052 consensus_queue.cc:260] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.121196 13052 ts_tablet_manager.cc:1434] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:03:12.120523 12755 catalog_manager.cc:5697] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "3637c09fbfb3492dbfd64b820edb64d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:12.122143 13052 tablet_bootstrap.cc:492] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc: Bootstrap starting.
I20260709 13:03:12.123191 13054 ts_tablet_manager.cc:1434] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.003s
I20260709 13:03:12.124135 13054 tablet_bootstrap.cc:492] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70: Bootstrap starting.
I20260709 13:03:12.129519 13073 raft_consensus.cc:493] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.129978 13073 raft_consensus.cc:515] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.132316 13073 leader_election.cc:290] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.132933 13052 tablet_bootstrap.cc:654] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.135106 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014d24f18def4caabe406d752914e49a" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" is_pre_election: true
W20260709 13:03:12.136590 12826 leader_election.cc:343] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.139261 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014d24f18def4caabe406d752914e49a" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1" is_pre_election: true
I20260709 13:03:12.140076 13008 raft_consensus.cc:2468] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 0.
I20260709 13:03:12.141893 12826 leader_election.cc:304] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:12.142946 13073 raft_consensus.cc:2804] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:12.143365 13073 raft_consensus.cc:493] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.143800 13073 raft_consensus.cc:3060] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.146293 13054 tablet_bootstrap.cc:654] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.146780 13053 tablet_bootstrap.cc:654] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.151865 13073 raft_consensus.cc:515] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.153887 13073 leader_election.cc:290] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Requested vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.155099 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014d24f18def4caabe406d752914e49a" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
W20260709 13:03:12.157357 12826 leader_election.cc:343] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.158527 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014d24f18def4caabe406d752914e49a" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:12.159292 13008 raft_consensus.cc:3060] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.168862 13059 raft_consensus.cc:515] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.172716 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a675310e5d64451be0e2037ae21c12d" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:12.173588 12934 raft_consensus.cc:3060] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.184737 13008 raft_consensus.cc:2468] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 1.
I20260709 13:03:12.186504 12826 leader_election.cc:304] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:12.187541 13073 raft_consensus.cc:2804] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:12.188380 12934 raft_consensus.cc:2468] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 1.
I20260709 13:03:12.190300 12826 leader_election.cc:304] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:12.196570 13059 leader_election.cc:290] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Requested vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.197491 13059 raft_consensus.cc:2804] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:12.200217 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a675310e5d64451be0e2037ae21c12d" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:12.200989 13008 raft_consensus.cc:3060] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.207314 13073 raft_consensus.cc:697] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Becoming Leader. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:12.207343 13059 raft_consensus.cc:697] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Becoming Leader. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:12.208298 13073 consensus_queue.cc:237] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.208761 13059 consensus_queue.cc:237] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.209256 13054 tablet_bootstrap.cc:492] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70: No bootstrap required, opened a new log
I20260709 13:03:12.209878 13054 ts_tablet_manager.cc:1403] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent bootstrapping tablet: real 0.086s	user 0.016s	sys 0.023s
I20260709 13:03:12.212965 13053 tablet_bootstrap.cc:492] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1: No bootstrap required, opened a new log
I20260709 13:03:12.212734 13054 raft_consensus.cc:359] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.213420 13054 raft_consensus.cc:385] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.213794 13054 raft_consensus.cc:740] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.214020 13053 ts_tablet_manager.cc:1403] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1: Time spent bootstrapping tablet: real 0.098s	user 0.036s	sys 0.035s
I20260709 13:03:12.214542 13054 consensus_queue.cc:260] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.216243 13052 tablet_bootstrap.cc:492] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc: No bootstrap required, opened a new log
I20260709 13:03:12.216785 13052 ts_tablet_manager.cc:1403] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc: Time spent bootstrapping tablet: real 0.095s	user 0.031s	sys 0.026s
I20260709 13:03:12.217041 13053 raft_consensus.cc:359] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.217733 13053 raft_consensus.cc:385] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.218089 13053 raft_consensus.cc:740] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.218433 13008 raft_consensus.cc:2468] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 1.
I20260709 13:03:12.219789 13052 raft_consensus.cc:359] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.220485 13054 ts_tablet_manager.cc:1434] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:03:12.220563 13052 raft_consensus.cc:385] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.221055 13052 raft_consensus.cc:740] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.221481 13054 tablet_bootstrap.cc:492] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70: Bootstrap starting.
I20260709 13:03:12.225250 13052 consensus_queue.cc:260] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.230319 13052 ts_tablet_manager.cc:1434] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 13:03:12.231370 13052 tablet_bootstrap.cc:492] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc: Bootstrap starting.
I20260709 13:03:12.231166 12755 catalog_manager.cc:5697] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc reported cstate change: term changed from 0 to 1, leader changed from <none> to be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65). New cstate: current_term: 1 leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.218848 13053 consensus_queue.cc:260] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.236032 13053 ts_tablet_manager.cc:1434] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:03:12.238217 13052 tablet_bootstrap.cc:654] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.240774 13054 tablet_bootstrap.cc:654] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.249614 13052 tablet_bootstrap.cc:492] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc: No bootstrap required, opened a new log
I20260709 13:03:12.250025 13052 ts_tablet_manager.cc:1403] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.003s
I20260709 13:03:12.252257 13052 raft_consensus.cc:359] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.252751 13052 raft_consensus.cc:385] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.252930 13052 raft_consensus.cc:740] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.255254 13052 consensus_queue.cc:260] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.255544 13054 tablet_bootstrap.cc:492] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70: No bootstrap required, opened a new log
I20260709 13:03:12.256553 13054 ts_tablet_manager.cc:1403] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.005s
I20260709 13:03:12.257562 13073 consensus_queue.cc:1048] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.259130 13054 raft_consensus.cc:359] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.259794 13054 raft_consensus.cc:385] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.260116 13052 ts_tablet_manager.cc:1434] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.001s
I20260709 13:03:12.260094 13054 raft_consensus.cc:740] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.261165 13052 tablet_bootstrap.cc:492] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc: Bootstrap starting.
I20260709 13:03:12.260941 13054 consensus_queue.cc:260] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.263043 13054 ts_tablet_manager.cc:1434] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:03:12.263903 13054 tablet_bootstrap.cc:492] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70: Bootstrap starting.
I20260709 13:03:12.263976 13059 consensus_queue.cc:1048] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:12.270725 13052 tablet_bootstrap.cc:654] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.270814 13054 tablet_bootstrap.cc:654] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.273244 12755 catalog_manager.cc:5697] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc reported cstate change: term changed from 0 to 1, leader changed from <none> to be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65). New cstate: current_term: 1 leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.292865 13054 tablet_bootstrap.cc:492] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70: No bootstrap required, opened a new log
I20260709 13:03:12.293372 13054 ts_tablet_manager.cc:1403] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.014s
I20260709 13:03:12.296123 13054 raft_consensus.cc:359] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.296816 13054 raft_consensus.cc:385] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.297134 13054 raft_consensus.cc:740] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.297848 13054 consensus_queue.cc:260] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.300201 13054 ts_tablet_manager.cc:1434] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:03:12.308028 13059 raft_consensus.cc:493] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.308508 13059 raft_consensus.cc:515] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.310708 13059 leader_election.cc:290] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.313467 12935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be387e9a3cf48a4b518280c92643005" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" is_pre_election: true
I20260709 13:03:12.314144 12935 raft_consensus.cc:2468] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 0.
I20260709 13:03:12.315300 12826 leader_election.cc:304] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:12.317426 13098 raft_consensus.cc:2804] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:12.317813 13098 raft_consensus.cc:493] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.318197 13098 raft_consensus.cc:3060] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.313467 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be387e9a3cf48a4b518280c92643005" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1" is_pre_election: true
I20260709 13:03:12.323274 13009 raft_consensus.cc:2468] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 0.
I20260709 13:03:12.328284 13098 raft_consensus.cc:515] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.331061 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be387e9a3cf48a4b518280c92643005" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:12.331631 12934 raft_consensus.cc:3060] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.339773 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be387e9a3cf48a4b518280c92643005" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:12.340374 13009 raft_consensus.cc:3060] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.345197 13052 tablet_bootstrap.cc:492] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc: No bootstrap required, opened a new log
I20260709 13:03:12.345659 13052 ts_tablet_manager.cc:1403] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc: Time spent bootstrapping tablet: real 0.085s	user 0.009s	sys 0.045s
I20260709 13:03:12.347555 13009 raft_consensus.cc:2468] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 1.
I20260709 13:03:12.348246 13052 raft_consensus.cc:359] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.348903 13052 raft_consensus.cc:385] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.348717 12826 leader_election.cc:304] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:12.349598 13059 raft_consensus.cc:2804] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:12.349197 13052 raft_consensus.cc:740] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.350086 13059 raft_consensus.cc:697] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Becoming Leader. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:12.350696 13052 consensus_queue.cc:260] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.350886 13059 consensus_queue.cc:237] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.353812 13052 ts_tablet_manager.cc:1434] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:03:12.353907 13098 leader_election.cc:290] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Requested vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.355896 13090 raft_consensus.cc:493] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.356329 13090 raft_consensus.cc:515] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.357215 12934 raft_consensus.cc:2468] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 1.
I20260709 13:03:12.358314 13090 leader_election.cc:290] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:12.358484 13052 tablet_bootstrap.cc:492] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc: Bootstrap starting.
I20260709 13:03:12.359795 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc" is_pre_election: true
I20260709 13:03:12.360062 12935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" is_pre_election: true
I20260709 13:03:12.360720 12935 raft_consensus.cc:2468] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 0.
W20260709 13:03:12.361205 12972 leader_election.cc:343] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.362072 12975 leader_election.cc:304] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, 3637c09fbfb3492dbfd64b820edb64d1; no voters: be35d11a361b4ef8b3442d16fa47dadc
I20260709 13:03:12.363142 13090 raft_consensus.cc:2804] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:12.363535 13090 raft_consensus.cc:493] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.363931 13090 raft_consensus.cc:3060] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.364761 13052 tablet_bootstrap.cc:654] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.364813 12757 catalog_manager.cc:5697] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc reported cstate change: term changed from 0 to 1, leader changed from <none> to be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65). New cstate: current_term: 1 leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.370747 13090 raft_consensus.cc:515] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.374171 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:12.375716 12934 raft_consensus.cc:3060] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.376165 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:12.374673 13090 leader_election.cc:290] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
W20260709 13:03:12.377624 12972 leader_election.cc:343] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.383448 12934 raft_consensus.cc:2468] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 1.
I20260709 13:03:12.385671 12975 leader_election.cc:304] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, 3637c09fbfb3492dbfd64b820edb64d1; no voters: be35d11a361b4ef8b3442d16fa47dadc
I20260709 13:03:12.386834 13090 raft_consensus.cc:2804] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:12.387312 13090 raft_consensus.cc:697] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 LEADER]: Becoming Leader. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Running, Role: LEADER
I20260709 13:03:12.387616 13052 tablet_bootstrap.cc:492] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc: No bootstrap required, opened a new log
I20260709 13:03:12.388082 13052 ts_tablet_manager.cc:1403] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.007s
I20260709 13:03:12.388115 13090 consensus_queue.cc:237] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.390678 13052 raft_consensus.cc:359] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.391299 13052 raft_consensus.cc:385] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.391525 13052 raft_consensus.cc:740] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.392213 13052 consensus_queue.cc:260] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.394090 13052 ts_tablet_manager.cc:1434] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:03:12.395825 13098 raft_consensus.cc:493] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.396261 13098 raft_consensus.cc:515] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.399325 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" is_pre_election: true
I20260709 13:03:12.399509 13098 leader_election.cc:290] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.399942 12934 raft_consensus.cc:2468] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 0.
I20260709 13:03:12.401038 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1" is_pre_election: true
I20260709 13:03:12.401142 12826 leader_election.cc:304] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:12.401714 13008 raft_consensus.cc:2468] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 0.
I20260709 13:03:12.407558 13098 raft_consensus.cc:2804] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:12.407984 13098 raft_consensus.cc:493] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.408308 13098 raft_consensus.cc:3060] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.413865 13098 raft_consensus.cc:515] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.415855 13098 leader_election.cc:290] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Requested vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.416885 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:12.417203 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:12.417659 12934 raft_consensus.cc:3060] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.417822 13008 raft_consensus.cc:3060] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.417317 12754 catalog_manager.cc:5697] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "3637c09fbfb3492dbfd64b820edb64d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:12.425009 12934 raft_consensus.cc:2468] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 1.
I20260709 13:03:12.425138 13008 raft_consensus.cc:2468] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 1.
I20260709 13:03:12.427073 12826 leader_election.cc:304] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:12.427302 12860 raft_consensus.cc:3060] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.428483 13098 raft_consensus.cc:2804] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:12.428951 13098 raft_consensus.cc:697] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Becoming Leader. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:12.429755 13098 consensus_queue.cc:237] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.433940 13058 consensus_queue.cc:1048] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:12.438755 12754 catalog_manager.cc:5697] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc reported cstate change: term changed from 0 to 1, leader changed from <none> to be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65). New cstate: current_term: 1 leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.451311 13058 consensus_queue.cc:1048] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:12.456282 12860 raft_consensus.cc:3060] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.457615 13060 consensus_queue.cc:1048] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.464869 13060 consensus_queue.cc:1048] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:12.520314 13116 raft_consensus.cc:493] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.520850 13116 raft_consensus.cc:515] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.522991 13116 leader_election.cc:290] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.524400 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014d24f18def4caabe406d752914e49a" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1" is_pre_election: true
I20260709 13:03:12.524596 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014d24f18def4caabe406d752914e49a" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc" is_pre_election: true
I20260709 13:03:12.525189 13008 raft_consensus.cc:2393] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in current term 1: Already voted for candidate be35d11a361b4ef8b3442d16fa47dadc in this term.
I20260709 13:03:12.526154 12901 leader_election.cc:304] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70; no voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc
I20260709 13:03:12.526799 13116 raft_consensus.cc:3060] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.533494 13116 raft_consensus.cc:2749] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:12.537950 13098 consensus_queue.cc:1048] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.551011 13059 consensus_queue.cc:1048] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.586934 12934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
mode: GRACEFUL
new_leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
 from {username='slave'} at 127.0.0.1:51026
I20260709 13:03:12.587651 12934 raft_consensus.cc:606] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 LEADER]: Received request to transfer leadership to TS be35d11a361b4ef8b3442d16fa47dadc
I20260709 13:03:12.601912 13008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a6fd117644347009d06ef121ef2ee5f"
dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
mode: GRACEFUL
new_leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:34888
I20260709 13:03:12.602468 13008 raft_consensus.cc:606] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 LEADER]: Received request to transfer leadership to TS 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:12.616447 12860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9"
dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
mode: GRACEFUL
new_leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:34704
I20260709 13:03:12.616989 12860 raft_consensus.cc:606] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Received request to transfer leadership to TS 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:12.622614 13008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3"
dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
mode: GRACEFUL
new_leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:34888
I20260709 13:03:12.623248 13008 raft_consensus.cc:606] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 LEADER]: Received request to transfer leadership to TS 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:12.628791 12860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "014d24f18def4caabe406d752914e49a"
dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
mode: GRACEFUL
new_leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:34704
I20260709 13:03:12.629427 12860 raft_consensus.cc:606] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Received request to transfer leadership to TS 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:12.634824 12860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5be387e9a3cf48a4b518280c92643005"
dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
mode: GRACEFUL
new_leader_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
 from {username='slave'} at 127.0.0.1:34704
I20260709 13:03:12.635401 12860 raft_consensus.cc:606] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Received request to transfer leadership to TS 3637c09fbfb3492dbfd64b820edb64d1
I20260709 13:03:12.700714 13059 consensus_queue.cc:1048] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.713034 13099 consensus_queue.cc:1048] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:12.722255 13059 consensus_queue.cc:1048] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.732189 13059 consensus_queue.cc:1048] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.749573 13073 raft_consensus.cc:993] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc: : Instructing follower 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:12.750083 13059 raft_consensus.cc:1081] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Signalling peer 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:12.752140 12934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "014d24f18def4caabe406d752914e49a"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:50998
I20260709 13:03:12.752774 12934 raft_consensus.cc:493] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:12.753152 12934 raft_consensus.cc:3060] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.753474 13059 raft_consensus.cc:493] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.753952 13059 raft_consensus.cc:515] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.756141 13059 leader_election.cc:290] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.757612 12935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" is_pre_election: true
I20260709 13:03:12.758400 12935 raft_consensus.cc:2393] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be35d11a361b4ef8b3442d16fa47dadc in current term 1: Already voted for candidate 3637c09fbfb3492dbfd64b820edb64d1 in this term.
I20260709 13:03:12.759500 12934 raft_consensus.cc:515] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.760584 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1" is_pre_election: true
I20260709 13:03:12.762419 12934 leader_election.cc:290] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.762619 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014d24f18def4caabe406d752914e49a" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:12.763186 12860 raft_consensus.cc:3055] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:12.763474 12860 raft_consensus.cc:740] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:12.763509 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014d24f18def4caabe406d752914e49a" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:12.764635 13008 raft_consensus.cc:3060] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.764935 12860 consensus_queue.cc:260] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.766149 12860 raft_consensus.cc:3060] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.767230 12826 leader_election.cc:304] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be35d11a361b4ef8b3442d16fa47dadc; no voters: 2f97ad8ad4894210b3679e4f7f3ffa70, 3637c09fbfb3492dbfd64b820edb64d1
I20260709 13:03:12.768777 13059 raft_consensus.cc:3060] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.771626 13008 raft_consensus.cc:2468] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
I20260709 13:03:12.772809 12901 leader_election.cc:304] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, 3637c09fbfb3492dbfd64b820edb64d1; no voters: 
I20260709 13:03:12.773594 13116 raft_consensus.cc:2804] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:12.774088 13116 raft_consensus.cc:697] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Leader. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:12.774863 12860 raft_consensus.cc:2468] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
I20260709 13:03:12.775007 13116 consensus_queue.cc:237] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.779224 13059 raft_consensus.cc:2749] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:12.782598 12757 catalog_manager.cc:5697] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 reported cstate change: term changed from 1 to 2, leader changed from be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65) to 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.800750 13059 consensus_queue.cc:1048] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.808354 13073 raft_consensus.cc:993] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc: : Instructing follower 3637c09fbfb3492dbfd64b820edb64d1 to start an election
I20260709 13:03:12.808800 13073 raft_consensus.cc:1081] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Signalling peer 3637c09fbfb3492dbfd64b820edb64d1 to start an election
I20260709 13:03:12.810714 13009 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5be387e9a3cf48a4b518280c92643005"
dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
 from {username='slave'} at 127.0.0.1:34868
I20260709 13:03:12.811244 13009 raft_consensus.cc:493] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:12.811625 13009 raft_consensus.cc:3060] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.813843 13098 consensus_queue.cc:1048] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.820797 13009 raft_consensus.cc:515] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.822600 13008 raft_consensus.cc:1240] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Rejecting Update request from peer be35d11a361b4ef8b3442d16fa47dadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:12.823416 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be387e9a3cf48a4b518280c92643005" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:12.824072 12860 raft_consensus.cc:3055] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:12.824389 12860 raft_consensus.cc:740] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:12.825243 12860 consensus_queue.cc:260] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.826166 12860 raft_consensus.cc:3060] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:12.829273 13098 consensus_queue.cc:1175] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:03:12.833802 12860 raft_consensus.cc:2468] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 2.
I20260709 13:03:12.835065 13009 leader_election.cc:290] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 2 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:12.836156 12935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be387e9a3cf48a4b518280c92643005" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:12.837219 12972 leader_election.cc:304] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:12.837685 12935 raft_consensus.cc:3060] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.838029 13060 raft_consensus.cc:2804] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:12.838534 13060 raft_consensus.cc:697] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 LEADER]: Becoming Leader. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Running, Role: LEADER
I20260709 13:03:12.839332 13060 consensus_queue.cc:237] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.840309 13090 consensus_queue.cc:1048] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.849104 12935 raft_consensus.cc:2468] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 2.
I20260709 13:03:12.849387 12754 catalog_manager.cc:5697] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 reported cstate change: term changed from 1 to 2, leader changed from be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65) to 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "3637c09fbfb3492dbfd64b820edb64d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:12.868392 13058 raft_consensus.cc:993] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70: : Instructing follower be35d11a361b4ef8b3442d16fa47dadc to start an election
I20260709 13:03:12.868973 13058 raft_consensus.cc:1081] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 LEADER]: Signalling peer be35d11a361b4ef8b3442d16fa47dadc to start an election
I20260709 13:03:12.871913 12859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb"
dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
 from {username='slave'} at 127.0.0.1:34688
I20260709 13:03:12.872483 12859 raft_consensus.cc:493] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:12.872891 12859 raft_consensus.cc:3060] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.872691 13134 consensus_queue.cc:1048] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.880925 12859 raft_consensus.cc:515] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.883148 12859 leader_election.cc:290] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 2 election: Requested vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.884042 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:12.884658 12934 raft_consensus.cc:3055] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:12.885017 12934 raft_consensus.cc:740] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:12.885723 12934 consensus_queue.cc:260] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.886691 12934 raft_consensus.cc:3060] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.887416 13060 raft_consensus.cc:993] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1: : Instructing follower 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:12.887859 13060 raft_consensus.cc:1081] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 LEADER]: Signalling peer 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:12.887789 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:12.888430 13009 raft_consensus.cc:3060] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.889667 12935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:51014
I20260709 13:03:12.890208 12935 raft_consensus.cc:493] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:12.890542 12935 raft_consensus.cc:3060] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.896507 13009 raft_consensus.cc:2468] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 2.
I20260709 13:03:12.896945 12935 raft_consensus.cc:515] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.897619 12934 raft_consensus.cc:2468] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 2.
I20260709 13:03:12.897799 12826 leader_election.cc:304] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:12.899219 13098 raft_consensus.cc:2804] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:12.899698 13098 raft_consensus.cc:697] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 2 LEADER]: Becoming Leader. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:12.900553 13098 consensus_queue.cc:237] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.901403 12859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:12.901947 12859 raft_consensus.cc:3060] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.906565 12935 leader_election.cc:290] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.908121 12859 raft_consensus.cc:2468] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
I20260709 13:03:12.908423 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:12.909008 13009 raft_consensus.cc:3055] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:12.909228 12898 leader_election.cc:304] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:12.909403 13009 raft_consensus.cc:740] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Running, Role: LEADER
I20260709 13:03:12.910104 13086 raft_consensus.cc:2804] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:12.910573 13086 raft_consensus.cc:697] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Leader. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:12.910307 13009 consensus_queue.cc:260] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.911494 13009 raft_consensus.cc:3060] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.911335 13086 consensus_queue.cc:237] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.909067 12757 catalog_manager.cc:5697] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc reported cstate change: term changed from 1 to 2, leader changed from 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66) to be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65). New cstate: current_term: 2 leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.919503 13009 raft_consensus.cc:2468] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
I20260709 13:03:12.919240 12754 catalog_manager.cc:5697] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 reported cstate change: term changed from 1 to 2, leader changed from 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67) to 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.926795 13098 consensus_queue.cc:1048] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.933554 13073 raft_consensus.cc:993] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc: : Instructing follower 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:12.933980 13073 raft_consensus.cc:1081] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Signalling peer 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:12.935477 12934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:50998
I20260709 13:03:12.936002 12934 raft_consensus.cc:493] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:12.936381 12934 raft_consensus.cc:3060] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.938556 13099 consensus_queue.cc:1048] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.944666 12934 raft_consensus.cc:515] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.946192 12935 raft_consensus.cc:1240] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Rejecting Update request from peer be35d11a361b4ef8b3442d16fa47dadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:12.947909 12859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:12.948657 12859 raft_consensus.cc:3055] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:12.949048 12859 raft_consensus.cc:740] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:12.949353 13099 consensus_queue.cc:1059] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.949887 12934 leader_election.cc:290] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:12.951066 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:12.953286 12859 consensus_queue.cc:260] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.954243 12859 raft_consensus.cc:3060] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.959806 12859 raft_consensus.cc:2468] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
W20260709 13:03:12.960760 13059 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:03:12.960712 12898 leader_election.cc:304] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 3637c09fbfb3492dbfd64b820edb64d1
I20260709 13:03:12.962036 13116 raft_consensus.cc:2804] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:12.962612 13116 raft_consensus.cc:697] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Leader. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:12.963505 13116 consensus_queue.cc:237] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:12.971962 12754 catalog_manager.cc:5697] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 reported cstate change: term changed from 1 to 2, leader changed from be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65) to 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:13.003635 13134 raft_consensus.cc:993] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1: : Instructing follower 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:13.004140 13090 raft_consensus.cc:1081] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 LEADER]: Signalling peer 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:13.005570 12934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a6fd117644347009d06ef121ef2ee5f"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:51014
I20260709 13:03:13.006160 12934 raft_consensus.cc:493] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:13.006528 12934 raft_consensus.cc:3060] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:13.013062 12934 raft_consensus.cc:515] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:13.015084 12934 leader_election.cc:290] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:13.015816 12859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6fd117644347009d06ef121ef2ee5f" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:13.016274 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6fd117644347009d06ef121ef2ee5f" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:13.016423 12859 raft_consensus.cc:3060] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:13.016834 13009 raft_consensus.cc:3055] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:13.017156 13009 raft_consensus.cc:740] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Running, Role: LEADER
I20260709 13:03:13.017987 13009 consensus_queue.cc:260] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:13.019035 13009 raft_consensus.cc:3060] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:13.022970 12859 raft_consensus.cc:2468] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
I20260709 13:03:13.024184 12898 leader_election.cc:304] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:13.024987 13086 raft_consensus.cc:2804] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:13.025489 13086 raft_consensus.cc:697] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Leader. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:13.025595 13009 raft_consensus.cc:2468] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
I20260709 13:03:13.026345 13086 consensus_queue.cc:237] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:13.033232 12754 catalog_manager.cc:5697] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 reported cstate change: term changed from 1 to 2, leader changed from 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67) to 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:13.206024 12859 raft_consensus.cc:1275] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:13.207685 13086 consensus_queue.cc:1048] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:13.217136 13009 raft_consensus.cc:1275] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:13.219326 13058 consensus_queue.cc:1048] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:13.249294 12934 raft_consensus.cc:1275] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Refusing update from remote peer 3637c09fbfb3492dbfd64b820edb64d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:13.250813 13090 consensus_queue.cc:1048] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:13.265832 12859 raft_consensus.cc:1275] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Refusing update from remote peer 3637c09fbfb3492dbfd64b820edb64d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:13.267970 13138 consensus_queue.cc:1048] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:13.357247 13009 raft_consensus.cc:1275] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Refusing update from remote peer be35d11a361b4ef8b3442d16fa47dadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:13.358534 13059 consensus_queue.cc:1048] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:13.367868 12934 raft_consensus.cc:1275] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Refusing update from remote peer be35d11a361b4ef8b3442d16fa47dadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:13.369480 13099 consensus_queue.cc:1048] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:13.406625 12859 raft_consensus.cc:1275] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:13.407843 13058 consensus_queue.cc:1048] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:13.415002 13009 raft_consensus.cc:3060] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:13.422520 13009 raft_consensus.cc:1275] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:13.423833 13058 consensus_queue.cc:1048] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:13.432972 13009 raft_consensus.cc:1275] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:13.434214 13058 consensus_queue.cc:1048] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:13.441645 12859 raft_consensus.cc:1275] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:13.442392 12860 raft_consensus.cc:1275] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:13.445266 13116 consensus_queue.cc:1048] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:13.445690 13139 consensus_queue.cc:1048] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:13.455334 13009 raft_consensus.cc:1275] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:13.456524 13116 consensus_queue.cc:1048] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:14.790939 12881 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5060 after lost signal to thread 12728
W20260709 13:03:14.791646 12881 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5740 after lost signal to thread 12791
W20260709 13:03:14.792410 12881 debug-util.cc:398] Leaking SignalData structure 0x7b080018d2c0 after lost signal to thread 12884
W20260709 13:03:14.793030 12881 debug-util.cc:398] Leaking SignalData structure 0x7b080018d8a0 after lost signal to thread 12959
W20260709 13:03:14.793624 12881 debug-util.cc:398] Leaking SignalData structure 0x7b08000c6160 after lost signal to thread 13033
I20260709 13:03:15.720744 12725 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:03:15.721602 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:15.739398 12860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb"
dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
mode: GRACEFUL
new_leader_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
 from {username='slave'} at 127.0.0.1:34716
I20260709 13:03:15.739923 12860 raft_consensus.cc:606] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 2 LEADER]: Received request to transfer leadership to TS 3637c09fbfb3492dbfd64b820edb64d1
I20260709 13:03:15.750957 12934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
mode: GRACEFUL
new_leader_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
 from {username='slave'} at 127.0.0.1:51040
I20260709 13:03:15.751471 12934 raft_consensus.cc:606] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Received request to transfer leadership to TS 3637c09fbfb3492dbfd64b820edb64d1
I20260709 13:03:15.752602 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:03:15.753221 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:15.923336 13147 raft_consensus.cc:993] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc: : Instructing follower 3637c09fbfb3492dbfd64b820edb64d1 to start an election
I20260709 13:03:15.923856 13142 raft_consensus.cc:1081] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 2 LEADER]: Signalling peer 3637c09fbfb3492dbfd64b820edb64d1 to start an election
I20260709 13:03:15.925416 13009 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb"
dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
 from {username='slave'} at 127.0.0.1:34868
I20260709 13:03:15.925912 13009 raft_consensus.cc:493] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:15.926302 13009 raft_consensus.cc:3060] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:15.930740 13009 raft_consensus.cc:515] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:15.932253 13009 leader_election.cc:290] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 3 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:15.933146 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:15.933358 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:15.933756 12860 raft_consensus.cc:3055] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:15.933955 12934 raft_consensus.cc:3060] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:15.934165 12860 raft_consensus.cc:740] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:15.935070 12860 consensus_queue.cc:260] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:15.936336 12860 raft_consensus.cc:3060] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:15.938567 12934 raft_consensus.cc:2468] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 3.
I20260709 13:03:15.939600 12975 leader_election.cc:304] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, 3637c09fbfb3492dbfd64b820edb64d1; no voters: 
I20260709 13:03:15.940289 13141 raft_consensus.cc:2804] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:15.941576 13141 raft_consensus.cc:697] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Becoming Leader. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Running, Role: LEADER
I20260709 13:03:15.942406 13141 consensus_queue.cc:237] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:15.943570 12860 raft_consensus.cc:2468] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 3.
I20260709 13:03:15.949178 12754 catalog_manager.cc:5697] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 reported cstate change: term changed from 2 to 3, leader changed from be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65) to 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "3637c09fbfb3492dbfd64b820edb64d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:15.963584 13154 raft_consensus.cc:993] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70: : Instructing follower 3637c09fbfb3492dbfd64b820edb64d1 to start an election
I20260709 13:03:15.964054 13143 raft_consensus.cc:1081] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Signalling peer 3637c09fbfb3492dbfd64b820edb64d1 to start an election
I20260709 13:03:15.965762 13009 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3"
dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
 from {username='slave'} at 127.0.0.1:34874
I20260709 13:03:15.966363 13009 raft_consensus.cc:493] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:15.966722 13009 raft_consensus.cc:3060] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:15.973352 13009 raft_consensus.cc:515] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:15.976280 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:15.976665 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:15.976904 12860 raft_consensus.cc:3060] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:15.977236 12934 raft_consensus.cc:3055] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:15.977571 12934 raft_consensus.cc:740] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:15.978266 12934 consensus_queue.cc:260] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:15.979220 12934 raft_consensus.cc:3060] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:15.984609 12860 raft_consensus.cc:2468] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 3.
I20260709 13:03:15.985368 12934 raft_consensus.cc:2468] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 3.
I20260709 13:03:15.986137 12972 leader_election.cc:304] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:15.975739 13009 leader_election.cc:290] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 3 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:15.987152 13141 raft_consensus.cc:2804] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:15.987654 13141 raft_consensus.cc:697] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Becoming Leader. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Running, Role: LEADER
I20260709 13:03:15.988442 13141 consensus_queue.cc:237] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:15.996275 12757 catalog_manager.cc:5697] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 reported cstate change: term changed from 2 to 3, leader changed from 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66) to 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "3637c09fbfb3492dbfd64b820edb64d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:16.476640 12860 raft_consensus.cc:1275] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Refusing update from remote peer 3637c09fbfb3492dbfd64b820edb64d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:16.478016 13141 consensus_queue.cc:1048] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:16.488392 12934 raft_consensus.cc:1275] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Refusing update from remote peer 3637c09fbfb3492dbfd64b820edb64d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:16.489912 13148 consensus_queue.cc:1048] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:16.500744 12859 raft_consensus.cc:1275] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Refusing update from remote peer 3637c09fbfb3492dbfd64b820edb64d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:16.502605 13148 consensus_queue.cc:1048] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:16.526501 12935 raft_consensus.cc:1275] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Refusing update from remote peer 3637c09fbfb3492dbfd64b820edb64d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:16.528937 13148 consensus_queue.cc:1048] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:16.754175 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:16.763976 12859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f5a6eb6aa46488c915db40e5af29f28"
dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
mode: GRACEFUL
new_leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:34704
I20260709 13:03:16.764642 12859 raft_consensus.cc:606] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Received request to transfer leadership to TS 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:16.769619 12859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44"
dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
mode: GRACEFUL
new_leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:34704
I20260709 13:03:16.770260 12859 raft_consensus.cc:606] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Received request to transfer leadership to TS 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:16.774901 12859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a675310e5d64451be0e2037ae21c12d"
dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
mode: GRACEFUL
new_leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:34704
I20260709 13:03:16.775527 12859 raft_consensus.cc:606] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Received request to transfer leadership to TS 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:16.780323 13009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb"
dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
mode: GRACEFUL
new_leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:34888
I20260709 13:03:16.780766 13009 raft_consensus.cc:606] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Received request to transfer leadership to TS 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:16.791472 13009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3"
dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
mode: GRACEFUL
new_leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:34888
I20260709 13:03:16.792011 13009 raft_consensus.cc:606] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Received request to transfer leadership to TS 2f97ad8ad4894210b3679e4f7f3ffa70
I20260709 13:03:16.945673 13166 raft_consensus.cc:993] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc: : Instructing follower 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:16.946172 13142 raft_consensus.cc:1081] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Signalling peer 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:16.947583 12935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f5a6eb6aa46488c915db40e5af29f28"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:50998
I20260709 13:03:16.948117 12935 raft_consensus.cc:493] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:16.948418 12935 raft_consensus.cc:3060] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:16.952952 12935 raft_consensus.cc:515] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:16.954694 12935 leader_election.cc:290] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:16.955490 12859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5a6eb6aa46488c915db40e5af29f28" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:16.956105 12859 raft_consensus.cc:3055] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:16.956130 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5a6eb6aa46488c915db40e5af29f28" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:16.956568 12859 raft_consensus.cc:740] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:16.956774 13009 raft_consensus.cc:3060] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:16.957408 12859 consensus_queue.cc:260] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:16.958633 12859 raft_consensus.cc:3060] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:16.963227 13009 raft_consensus.cc:2468] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
I20260709 13:03:16.964560 12901 leader_election.cc:304] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, 3637c09fbfb3492dbfd64b820edb64d1; no voters: 
I20260709 13:03:16.965086 12859 raft_consensus.cc:2468] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
I20260709 13:03:16.965480 13143 raft_consensus.cc:2804] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:16.966123 13143 raft_consensus.cc:697] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Leader. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:16.967017 13143 consensus_queue.cc:237] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:16.975349 12754 catalog_manager.cc:5697] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 reported cstate change: term changed from 1 to 2, leader changed from be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65) to 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:17.026513 13158 raft_consensus.cc:993] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1: : Instructing follower 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:17.027012 13163 raft_consensus.cc:1081] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Signalling peer 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:17.028604 12935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:51014
I20260709 13:03:17.029134 12935 raft_consensus.cc:493] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:17.029456 12935 raft_consensus.cc:3060] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.034552 12935 raft_consensus.cc:515] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:17.036630 12935 leader_election.cc:290] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 4 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:17.037601 12859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:17.038278 12859 raft_consensus.cc:3060] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.038395 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e6a1b8c03e44012867f7f5c7fa73cbb" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:17.038988 13009 raft_consensus.cc:3055] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:17.039237 13009 raft_consensus.cc:740] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Running, Role: LEADER
I20260709 13:03:17.039913 13009 consensus_queue.cc:260] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:17.040710 13009 raft_consensus.cc:3060] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.045744 12859 raft_consensus.cc:2468] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 4.
I20260709 13:03:17.048218 13142 raft_consensus.cc:993] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc: : Instructing follower 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:17.048624 13142 raft_consensus.cc:1081] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Signalling peer 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:17.050024 12898 leader_election.cc:304] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:17.050931 13143 raft_consensus.cc:2804] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:17.052016 13143 raft_consensus.cc:697] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 4 LEADER]: Becoming Leader. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:17.052867 13009 raft_consensus.cc:2468] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 4.
I20260709 13:03:17.052909 13143 consensus_queue.cc:237] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:17.053964 12935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:50998
I20260709 13:03:17.054539 13158 raft_consensus.cc:993] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1: : Instructing follower 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:17.054587 12935 raft_consensus.cc:493] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:17.055051 13158 raft_consensus.cc:1081] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Signalling peer 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:17.055109 12935 raft_consensus.cc:3060] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:17.062700 12934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:51014
I20260709 13:03:17.063383 12934 raft_consensus.cc:493] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:17.063184 12935 raft_consensus.cc:515] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:17.063868 12934 raft_consensus.cc:3060] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.065464 12935 leader_election.cc:290] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:17.061229 12755 catalog_manager.cc:5697] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 reported cstate change: term changed from 3 to 4, leader changed from 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67) to 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66). New cstate: current_term: 4 leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:17.069819 12859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:17.070448 12859 raft_consensus.cc:3055] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:17.070780 12859 raft_consensus.cc:740] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:17.071556 12859 consensus_queue.cc:260] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:17.072714 12859 raft_consensus.cc:3060] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:17.072566 12934 raft_consensus.cc:515] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:17.080713 12859 raft_consensus.cc:2468] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
I20260709 13:03:17.081189 12934 leader_election.cc:290] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 4 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:17.082548 12898 leader_election.cc:304] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:17.083230 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:17.083715 13143 raft_consensus.cc:2804] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:17.083979 12860 raft_consensus.cc:3060] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.084254 13143 raft_consensus.cc:697] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Leader. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:17.085160 13143 consensus_queue.cc:237] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:17.092413 13009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:17.092983 13009 raft_consensus.cc:3060] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:17.098338 12860 raft_consensus.cc:2468] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 4.
I20260709 13:03:17.099370 12858 raft_consensus.cc:1240] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 4 FOLLOWER]: Rejecting Update request from peer 3637c09fbfb3492dbfd64b820edb64d1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:17.100384 12898 leader_election.cc:304] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:17.101820 13164 raft_consensus.cc:2804] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:17.102437 13164 raft_consensus.cc:697] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 4 LEADER]: Becoming Leader. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:17.102608 13163 consensus_queue.cc:1059] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:17.103549 13163 raft_consensus.cc:3055] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:17.103886 13163 raft_consensus.cc:740] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Running, Role: LEADER
I20260709 13:03:17.103295 13164 consensus_queue.cc:237] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:17.104661 13163 consensus_queue.cc:260] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:17.105716 13163 raft_consensus.cc:3060] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.112485 13009 raft_consensus.cc:2468] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
I20260709 13:03:17.115660 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed86e84c2abc4214af3e60aa476051e3" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:17.121802 13008 raft_consensus.cc:2468] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 4.
I20260709 13:03:17.128094 13166 raft_consensus.cc:993] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc: : Instructing follower 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:17.128537 13166 raft_consensus.cc:1081] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Signalling peer 2f97ad8ad4894210b3679e4f7f3ffa70 to start an election
I20260709 13:03:17.130147 12934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a675310e5d64451be0e2037ae21c12d"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
 from {username='slave'} at 127.0.0.1:50998
I20260709 13:03:17.130996 12934 raft_consensus.cc:493] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:17.131366 12934 raft_consensus.cc:3060] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:17.133580 12755 catalog_manager.cc:5697] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 reported cstate change: term changed from 1 to 2, leader changed from be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65) to 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:17.135785 12934 raft_consensus.cc:515] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:17.138248 12860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a675310e5d64451be0e2037ae21c12d" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:17.138857 12860 raft_consensus.cc:3055] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:17.139174 12860 raft_consensus.cc:740] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:17.139111 13008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a675310e5d64451be0e2037ae21c12d" candidate_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:17.137856 12934 leader_election.cc:290] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:17.139791 13008 raft_consensus.cc:3060] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:17.141594 12860 consensus_queue.cc:260] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:17.142902 12860 raft_consensus.cc:3060] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:17.149097 12860 raft_consensus.cc:2468] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
I20260709 13:03:17.149092 13008 raft_consensus.cc:2468] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f97ad8ad4894210b3679e4f7f3ffa70 in term 2.
I20260709 13:03:17.149456 12755 catalog_manager.cc:5697] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 reported cstate change: term changed from 3 to 4, leader changed from 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67) to 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66). New cstate: current_term: 4 leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:17.150364 12898 leader_election.cc:304] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:17.151692 13143 raft_consensus.cc:2804] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:17.152140 13143 raft_consensus.cc:697] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Leader. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:17.152951 13143 consensus_queue.cc:237] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:17.165688 12755 catalog_manager.cc:5697] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 reported cstate change: term changed from 1 to 2, leader changed from be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65) to 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:17.551522 13008 raft_consensus.cc:1275] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:17.552050 13009 raft_consensus.cc:1275] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:17.553581 13004 raft_consensus.cc:1275] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 4 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:17.554159 13006 raft_consensus.cc:1275] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 4 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:17.555126 13164 consensus_queue.cc:1048] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:17.555217 13174 consensus_queue.cc:1048] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:17.557333 13171 consensus_queue.cc:1048] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:17.557548 13174 consensus_queue.cc:1048] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:17.569295 12860 raft_consensus.cc:1275] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 4 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:17.578323 13164 consensus_queue.cc:1048] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:17.587033 12860 raft_consensus.cc:1275] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:17.591192 13143 consensus_queue.cc:1048] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:17.597446 12860 raft_consensus.cc:1275] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 4 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:17.601243 13164 consensus_queue.cc:1048] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:17.616117 12858 raft_consensus.cc:1275] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:17.623253 13186 consensus_queue.cc:1048] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:17.634919 12856 raft_consensus.cc:1275] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:17.637383 13188 consensus_queue.cc:1048] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:17.649252 13004 raft_consensus.cc:1275] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Refusing update from remote peer 2f97ad8ad4894210b3679e4f7f3ffa70: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:17.650833 13187 consensus_queue.cc:1048] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:03:18.425479 12881 debug-util.cc:398] Leaking SignalData structure 0x7b0800174e40 after lost signal to thread 12728
W20260709 13:03:18.426224 12881 debug-util.cc:398] Leaking SignalData structure 0x7b080017dd60 after lost signal to thread 12791
W20260709 13:03:18.426924 12881 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5500 after lost signal to thread 12884
W20260709 13:03:18.427524 12881 debug-util.cc:398] Leaking SignalData structure 0x7b0800174880 after lost signal to thread 12959
W20260709 13:03:18.428139 12881 debug-util.cc:398] Leaking SignalData structure 0x7b08001dcc60 after lost signal to thread 13033
I20260709 13:03:20.038663 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:20.046005 12934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
mode: GRACEFUL
new_leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
 from {username='slave'} at 127.0.0.1:51040
I20260709 13:03:20.046732 12934 raft_consensus.cc:606] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Received request to transfer leadership to TS be35d11a361b4ef8b3442d16fa47dadc
I20260709 13:03:20.049280 12934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a6fd117644347009d06ef121ef2ee5f"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
mode: GRACEFUL
new_leader_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
 from {username='slave'} at 127.0.0.1:51040
I20260709 13:03:20.049814 12934 raft_consensus.cc:606] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Received request to transfer leadership to TS 3637c09fbfb3492dbfd64b820edb64d1
I20260709 13:03:20.052177 12934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a675310e5d64451be0e2037ae21c12d"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
mode: GRACEFUL
new_leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
 from {username='slave'} at 127.0.0.1:51040
I20260709 13:03:20.052538 12934 raft_consensus.cc:606] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Received request to transfer leadership to TS be35d11a361b4ef8b3442d16fa47dadc
I20260709 13:03:20.054495 12934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f5a6eb6aa46488c915db40e5af29f28"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
mode: GRACEFUL
new_leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
 from {username='slave'} at 127.0.0.1:51040
I20260709 13:03:20.054844 12934 raft_consensus.cc:606] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Received request to transfer leadership to TS be35d11a361b4ef8b3442d16fa47dadc
I20260709 13:03:20.056746 12934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9"
dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
mode: GRACEFUL
new_leader_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
 from {username='slave'} at 127.0.0.1:51040
I20260709 13:03:20.057140 12934 raft_consensus.cc:606] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Received request to transfer leadership to TS 3637c09fbfb3492dbfd64b820edb64d1
I20260709 13:03:20.058142 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:20.058550 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:20.075781 13187 raft_consensus.cc:993] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70: : Instructing follower be35d11a361b4ef8b3442d16fa47dadc to start an election
I20260709 13:03:20.076318 13199 raft_consensus.cc:1081] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Signalling peer be35d11a361b4ef8b3442d16fa47dadc to start an election
I20260709 13:03:20.077790 12859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44"
dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
 from {username='slave'} at 127.0.0.1:34688
I20260709 13:03:20.078293 12859 raft_consensus.cc:493] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:20.078645 12859 raft_consensus.cc:3060] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.083546 12859 raft_consensus.cc:515] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:20.085098 12859 leader_election.cc:290] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 3 election: Requested vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:20.085992 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:20.086458 13004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377ae8d110aa4a12a45ebe1597cb1a44" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:20.086700 12934 raft_consensus.cc:3055] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:20.087010 13004 raft_consensus.cc:3060] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.087006 12934 raft_consensus.cc:740] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:20.087911 12934 consensus_queue.cc:260] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:20.088783 12934 raft_consensus.cc:3060] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.093061 13004 raft_consensus.cc:2468] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 3.
I20260709 13:03:20.094238 12826 leader_election.cc:304] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:20.094278 12934 raft_consensus.cc:2468] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 3.
I20260709 13:03:20.095618 13200 raft_consensus.cc:2804] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:20.096340 13200 raft_consensus.cc:697] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 3 LEADER]: Becoming Leader. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:20.097268 13200 consensus_queue.cc:237] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:20.105217 12755 catalog_manager.cc:5697] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc reported cstate change: term changed from 2 to 3, leader changed from 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66) to be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65). New cstate: current_term: 3 leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:20.129034 13187 raft_consensus.cc:993] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70: : Instructing follower 3637c09fbfb3492dbfd64b820edb64d1 to start an election
I20260709 13:03:20.129462 13199 raft_consensus.cc:1081] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Signalling peer 3637c09fbfb3492dbfd64b820edb64d1 to start an election
I20260709 13:03:20.130952 13004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9"
dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
 from {username='slave'} at 127.0.0.1:34874
I20260709 13:03:20.131429 13004 raft_consensus.cc:493] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:20.131735 13004 raft_consensus.cc:3060] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.136373 13004 raft_consensus.cc:515] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:20.138172 13004 leader_election.cc:290] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 3 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:20.138952 12859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:20.139092 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff4ec626ca49ccadc3e871a4a9f7b9" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:20.139531 12859 raft_consensus.cc:3060] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.139683 12934 raft_consensus.cc:3055] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:20.140019 12934 raft_consensus.cc:740] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:20.140575 12934 consensus_queue.cc:260] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:20.141459 12934 raft_consensus.cc:3060] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.143775 12859 raft_consensus.cc:2468] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 3.
I20260709 13:03:20.144953 12972 leader_election.cc:304] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:20.145831 13197 raft_consensus.cc:2804] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:20.146031 12934 raft_consensus.cc:2468] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 3.
I20260709 13:03:20.147045 13197 raft_consensus.cc:697] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Becoming Leader. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Running, Role: LEADER
I20260709 13:03:20.147859 13197 consensus_queue.cc:237] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:20.152400 13187 raft_consensus.cc:993] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70: : Instructing follower be35d11a361b4ef8b3442d16fa47dadc to start an election
I20260709 13:03:20.152918 13199 raft_consensus.cc:1081] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Signalling peer be35d11a361b4ef8b3442d16fa47dadc to start an election
I20260709 13:03:20.154452 12859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a675310e5d64451be0e2037ae21c12d"
dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
 from {username='slave'} at 127.0.0.1:34688
I20260709 13:03:20.154932 12859 raft_consensus.cc:493] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:20.155251 12859 raft_consensus.cc:3060] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.155930 12755 catalog_manager.cc:5697] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 reported cstate change: term changed from 2 to 3, leader changed from 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66) to 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "3637c09fbfb3492dbfd64b820edb64d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:20.160351 12859 raft_consensus.cc:515] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:20.162108 12859 leader_election.cc:290] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 3 election: Requested vote from peers 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627), 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003)
I20260709 13:03:20.162860 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a675310e5d64451be0e2037ae21c12d" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:20.163343 12934 raft_consensus.cc:3055] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:20.163525 12934 raft_consensus.cc:740] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:20.164177 12934 consensus_queue.cc:260] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:20.164160 13004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a675310e5d64451be0e2037ae21c12d" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:20.164986 12934 raft_consensus.cc:3060] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.165222 13004 raft_consensus.cc:3060] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.169811 12934 raft_consensus.cc:2468] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 3.
I20260709 13:03:20.170707 13004 raft_consensus.cc:2468] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 3.
I20260709 13:03:20.170883 12826 leader_election.cc:304] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f97ad8ad4894210b3679e4f7f3ffa70, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:20.171708 13200 raft_consensus.cc:2804] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:20.172153 13200 raft_consensus.cc:697] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 3 LEADER]: Becoming Leader. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:20.172971 13200 consensus_queue.cc:237] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:20.179375 12755 catalog_manager.cc:5697] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc reported cstate change: term changed from 2 to 3, leader changed from 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66) to be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65). New cstate: current_term: 3 leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:20.273531 13187 raft_consensus.cc:993] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70: : Instructing follower 3637c09fbfb3492dbfd64b820edb64d1 to start an election
I20260709 13:03:20.274008 13199 raft_consensus.cc:1081] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Signalling peer 3637c09fbfb3492dbfd64b820edb64d1 to start an election
I20260709 13:03:20.275473 13004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a6fd117644347009d06ef121ef2ee5f"
dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
 from {username='slave'} at 127.0.0.1:34874
I20260709 13:03:20.276069 13004 raft_consensus.cc:493] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:20.276437 13004 raft_consensus.cc:3060] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.281075 13004 raft_consensus.cc:515] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:20.282984 13004 leader_election.cc:290] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 3 election: Requested vote from peers be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:20.283622 12859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6fd117644347009d06ef121ef2ee5f" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
I20260709 13:03:20.283958 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6fd117644347009d06ef121ef2ee5f" candidate_uuid: "3637c09fbfb3492dbfd64b820edb64d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:20.284224 12859 raft_consensus.cc:3060] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.284584 12934 raft_consensus.cc:3055] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:20.284854 12934 raft_consensus.cc:740] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:20.285458 12934 consensus_queue.cc:260] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:20.286304 12934 raft_consensus.cc:3060] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.288462 12859 raft_consensus.cc:2468] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 3.
I20260709 13:03:20.289345 12972 leader_election.cc:304] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:20.290109 13197 raft_consensus.cc:2804] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:20.290501 12934 raft_consensus.cc:2468] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3637c09fbfb3492dbfd64b820edb64d1 in term 3.
I20260709 13:03:20.290609 13197 raft_consensus.cc:697] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Becoming Leader. State: Replica: 3637c09fbfb3492dbfd64b820edb64d1, State: Running, Role: LEADER
I20260709 13:03:20.291524 13197 consensus_queue.cc:237] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } }
I20260709 13:03:20.298527 12755 catalog_manager.cc:5697] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 reported cstate change: term changed from 2 to 3, leader changed from 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66) to 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "3637c09fbfb3492dbfd64b820edb64d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:20.528160 13004 raft_consensus.cc:1275] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Refusing update from remote peer be35d11a361b4ef8b3442d16fa47dadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.529464 13200 consensus_queue.cc:1048] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.539212 12934 raft_consensus.cc:1275] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Refusing update from remote peer be35d11a361b4ef8b3442d16fa47dadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.540840 13201 consensus_queue.cc:1048] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:20.571604 13004 raft_consensus.cc:1275] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Refusing update from remote peer be35d11a361b4ef8b3442d16fa47dadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.572739 13201 consensus_queue.cc:1048] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.584070 12934 raft_consensus.cc:1275] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Refusing update from remote peer be35d11a361b4ef8b3442d16fa47dadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.586884 13201 consensus_queue.cc:1048] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:20.589982 13187 raft_consensus.cc:993] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70: : Instructing follower be35d11a361b4ef8b3442d16fa47dadc to start an election
I20260709 13:03:20.590543 13199 raft_consensus.cc:1081] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Signalling peer be35d11a361b4ef8b3442d16fa47dadc to start an election
I20260709 13:03:20.591975 12859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f5a6eb6aa46488c915db40e5af29f28"
dest_uuid: "be35d11a361b4ef8b3442d16fa47dadc"
 from {username='slave'} at 127.0.0.1:34688
I20260709 13:03:20.592468 12859 raft_consensus.cc:493] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:20.592859 12859 raft_consensus.cc:3060] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.597944 12859 raft_consensus.cc:515] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:20.599397 12859 leader_election.cc:290] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 3 election: Requested vote from peers 3637c09fbfb3492dbfd64b820edb64d1 (127.12.109.67:33003), 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627)
I20260709 13:03:20.600111 13004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5a6eb6aa46488c915db40e5af29f28" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3637c09fbfb3492dbfd64b820edb64d1"
I20260709 13:03:20.600284 12934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5a6eb6aa46488c915db40e5af29f28" candidate_uuid: "be35d11a361b4ef8b3442d16fa47dadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70"
I20260709 13:03:20.600669 13004 raft_consensus.cc:3060] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.600724 12934 raft_consensus.cc:3055] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:20.601244 12934 raft_consensus.cc:740] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2f97ad8ad4894210b3679e4f7f3ffa70, State: Running, Role: LEADER
I20260709 13:03:20.602003 12934 consensus_queue.cc:260] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:20.603282 12934 raft_consensus.cc:3060] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.606673 13004 raft_consensus.cc:2468] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 3.
I20260709 13:03:20.607331 12935 raft_consensus.cc:1275] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Refusing update from remote peer 3637c09fbfb3492dbfd64b820edb64d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.607705 12826 leader_election.cc:304] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3637c09fbfb3492dbfd64b820edb64d1, be35d11a361b4ef8b3442d16fa47dadc; no voters: 
I20260709 13:03:20.608860 13197 consensus_queue.cc:1048] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.609742 12934 raft_consensus.cc:2468] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be35d11a361b4ef8b3442d16fa47dadc in term 3.
I20260709 13:03:20.613090 13201 raft_consensus.cc:2804] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:20.613907 13201 raft_consensus.cc:697] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 3 LEADER]: Becoming Leader. State: Replica: be35d11a361b4ef8b3442d16fa47dadc, State: Running, Role: LEADER
I20260709 13:03:20.614786 13201 consensus_queue.cc:237] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } }
I20260709 13:03:20.630098 12859 raft_consensus.cc:1275] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Refusing update from remote peer 3637c09fbfb3492dbfd64b820edb64d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.632920 13207 consensus_queue.cc:1048] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:20.635855 12755 catalog_manager.cc:5697] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc reported cstate change: term changed from 2 to 3, leader changed from 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66) to be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65). New cstate: current_term: 3 leader_uuid: "be35d11a361b4ef8b3442d16fa47dadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:20.675146 12858 raft_consensus.cc:1275] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Refusing update from remote peer 3637c09fbfb3492dbfd64b820edb64d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.676491 13207 consensus_queue.cc:1048] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be35d11a361b4ef8b3442d16fa47dadc" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.685638 12934 raft_consensus.cc:1275] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Refusing update from remote peer 3637c09fbfb3492dbfd64b820edb64d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.687211 13207 consensus_queue.cc:1048] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:21.059369 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:21.065078 12725 tablet_server.cc:179] TabletServer@127.12.109.65:0 shutting down...
W20260709 13:03:21.090639 12972 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.097630 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:21.098603 12725 tablet_replica.cc:333] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc: stopping tablet replica
I20260709 13:03:21.099684 12725 raft_consensus.cc:2243] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.100387 12725 raft_consensus.cc:2272] T ed86e84c2abc4214af3e60aa476051e3 P be35d11a361b4ef8b3442d16fa47dadc [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.102732 12725 tablet_replica.cc:333] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc: stopping tablet replica
I20260709 13:03:21.103279 12725 raft_consensus.cc:2243] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.103865 12725 raft_consensus.cc:2272] T 5be387e9a3cf48a4b518280c92643005 P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:21.104403 12972 consensus_peers.cc:597] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 -> Peer be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425): Couldn't send request to peer be35d11a361b4ef8b3442d16fa47dadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.106855 12725 tablet_replica.cc:333] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc: stopping tablet replica
I20260709 13:03:21.107522 12725 raft_consensus.cc:2243] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.107733 13004 raft_consensus.cc:1275] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Refusing update from remote peer be35d11a361b4ef8b3442d16fa47dadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:21.108065 12725 raft_consensus.cc:2272] T 2e6a1b8c03e44012867f7f5c7fa73cbb P be35d11a361b4ef8b3442d16fa47dadc [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:21.108207 12972 consensus_peers.cc:597] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 -> Peer be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425): Couldn't send request to peer be35d11a361b4ef8b3442d16fa47dadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.109189 13201 consensus_queue.cc:1048] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637c09fbfb3492dbfd64b820edb64d1" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:21.110216 12725 tablet_replica.cc:333] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc: stopping tablet replica
I20260709 13:03:21.110839 12725 raft_consensus.cc:2243] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.111299 12725 raft_consensus.cc:2272] T edff4ec626ca49ccadc3e871a4a9f7b9 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.115543 12725 tablet_replica.cc:333] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc: stopping tablet replica
I20260709 13:03:21.116343 12725 raft_consensus.cc:2243] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.117504 12725 raft_consensus.cc:2272] T 014d24f18def4caabe406d752914e49a P be35d11a361b4ef8b3442d16fa47dadc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.119269 12934 raft_consensus.cc:1275] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Refusing update from remote peer be35d11a361b4ef8b3442d16fa47dadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:21.120128 12725 tablet_replica.cc:333] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc: stopping tablet replica
I20260709 13:03:21.120886 12725 raft_consensus.cc:2243] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.121389 12725 raft_consensus.cc:2272] T 3a6fd117644347009d06ef121ef2ee5f P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.122160 13217 consensus_queue.cc:1048] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f97ad8ad4894210b3679e4f7f3ffa70" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:21.123919 12725 tablet_replica.cc:333] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc: stopping tablet replica
I20260709 13:03:21.124629 12725 raft_consensus.cc:2243] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.125615 12725 raft_consensus.cc:2272] T 5a675310e5d64451be0e2037ae21c12d P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.128151 12725 tablet_replica.cc:333] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc: stopping tablet replica
I20260709 13:03:21.128712 12725 raft_consensus.cc:2243] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.129365 12725 raft_consensus.cc:2272] T 8f5a6eb6aa46488c915db40e5af29f28 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.131503 12725 tablet_replica.cc:333] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc: stopping tablet replica
I20260709 13:03:21.131872 12725 raft_consensus.cc:2243] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.132701 12725 raft_consensus.cc:2272] T 377ae8d110aa4a12a45ebe1597cb1a44 P be35d11a361b4ef8b3442d16fa47dadc [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:21.136963 12972 consensus_peers.cc:597] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 -> Peer be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425): Couldn't send request to peer be35d11a361b4ef8b3442d16fa47dadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.255378 12725 tablet_server.cc:196] TabletServer@127.12.109.65:0 shutdown complete.
I20260709 13:03:21.273861 12725 tablet_server.cc:179] TabletServer@127.12.109.66:0 shutting down...
W20260709 13:03:21.299826 12898 consensus_peers.cc:597] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 -> Peer be35d11a361b4ef8b3442d16fa47dadc (127.12.109.65:42425): Couldn't send request to peer be35d11a361b4ef8b3442d16fa47dadc. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.304145 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:21.304797 12725 tablet_replica.cc:333] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70: stopping tablet replica
I20260709 13:03:21.305352 12725 raft_consensus.cc:2243] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.306185 12725 raft_consensus.cc:2272] T ed86e84c2abc4214af3e60aa476051e3 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.307874 12725 tablet_replica.cc:333] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70: stopping tablet replica
I20260709 13:03:21.308370 12725 raft_consensus.cc:2243] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.308766 12725 raft_consensus.cc:2272] T 5be387e9a3cf48a4b518280c92643005 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.310691 12725 tablet_replica.cc:333] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70: stopping tablet replica
I20260709 13:03:21.311157 12725 raft_consensus.cc:2243] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.311566 12725 raft_consensus.cc:2272] T 377ae8d110aa4a12a45ebe1597cb1a44 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.313426 12725 tablet_replica.cc:333] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70: stopping tablet replica
I20260709 13:03:21.313899 12725 raft_consensus.cc:2243] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.314318 12725 raft_consensus.cc:2272] T 8f5a6eb6aa46488c915db40e5af29f28 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.316601 12725 tablet_replica.cc:333] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70: stopping tablet replica
I20260709 13:03:21.317122 12725 raft_consensus.cc:2243] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.317503 12725 raft_consensus.cc:2272] T 5a675310e5d64451be0e2037ae21c12d P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.319448 12725 tablet_replica.cc:333] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70: stopping tablet replica
I20260709 13:03:21.319928 12725 raft_consensus.cc:2243] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.320304 12725 raft_consensus.cc:2272] T 3a6fd117644347009d06ef121ef2ee5f P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.322228 12725 tablet_replica.cc:333] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70: stopping tablet replica
I20260709 13:03:21.322666 12725 raft_consensus.cc:2243] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.323025 12725 raft_consensus.cc:2272] T edff4ec626ca49ccadc3e871a4a9f7b9 P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.324710 12725 tablet_replica.cc:333] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70: stopping tablet replica
I20260709 13:03:21.325297 12725 raft_consensus.cc:2243] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.326012 12725 raft_consensus.cc:2272] T 014d24f18def4caabe406d752914e49a P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.327863 12725 tablet_replica.cc:333] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70: stopping tablet replica
I20260709 13:03:21.328425 12725 raft_consensus.cc:2243] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 4 LEADER]: Raft consensus shutting down.
W20260709 13:03:21.328896 12975 consensus_peers.cc:597] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 -> Peer 2f97ad8ad4894210b3679e4f7f3ffa70 (127.12.109.66:38627): Couldn't send request to peer 2f97ad8ad4894210b3679e4f7f3ffa70. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.329190 12725 raft_consensus.cc:2272] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 2f97ad8ad4894210b3679e4f7f3ffa70 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.347296 12725 tablet_server.cc:196] TabletServer@127.12.109.66:0 shutdown complete.
I20260709 13:03:21.363238 12725 tablet_server.cc:179] TabletServer@127.12.109.67:0 shutting down...
I20260709 13:03:21.387084 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:21.387755 12725 tablet_replica.cc:333] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1: stopping tablet replica
I20260709 13:03:21.388377 12725 raft_consensus.cc:2243] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.389093 12725 raft_consensus.cc:2272] T 5be387e9a3cf48a4b518280c92643005 P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.390938 12725 tablet_replica.cc:333] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1: stopping tablet replica
I20260709 13:03:21.391407 12725 raft_consensus.cc:2243] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.391767 12725 raft_consensus.cc:2272] T ed86e84c2abc4214af3e60aa476051e3 P 3637c09fbfb3492dbfd64b820edb64d1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.393378 12725 tablet_replica.cc:333] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1: stopping tablet replica
I20260709 13:03:21.393821 12725 raft_consensus.cc:2243] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.394217 12725 raft_consensus.cc:2272] T 014d24f18def4caabe406d752914e49a P 3637c09fbfb3492dbfd64b820edb64d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.395824 12725 tablet_replica.cc:333] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1: stopping tablet replica
I20260709 13:03:21.396245 12725 raft_consensus.cc:2243] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.396582 12725 raft_consensus.cc:2272] T 2e6a1b8c03e44012867f7f5c7fa73cbb P 3637c09fbfb3492dbfd64b820edb64d1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.398134 12725 tablet_replica.cc:333] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1: stopping tablet replica
I20260709 13:03:21.398543 12725 raft_consensus.cc:2243] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.399139 12725 raft_consensus.cc:2272] T edff4ec626ca49ccadc3e871a4a9f7b9 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.400832 12725 tablet_replica.cc:333] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1: stopping tablet replica
I20260709 13:03:21.401243 12725 raft_consensus.cc:2243] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.401576 12725 raft_consensus.cc:2272] T 5a675310e5d64451be0e2037ae21c12d P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.403331 12725 tablet_replica.cc:333] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1: stopping tablet replica
I20260709 13:03:21.403735 12725 raft_consensus.cc:2243] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.404322 12725 raft_consensus.cc:2272] T 3a6fd117644347009d06ef121ef2ee5f P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.406201 12725 tablet_replica.cc:333] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1: stopping tablet replica
I20260709 13:03:21.406630 12725 raft_consensus.cc:2243] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.407007 12725 raft_consensus.cc:2272] T 8f5a6eb6aa46488c915db40e5af29f28 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.408689 12725 tablet_replica.cc:333] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1: stopping tablet replica
I20260709 13:03:21.409173 12725 raft_consensus.cc:2243] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.409554 12725 raft_consensus.cc:2272] T 377ae8d110aa4a12a45ebe1597cb1a44 P 3637c09fbfb3492dbfd64b820edb64d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.427423 12725 tablet_server.cc:196] TabletServer@127.12.109.67:0 shutdown complete.
I20260709 13:03:21.443440 12725 master.cc:562] Master@127.12.109.126:42611 shutting down...
I20260709 13:03:21.580875 12725 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.581527 12725 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.581897 12725 tablet_replica.cc:333] T 00000000000000000000000000000000 P 976606eb42f54514b87d1653ce4a6bc8: stopping tablet replica
I20260709 13:03:21.603721 12725 master.cc:584] Master@127.12.109.126:42611 shutdown complete.
2f97ad8ad4894210b3679e4f7f3ffa70  4
3637c09fbfb3492dbfd64b820edb64d1  1
be35d11a361b4ef8b3442d16fa47dadc  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12631 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:03:21.649443 12725 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.109.126:34033
I20260709 13:03:21.650923 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:21.657500 13233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:21.657519 13232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:21.658914 13235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:21.659840 12725 server_base.cc:1061] running on GCE node
I20260709 13:03:21.660856 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:21.661089 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:21.661258 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602201661241 us; error 0 us; skew 500 ppm
I20260709 13:03:21.661854 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:21.664844 12725 webserver.cc:533] Webserver started at http://127.12.109.126:46791/ using document root <none> and password file <none>
I20260709 13:03:21.665462 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:21.665683 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:21.665982 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:21.667418 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/master-0-root/instance:
uuid: "22d20bbb75a744f286e9c9ee28e45a9c"
format_stamp: "Formatted at 2026-07-09 13:03:21 on dist-test-slave-zhdc"
I20260709 13:03:21.672683 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:03:21.676715 13240 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:21.677750 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.001s
I20260709 13:03:21.678102 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/master-0-root
uuid: "22d20bbb75a744f286e9c9ee28e45a9c"
format_stamp: "Formatted at 2026-07-09 13:03:21 on dist-test-slave-zhdc"
I20260709 13:03:21.678395 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:21.704186 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:21.705628 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:21.751181 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.126:34033
I20260709 13:03:21.751271 13291 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.126:34033 every 8 connection(s)
I20260709 13:03:21.755679 13292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:21.766881 13292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c: Bootstrap starting.
I20260709 13:03:21.772497 13292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:21.777653 13292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c: No bootstrap required, opened a new log
I20260709 13:03:21.780330 13292 raft_consensus.cc:359] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d20bbb75a744f286e9c9ee28e45a9c" member_type: VOTER }
I20260709 13:03:21.780781 13292 raft_consensus.cc:385] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:21.781075 13292 raft_consensus.cc:740] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d20bbb75a744f286e9c9ee28e45a9c, State: Initialized, Role: FOLLOWER
I20260709 13:03:21.781735 13292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d20bbb75a744f286e9c9ee28e45a9c" member_type: VOTER }
I20260709 13:03:21.782239 13292 raft_consensus.cc:399] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:03:21.782481 13292 raft_consensus.cc:493] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:03:21.782763 13292 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:21.787573 13292 raft_consensus.cc:515] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d20bbb75a744f286e9c9ee28e45a9c" member_type: VOTER }
I20260709 13:03:21.788192 13292 leader_election.cc:304] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22d20bbb75a744f286e9c9ee28e45a9c; no voters: 
I20260709 13:03:21.789517 13292 leader_election.cc:290] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:03:21.789906 13295 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:21.791491 13295 raft_consensus.cc:697] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [term 1 LEADER]: Becoming Leader. State: Replica: 22d20bbb75a744f286e9c9ee28e45a9c, State: Running, Role: LEADER
I20260709 13:03:21.792235 13295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d20bbb75a744f286e9c9ee28e45a9c" member_type: VOTER }
I20260709 13:03:21.792886 13292 sys_catalog.cc:565] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:03:21.797485 13296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22d20bbb75a744f286e9c9ee28e45a9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d20bbb75a744f286e9c9ee28e45a9c" member_type: VOTER } }
I20260709 13:03:21.797611 13297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22d20bbb75a744f286e9c9ee28e45a9c. Latest consensus state: current_term: 1 leader_uuid: "22d20bbb75a744f286e9c9ee28e45a9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d20bbb75a744f286e9c9ee28e45a9c" member_type: VOTER } }
I20260709 13:03:21.798534 13296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:21.798594 13297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:21.802419 13301 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:03:21.811565 13301 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:03:21.812745 12725 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:03:21.821272 13301 catalog_manager.cc:1383] Generated new cluster ID: 9194dea34239499d9a7c046e96cde63e
I20260709 13:03:21.821612 13301 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:03:21.841935 13301 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:03:21.843557 13301 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:03:21.856845 13301 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c: Generated new TSK 0
I20260709 13:03:21.857599 13301 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:03:21.879782 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:21.886933 13313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:21.888322 13314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:21.891206 12725 server_base.cc:1061] running on GCE node
W20260709 13:03:21.891245 13316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:21.892614 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:21.892845 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:21.892972 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602201892959 us; error 0 us; skew 500 ppm
I20260709 13:03:21.893464 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:21.896366 12725 webserver.cc:533] Webserver started at http://127.12.109.65:38819/ using document root <none> and password file <none>
I20260709 13:03:21.896879 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:21.897046 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:21.897275 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:21.898535 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-0-root/instance:
uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
format_stamp: "Formatted at 2026-07-09 13:03:21 on dist-test-slave-zhdc"
I20260709 13:03:21.903817 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:03:21.907546 13321 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:21.908656 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:03:21.909104 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-0-root
uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
format_stamp: "Formatted at 2026-07-09 13:03:21 on dist-test-slave-zhdc"
I20260709 13:03:21.909409 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:21.926014 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:21.927701 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:21.929459 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:21.932358 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:21.932613 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:21.932911 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:21.933116 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:21.981146 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.65:36659
I20260709 13:03:21.981252 13383 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.65:36659 every 8 connection(s)
I20260709 13:03:21.987159 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:21.997274 13388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:21.997330 13389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:22.001583 13391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:22.002861 12725 server_base.cc:1061] running on GCE node
I20260709 13:03:22.004133 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:22.004396 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:22.004607 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602202004587 us; error 0 us; skew 500 ppm
I20260709 13:03:22.005371 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:22.008956 12725 webserver.cc:533] Webserver started at http://127.12.109.66:39427/ using document root <none> and password file <none>
I20260709 13:03:22.009214 13384 heartbeater.cc:344] Connected to a master server at 127.12.109.126:34033
I20260709 13:03:22.009666 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:22.009692 13384 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:22.010208 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:22.010622 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:22.011104 13384 heartbeater.cc:507] Master 127.12.109.126:34033 requested a full tablet report, sending...
I20260709 13:03:22.012557 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-1-root/instance:
uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
format_stamp: "Formatted at 2026-07-09 13:03:22 on dist-test-slave-zhdc"
I20260709 13:03:22.014086 13257 ts_manager.cc:194] Registered new tserver with Master: 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659)
I20260709 13:03:22.017082 13257 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41950
I20260709 13:03:22.020198 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.004s	sys 0.005s
I20260709 13:03:22.024497 13396 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:22.025537 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:03:22.025900 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-1-root
uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
format_stamp: "Formatted at 2026-07-09 13:03:22 on dist-test-slave-zhdc"
I20260709 13:03:22.026319 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:22.047006 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:22.048503 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:22.050217 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:22.052755 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:22.053071 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:22.053336 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:22.053512 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:22.099886 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.66:40873
I20260709 13:03:22.100028 13458 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.66:40873 every 8 connection(s)
I20260709 13:03:22.105165 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:22.112697 13462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:22.116590 13463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:22.120712 13459 heartbeater.cc:344] Connected to a master server at 127.12.109.126:34033
I20260709 13:03:22.121333 13459 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:22.122690 13459 heartbeater.cc:507] Master 127.12.109.126:34033 requested a full tablet report, sending...
W20260709 13:03:22.122877 13465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:22.123531 12725 server_base.cc:1061] running on GCE node
I20260709 13:03:22.124840 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:22.125192 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:22.125412 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602202125390 us; error 0 us; skew 500 ppm
I20260709 13:03:22.126240 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:22.126291 13257 ts_manager.cc:194] Registered new tserver with Master: b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:22.128414 13257 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41954
I20260709 13:03:22.129546 12725 webserver.cc:533] Webserver started at http://127.12.109.67:37415/ using document root <none> and password file <none>
I20260709 13:03:22.130252 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:22.130534 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:22.130901 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:22.133369 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-2-root/instance:
uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
format_stamp: "Formatted at 2026-07-09 13:03:22 on dist-test-slave-zhdc"
I20260709 13:03:22.138248 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:03:22.141871 13470 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:22.142692 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:03:22.143023 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-2-root
uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
format_stamp: "Formatted at 2026-07-09 13:03:22 on dist-test-slave-zhdc"
I20260709 13:03:22.143344 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602188851028-12725-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:22.155321 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:22.156780 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:22.158540 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:22.161010 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:22.161248 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:22.161504 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:22.161695 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:22.208364 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.67:38829
I20260709 13:03:22.208465 13532 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.67:38829 every 8 connection(s)
I20260709 13:03:22.224525 13533 heartbeater.cc:344] Connected to a master server at 127.12.109.126:34033
I20260709 13:03:22.224998 13533 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:22.225907 13533 heartbeater.cc:507] Master 127.12.109.126:34033 requested a full tablet report, sending...
I20260709 13:03:22.228303 13257 ts_manager.cc:194] Registered new tserver with Master: 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:22.229059 12725 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01703727s
I20260709 13:03:22.229897 13257 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41964
I20260709 13:03:22.258795 13257 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41968:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:03:22.262006 13257 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:03:22.348526 13498 tablet_service.cc:1511] Processing CreateTablet for tablet 6ff76a0fb4b540189ec875888456487d (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:03:22.350044 13495 tablet_service.cc:1511] Processing CreateTablet for tablet 52a2af2ac34f4687853ed638c08aa2ea (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:03:22.351668 13497 tablet_service.cc:1511] Processing CreateTablet for tablet d41a55f23c614145bfc2940f9c65974a (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:03:22.353641 13424 tablet_service.cc:1511] Processing CreateTablet for tablet 6ff76a0fb4b540189ec875888456487d (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:03:22.355329 13424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ff76a0fb4b540189ec875888456487d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.355242 13421 tablet_service.cc:1511] Processing CreateTablet for tablet 52a2af2ac34f4687853ed638c08aa2ea (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:03:22.355791 13423 tablet_service.cc:1511] Processing CreateTablet for tablet d41a55f23c614145bfc2940f9c65974a (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:03:22.356843 13421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52a2af2ac34f4687853ed638c08aa2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.358192 13422 tablet_service.cc:1511] Processing CreateTablet for tablet fe14de2a972c421baeca97a332ae0470 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:03:22.359603 13422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe14de2a972c421baeca97a332ae0470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.364946 13420 tablet_service.cc:1511] Processing CreateTablet for tablet 8edd59f2f96343fa8009f3b79a2c9e1c (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:03:22.366922 13423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d41a55f23c614145bfc2940f9c65974a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.367501 13419 tablet_service.cc:1511] Processing CreateTablet for tablet 2898be53627a492a80cdd85831887325 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:03:22.350366 13498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ff76a0fb4b540189ec875888456487d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.369791 13494 tablet_service.cc:1511] Processing CreateTablet for tablet 8edd59f2f96343fa8009f3b79a2c9e1c (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:03:22.351281 13496 tablet_service.cc:1511] Processing CreateTablet for tablet fe14de2a972c421baeca97a332ae0470 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:03:22.372875 13493 tablet_service.cc:1511] Processing CreateTablet for tablet 2898be53627a492a80cdd85831887325 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:03:22.375355 13492 tablet_service.cc:1511] Processing CreateTablet for tablet c0095d3e66c140d68b6b02999b2fe201 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:03:22.377329 13490 tablet_service.cc:1511] Processing CreateTablet for tablet d322d4b0355d4e8ead708eb5b89375b9 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:03:22.378022 13489 tablet_service.cc:1511] Processing CreateTablet for tablet b41974e5cf5d41d5819d2fdb1691b5d9 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:03:22.379827 13491 tablet_service.cc:1511] Processing CreateTablet for tablet 14a95014835046778206b3d34ce5dbd9 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:03:22.381852 13420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8edd59f2f96343fa8009f3b79a2c9e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.382685 13419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2898be53627a492a80cdd85831887325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.371467 13494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8edd59f2f96343fa8009f3b79a2c9e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.387955 13418 tablet_service.cc:1511] Processing CreateTablet for tablet c0095d3e66c140d68b6b02999b2fe201 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:03:22.389423 13418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0095d3e66c140d68b6b02999b2fe201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.388886 13417 tablet_service.cc:1511] Processing CreateTablet for tablet 14a95014835046778206b3d34ce5dbd9 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:03:22.390499 13491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14a95014835046778206b3d34ce5dbd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.391630 13416 tablet_service.cc:1511] Processing CreateTablet for tablet d322d4b0355d4e8ead708eb5b89375b9 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:03:22.393507 13415 tablet_service.cc:1511] Processing CreateTablet for tablet b41974e5cf5d41d5819d2fdb1691b5d9 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:03:22.394416 13490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d322d4b0355d4e8ead708eb5b89375b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.394901 13417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14a95014835046778206b3d34ce5dbd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.395169 13493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2898be53627a492a80cdd85831887325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.397763 13489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41974e5cf5d41d5819d2fdb1691b5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.399150 13415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41974e5cf5d41d5819d2fdb1691b5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.402424 13492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0095d3e66c140d68b6b02999b2fe201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.402948 13416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d322d4b0355d4e8ead708eb5b89375b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.414872 13496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe14de2a972c421baeca97a332ae0470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.426518 13495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52a2af2ac34f4687853ed638c08aa2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.430552 13497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d41a55f23c614145bfc2940f9c65974a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.472198 13349 tablet_service.cc:1511] Processing CreateTablet for tablet 6ff76a0fb4b540189ec875888456487d (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:03:22.480916 13347 tablet_service.cc:1511] Processing CreateTablet for tablet fe14de2a972c421baeca97a332ae0470 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:03:22.485190 13342 tablet_service.cc:1511] Processing CreateTablet for tablet 14a95014835046778206b3d34ce5dbd9 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:03:22.489347 13348 tablet_service.cc:1511] Processing CreateTablet for tablet d41a55f23c614145bfc2940f9c65974a (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:03:22.492427 13349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ff76a0fb4b540189ec875888456487d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.500702 13348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d41a55f23c614145bfc2940f9c65974a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.501281 13340 tablet_service.cc:1511] Processing CreateTablet for tablet b41974e5cf5d41d5819d2fdb1691b5d9 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:03:22.500731 13346 tablet_service.cc:1511] Processing CreateTablet for tablet 52a2af2ac34f4687853ed638c08aa2ea (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:03:22.503005 13340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41974e5cf5d41d5819d2fdb1691b5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.503432 13341 tablet_service.cc:1511] Processing CreateTablet for tablet d322d4b0355d4e8ead708eb5b89375b9 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:03:22.504506 13346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52a2af2ac34f4687853ed638c08aa2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.496112 13344 tablet_service.cc:1511] Processing CreateTablet for tablet 2898be53627a492a80cdd85831887325 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:03:22.496112 13345 tablet_service.cc:1511] Processing CreateTablet for tablet 8edd59f2f96343fa8009f3b79a2c9e1c (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:03:22.496114 13343 tablet_service.cc:1511] Processing CreateTablet for tablet c0095d3e66c140d68b6b02999b2fe201 (DEFAULT_TABLE table=test-workload [id=ba17c67db4ce4bdd8bca2d8a3b4dba44]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:03:22.517724 13341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d322d4b0355d4e8ead708eb5b89375b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.520848 13343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0095d3e66c140d68b6b02999b2fe201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.522396 13344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2898be53627a492a80cdd85831887325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.524253 13345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8edd59f2f96343fa8009f3b79a2c9e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.526072 13342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14a95014835046778206b3d34ce5dbd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.530501 13347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe14de2a972c421baeca97a332ae0470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.566478 13548 tablet_bootstrap.cc:492] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81: Bootstrap starting.
I20260709 13:03:22.590608 13549 tablet_bootstrap.cc:492] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b: Bootstrap starting.
I20260709 13:03:22.606952 13550 tablet_bootstrap.cc:492] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d: Bootstrap starting.
I20260709 13:03:22.608731 13549 tablet_bootstrap.cc:654] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.614233 13548 tablet_bootstrap.cc:654] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.624286 13549 tablet_bootstrap.cc:492] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b: No bootstrap required, opened a new log
I20260709 13:03:22.625010 13549 ts_tablet_manager.cc:1403] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent bootstrapping tablet: real 0.035s	user 0.003s	sys 0.009s
I20260709 13:03:22.628361 13550 tablet_bootstrap.cc:654] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.629129 13549 raft_consensus.cc:359] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.630237 13549 raft_consensus.cc:385] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.630631 13549 raft_consensus.cc:740] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.631587 13549 consensus_queue.cc:260] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.635715 13548 tablet_bootstrap.cc:492] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81: No bootstrap required, opened a new log
I20260709 13:03:22.636267 13548 ts_tablet_manager.cc:1403] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent bootstrapping tablet: real 0.070s	user 0.020s	sys 0.006s
I20260709 13:03:22.639525 13548 raft_consensus.cc:359] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.639948 13533 heartbeater.cc:499] Master 127.12.109.126:34033 was elected leader, sending a full tablet report...
I20260709 13:03:22.640432 13549 ts_tablet_manager.cc:1434] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent starting tablet: real 0.015s	user 0.017s	sys 0.000s
I20260709 13:03:22.641546 13550 tablet_bootstrap.cc:492] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d: No bootstrap required, opened a new log
I20260709 13:03:22.641947 13549 tablet_bootstrap.cc:492] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b: Bootstrap starting.
I20260709 13:03:22.642043 13550 ts_tablet_manager.cc:1403] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.006s
I20260709 13:03:22.643276 13548 raft_consensus.cc:385] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.643596 13548 raft_consensus.cc:740] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.644511 13548 consensus_queue.cc:260] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.646922 13459 heartbeater.cc:499] Master 127.12.109.126:34033 was elected leader, sending a full tablet report...
I20260709 13:03:22.647464 13548 ts_tablet_manager.cc:1434] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:03:22.648434 13548 tablet_bootstrap.cc:492] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81: Bootstrap starting.
I20260709 13:03:22.649119 13549 tablet_bootstrap.cc:654] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.655135 13548 tablet_bootstrap.cc:654] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.658248 13550 raft_consensus.cc:359] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.659137 13550 raft_consensus.cc:385] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.659457 13550 raft_consensus.cc:740] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.660820 13548 tablet_bootstrap.cc:492] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81: No bootstrap required, opened a new log
I20260709 13:03:22.660353 13550 consensus_queue.cc:260] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.661303 13548 ts_tablet_manager.cc:1403] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent bootstrapping tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 13:03:22.662832 13549 tablet_bootstrap.cc:492] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b: No bootstrap required, opened a new log
I20260709 13:03:22.663092 13384 heartbeater.cc:499] Master 127.12.109.126:34033 was elected leader, sending a full tablet report...
I20260709 13:03:22.663343 13549 ts_tablet_manager.cc:1403] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.004s
I20260709 13:03:22.664309 13548 raft_consensus.cc:359] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.664875 13548 raft_consensus.cc:385] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.665131 13548 raft_consensus.cc:740] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.665835 13548 consensus_queue.cc:260] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.668076 13550 ts_tablet_manager.cc:1434] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.003s
I20260709 13:03:22.666980 13549 raft_consensus.cc:359] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.668470 13549 raft_consensus.cc:385] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.668807 13549 raft_consensus.cc:740] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.669257 13550 tablet_bootstrap.cc:492] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d: Bootstrap starting.
I20260709 13:03:22.670668 13549 consensus_queue.cc:260] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.673117 13548 ts_tablet_manager.cc:1434] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:03:22.676563 13548 tablet_bootstrap.cc:492] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81: Bootstrap starting.
I20260709 13:03:22.677510 13550 tablet_bootstrap.cc:654] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.683244 13549 ts_tablet_manager.cc:1434] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.004s
I20260709 13:03:22.683434 13548 tablet_bootstrap.cc:654] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.684353 13549 tablet_bootstrap.cc:492] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b: Bootstrap starting.
I20260709 13:03:22.692155 13549 tablet_bootstrap.cc:654] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.697536 13548 tablet_bootstrap.cc:492] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81: No bootstrap required, opened a new log
I20260709 13:03:22.697734 13550 tablet_bootstrap.cc:492] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d: No bootstrap required, opened a new log
I20260709 13:03:22.698125 13548 ts_tablet_manager.cc:1403] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.009s
I20260709 13:03:22.698203 13550 ts_tablet_manager.cc:1403] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.004s
I20260709 13:03:22.701596 13550 raft_consensus.cc:359] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.701460 13548 raft_consensus.cc:359] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.702497 13548 raft_consensus.cc:385] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.702497 13550 raft_consensus.cc:385] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.702988 13548 raft_consensus.cc:740] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.703084 13550 raft_consensus.cc:740] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.704047 13548 consensus_queue.cc:260] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.706555 13548 ts_tablet_manager.cc:1434] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:03:22.707605 13548 tablet_bootstrap.cc:492] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81: Bootstrap starting.
I20260709 13:03:22.704113 13550 consensus_queue.cc:260] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.713397 13550 ts_tablet_manager.cc:1434] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:03:22.714468 13550 tablet_bootstrap.cc:492] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d: Bootstrap starting.
I20260709 13:03:22.714833 13548 tablet_bootstrap.cc:654] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.715917 13549 tablet_bootstrap.cc:492] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b: No bootstrap required, opened a new log
I20260709 13:03:22.716427 13549 ts_tablet_manager.cc:1403] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.011s
I20260709 13:03:22.719853 13549 raft_consensus.cc:359] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.720638 13549 raft_consensus.cc:385] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.720939 13549 raft_consensus.cc:740] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.721501 13550 tablet_bootstrap.cc:654] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.721804 13549 consensus_queue.cc:260] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.724833 13549 ts_tablet_manager.cc:1434] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:03:22.725860 13549 tablet_bootstrap.cc:492] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b: Bootstrap starting.
I20260709 13:03:22.730566 13548 tablet_bootstrap.cc:492] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81: No bootstrap required, opened a new log
I20260709 13:03:22.731137 13548 ts_tablet_manager.cc:1403] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.013s
I20260709 13:03:22.732797 13549 tablet_bootstrap.cc:654] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.734179 13548 raft_consensus.cc:359] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:22.734972 13548 raft_consensus.cc:385] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.735303 13548 raft_consensus.cc:740] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.736088 13548 consensus_queue.cc:260] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:22.738416 13548 ts_tablet_manager.cc:1434] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:03:22.739648 13548 tablet_bootstrap.cc:492] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81: Bootstrap starting.
I20260709 13:03:22.743250 13555 raft_consensus.cc:493] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.743842 13555 raft_consensus.cc:515] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.747959 13549 tablet_bootstrap.cc:492] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b: No bootstrap required, opened a new log
I20260709 13:03:22.748723 13549 ts_tablet_manager.cc:1403] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent bootstrapping tablet: real 0.023s	user 0.003s	sys 0.016s
I20260709 13:03:22.753227 13549 raft_consensus.cc:359] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.754278 13549 raft_consensus.cc:385] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.754668 13549 raft_consensus.cc:740] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.755873 13549 consensus_queue.cc:260] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.758610 13555 leader_election.cc:290] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:22.759538 13549 ts_tablet_manager.cc:1434] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.004s
I20260709 13:03:22.760385 13549 tablet_bootstrap.cc:492] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b: Bootstrap starting.
I20260709 13:03:22.765136 13550 tablet_bootstrap.cc:492] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d: No bootstrap required, opened a new log
I20260709 13:03:22.765789 13550 ts_tablet_manager.cc:1403] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent bootstrapping tablet: real 0.052s	user 0.005s	sys 0.029s
I20260709 13:03:22.766675 13549 tablet_bootstrap.cc:654] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.769398 13550 raft_consensus.cc:359] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.770665 13550 raft_consensus.cc:385] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.771140 13550 raft_consensus.cc:740] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.773242 13548 tablet_bootstrap.cc:654] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.772449 13550 consensus_queue.cc:260] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.775802 13550 ts_tablet_manager.cc:1434] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:03:22.777012 13550 tablet_bootstrap.cc:492] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d: Bootstrap starting.
I20260709 13:03:22.784475 13550 tablet_bootstrap.cc:654] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.791234 13554 raft_consensus.cc:493] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.792024 13554 raft_consensus.cc:515] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.802281 13554 leader_election.cc:290] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
W20260709 13:03:22.807016 13305 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:22.817369 13555 raft_consensus.cc:493] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.817982 13555 raft_consensus.cc:515] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.820885 13555 leader_election.cc:290] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:22.825629 13555 raft_consensus.cc:493] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.826452 13555 raft_consensus.cc:515] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.829704 13555 leader_election.cc:290] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:22.832839 13558 raft_consensus.cc:493] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.833618 13558 raft_consensus.cc:515] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.844003 13558 leader_election.cc:290] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:22.887877 13558 raft_consensus.cc:493] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.888679 13558 raft_consensus.cc:515] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.891848 13558 leader_election.cc:290] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:22.905057 13549 tablet_bootstrap.cc:492] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b: No bootstrap required, opened a new log
I20260709 13:03:22.905709 13549 ts_tablet_manager.cc:1403] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent bootstrapping tablet: real 0.145s	user 0.064s	sys 0.077s
I20260709 13:03:22.909170 13549 raft_consensus.cc:359] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.910043 13549 raft_consensus.cc:385] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.910424 13549 raft_consensus.cc:740] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.911494 13549 consensus_queue.cc:260] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.914211 13549 ts_tablet_manager.cc:1434] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.007s
I20260709 13:03:22.917196 13549 tablet_bootstrap.cc:492] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b: Bootstrap starting.
I20260709 13:03:22.920022 13554 raft_consensus.cc:493] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.920625 13554 raft_consensus.cc:515] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.922978 13554 leader_election.cc:290] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:22.927459 13549 tablet_bootstrap.cc:654] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:22.940131 13359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:22.940502 13358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:22.941390 13357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:22.941555 13358 raft_consensus.cc:2468] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 0.
I20260709 13:03:22.942744 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d322d4b0355d4e8ead708eb5b89375b9" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" is_pre_election: true
I20260709 13:03:22.943923 13433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" is_pre_election: true
I20260709 13:03:22.942869 13508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d322d4b0355d4e8ead708eb5b89375b9" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:22.945323 13433 raft_consensus.cc:2468] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 0.
I20260709 13:03:22.946089 13508 raft_consensus.cc:2468] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 0.
W20260709 13:03:22.946195 13325 leader_election.cc:343] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:22.942710 13357 raft_consensus.cc:2468] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 0.
I20260709 13:03:22.948010 13323 leader_election.cc:304] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: b876f4d43c9c4bc2b673f4ef507f2e81
I20260709 13:03:22.949379 13558 raft_consensus.cc:2804] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:22.949745 13558 raft_consensus.cc:493] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.950098 13558 raft_consensus.cc:3060] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:22.951280 13548 tablet_bootstrap.cc:492] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81: No bootstrap required, opened a new log
I20260709 13:03:22.950258 13325 leader_election.cc:304] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:22.952085 13548 ts_tablet_manager.cc:1403] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent bootstrapping tablet: real 0.213s	user 0.051s	sys 0.075s
W20260709 13:03:22.951253 13399 leader_election.cc:343] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:22.953971 13399 leader_election.cc:304] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:22.955384 13399 leader_election.cc:304] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:22.955509 13548 raft_consensus.cc:359] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.956331 13548 raft_consensus.cc:385] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.956655 13548 raft_consensus.cc:740] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.956846 13555 raft_consensus.cc:2804] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:22.957404 13555 raft_consensus.cc:493] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.957438 13548 consensus_queue.cc:260] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.947301 13508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:22.958024 13555 raft_consensus.cc:3060] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:22.961011 13578 raft_consensus.cc:2804] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:22.961655 13578 raft_consensus.cc:493] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.967998 13550 tablet_bootstrap.cc:492] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d: No bootstrap required, opened a new log
I20260709 13:03:22.968606 13550 ts_tablet_manager.cc:1403] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent bootstrapping tablet: real 0.192s	user 0.043s	sys 0.111s
I20260709 13:03:22.968901 13549 tablet_bootstrap.cc:492] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b: No bootstrap required, opened a new log
I20260709 13:03:22.969272 13578 raft_consensus.cc:3060] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:22.969517 13549 ts_tablet_manager.cc:1403] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.002s
I20260709 13:03:22.972378 13358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:22.973430 13550 raft_consensus.cc:359] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.959724 13508 raft_consensus.cc:2468] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 0.
I20260709 13:03:22.976493 13357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:22.978875 13550 raft_consensus.cc:385] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:22.979631 13507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:22.982550 13550 raft_consensus.cc:740] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Initialized, Role: FOLLOWER
I20260709 13:03:22.983366 13507 raft_consensus.cc:2468] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 0.
I20260709 13:03:22.983814 13550 consensus_queue.cc:260] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.985095 13398 leader_election.cc:304] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 5b3a6dbb1e52470ea290020cb1d60b2d
I20260709 13:03:22.978180 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
W20260709 13:03:22.989283 13473 leader_election.cc:343] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:22.986544 13357 raft_consensus.cc:2468] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 0.
I20260709 13:03:22.991279 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:22.997148 13433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" is_pre_election: true
I20260709 13:03:22.996903 13549 raft_consensus.cc:359] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:22.998421 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" is_pre_election: true
I20260709 13:03:22.989846 13506 raft_consensus.cc:2468] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 0.
I20260709 13:03:23.007591 13473 leader_election.cc:304] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 
I20260709 13:03:23.002696 13505 raft_consensus.cc:2468] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 0.
I20260709 13:03:23.005206 13558 raft_consensus.cc:515] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.005254 13549 raft_consensus.cc:385] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.009029 13548 ts_tablet_manager.cc:1434] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent starting tablet: real 0.057s	user 0.021s	sys 0.013s
I20260709 13:03:23.011525 13554 raft_consensus.cc:2804] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:23.013048 13549 raft_consensus.cc:740] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.015010 13558 leader_election.cc:290] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 election: Requested vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.015388 13550 ts_tablet_manager.cc:1434] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent starting tablet: real 0.046s	user 0.010s	sys 0.005s
W20260709 13:03:23.013536 13474 leader_election.cc:343] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:23.014850 13555 raft_consensus.cc:515] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.016535 13550 tablet_bootstrap.cc:492] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d: Bootstrap starting.
I20260709 13:03:23.013666 13554 raft_consensus.cc:493] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.017352 13554 raft_consensus.cc:3060] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.018625 13577 raft_consensus.cc:2804] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:23.018576 13581 raft_consensus.cc:493] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.019178 13577 raft_consensus.cc:493] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.020352 13577 raft_consensus.cc:3060] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.015937 13549 consensus_queue.cc:260] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.016593 13474 leader_election.cc:304] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81
I20260709 13:03:23.024011 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d322d4b0355d4e8ead708eb5b89375b9" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:23.025926 13548 tablet_bootstrap.cc:492] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81: Bootstrap starting.
I20260709 13:03:23.026033 13505 raft_consensus.cc:3060] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.037972 13550 tablet_bootstrap.cc:654] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.037806 13357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:23.038525 13357 raft_consensus.cc:3060] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.040278 13548 tablet_bootstrap.cc:654] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.041447 13433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d322d4b0355d4e8ead708eb5b89375b9" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:23.041925 13582 raft_consensus.cc:2804] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:23.042469 13582 raft_consensus.cc:493] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.042822 13582 raft_consensus.cc:3060] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:23.043499 13325 leader_election.cc:343] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:23.030568 13555 leader_election.cc:290] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.019311 13581 raft_consensus.cc:515] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:23.046499 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:23.047922 13506 raft_consensus.cc:3060] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.056797 13578 raft_consensus.cc:515] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.058439 13434 raft_consensus.cc:2393] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a2f043eb0c34e709f91d56f7adc5d2b in current term 1: Already voted for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in this term.
I20260709 13:03:23.062426 13584 raft_consensus.cc:2749] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:23.065742 13506 raft_consensus.cc:2468] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 1.
I20260709 13:03:23.066725 13357 raft_consensus.cc:2468] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 1.
I20260709 13:03:23.068305 13398 leader_election.cc:304] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:23.069592 13555 raft_consensus.cc:2804] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:23.070861 13581 leader_election.cc:290] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829), 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659)
I20260709 13:03:23.071949 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ff76a0fb4b540189ec875888456487d" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:23.072190 13358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ff76a0fb4b540189ec875888456487d" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:23.072302 13554 raft_consensus.cc:515] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
W20260709 13:03:23.073889 13398 leader_election.cc:343] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:23.070262 13555 raft_consensus.cc:697] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Becoming Leader. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
W20260709 13:03:23.075547 13399 leader_election.cc:343] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:23.076036 13399 leader_election.cc:304] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b
I20260709 13:03:23.078447 13505 raft_consensus.cc:2468] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 1.
I20260709 13:03:23.076010 13555 consensus_queue.cc:237] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.079377 13581 raft_consensus.cc:2749] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:23.079829 13323 leader_election.cc:304] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: b876f4d43c9c4bc2b673f4ef507f2e81
I20260709 13:03:23.059036 13582 raft_consensus.cc:515] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.083397 13583 raft_consensus.cc:2804] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:23.085378 13583 raft_consensus.cc:697] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Becoming Leader. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:23.083619 13357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:23.090353 13358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:23.086721 13583 consensus_queue.cc:237] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.091506 13550 tablet_bootstrap.cc:492] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d: No bootstrap required, opened a new log
W20260709 13:03:23.092141 13399 leader_election.cc:343] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:23.092183 13550 ts_tablet_manager.cc:1403] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent bootstrapping tablet: real 0.076s	user 0.016s	sys 0.017s
I20260709 13:03:23.094851 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:23.095702 13434 raft_consensus.cc:2393] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a2f043eb0c34e709f91d56f7adc5d2b in current term 1: Already voted for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in this term.
I20260709 13:03:23.099205 13582 leader_election.cc:290] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.095888 13554 leader_election.cc:290] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:23.099874 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:23.102634 13505 raft_consensus.cc:3060] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.104094 13549 ts_tablet_manager.cc:1434] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent starting tablet: real 0.134s	user 0.013s	sys 0.001s
I20260709 13:03:23.109551 13358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:23.109443 13254 catalog_manager.cc:5697] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65). New cstate: current_term: 1 leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.113695 13578 leader_election.cc:290] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.112737 13549 tablet_bootstrap.cc:492] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b: Bootstrap starting.
I20260709 13:03:23.114378 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:23.115247 13506 raft_consensus.cc:2393] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b876f4d43c9c4bc2b673f4ef507f2e81 in current term 1: Already voted for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in this term.
I20260709 13:03:23.115036 13550 raft_consensus.cc:359] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.115823 13550 raft_consensus.cc:385] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.116204 13550 raft_consensus.cc:740] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.106513 13577 raft_consensus.cc:515] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.116892 13398 leader_election.cc:304] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b
I20260709 13:03:23.117020 13550 consensus_queue.cc:260] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.120498 13578 raft_consensus.cc:2749] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:23.122551 13357 raft_consensus.cc:2393] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a2f043eb0c34e709f91d56f7adc5d2b in current term 1: Already voted for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in this term.
I20260709 13:03:23.125382 13550 ts_tablet_manager.cc:1434] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:03:23.126919 13473 leader_election.cc:304] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81
I20260709 13:03:23.127321 13550 tablet_bootstrap.cc:492] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d: Bootstrap starting.
I20260709 13:03:23.130177 13587 raft_consensus.cc:2749] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:23.129355 13505 raft_consensus.cc:2468] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 1.
I20260709 13:03:23.133337 13398 leader_election.cc:304] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 5b3a6dbb1e52470ea290020cb1d60b2d
I20260709 13:03:23.134258 13578 raft_consensus.cc:2804] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:23.134879 13578 raft_consensus.cc:697] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Becoming Leader. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:23.136857 13549 tablet_bootstrap.cc:654] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.136839 13578 consensus_queue.cc:237] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.137872 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:23.138708 13434 raft_consensus.cc:2393] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b3a6dbb1e52470ea290020cb1d60b2d in current term 1: Already voted for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in this term.
I20260709 13:03:23.139496 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:23.140323 13577 leader_election.cc:290] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 election: Requested vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.140913 13505 raft_consensus.cc:2393] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b3a6dbb1e52470ea290020cb1d60b2d in current term 1: Already voted for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in this term.
I20260709 13:03:23.142475 13323 leader_election.cc:304] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d; no voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81
I20260709 13:03:23.118270 13256 catalog_manager.cc:5697] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 reported cstate change: term changed from 0 to 1, leader changed from <none> to b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.144611 13577 raft_consensus.cc:2749] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:23.147424 13550 tablet_bootstrap.cc:654] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.147462 13548 tablet_bootstrap.cc:492] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81: No bootstrap required, opened a new log
I20260709 13:03:23.148182 13548 ts_tablet_manager.cc:1403] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent bootstrapping tablet: real 0.123s	user 0.022s	sys 0.004s
I20260709 13:03:23.152315 13548 raft_consensus.cc:359] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.153472 13548 raft_consensus.cc:385] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.153949 13548 raft_consensus.cc:740] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.155153 13548 consensus_queue.cc:260] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.158205 13548 ts_tablet_manager.cc:1434] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:03:23.159400 13548 tablet_bootstrap.cc:492] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81: Bootstrap starting.
I20260709 13:03:23.160079 13550 tablet_bootstrap.cc:492] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d: No bootstrap required, opened a new log
I20260709 13:03:23.160681 13550 ts_tablet_manager.cc:1403] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.007s
I20260709 13:03:23.165505 13578 raft_consensus.cc:493] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.166445 13578 raft_consensus.cc:515] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.167943 13548 tablet_bootstrap.cc:654] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.165337 13550 raft_consensus.cc:359] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.168481 13550 raft_consensus.cc:385] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.168969 13550 raft_consensus.cc:740] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.171741 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a95014835046778206b3d34ce5dbd9" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:23.169548 13587 raft_consensus.cc:493] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:23.173300 13398 leader_election.cc:343] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:23.173010 13587 raft_consensus.cc:515] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.171475 13550 consensus_queue.cc:260] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.174408 13357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a95014835046778206b3d34ce5dbd9" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:23.175113 13357 raft_consensus.cc:2468] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 0.
I20260709 13:03:23.175747 13550 ts_tablet_manager.cc:1434] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:03:23.176582 13587 leader_election.cc:290] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:23.176862 13550 tablet_bootstrap.cc:492] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d: Bootstrap starting.
I20260709 13:03:23.177507 13399 leader_election.cc:304] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 7a2f043eb0c34e709f91d56f7adc5d2b
I20260709 13:03:23.170512 13578 leader_election.cc:290] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.178886 13582 raft_consensus.cc:2804] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:23.178814 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41974e5cf5d41d5819d2fdb1691b5d9" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" is_pre_election: true
I20260709 13:03:23.179431 13582 raft_consensus.cc:493] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.179973 13582 raft_consensus.cc:3060] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:23.183732 13474 leader_election.cc:343] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:23.186599 13357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41974e5cf5d41d5819d2fdb1691b5d9" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:23.187498 13549 tablet_bootstrap.cc:492] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b: No bootstrap required, opened a new log
I20260709 13:03:23.188104 13548 tablet_bootstrap.cc:492] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81: No bootstrap required, opened a new log
I20260709 13:03:23.188107 13549 ts_tablet_manager.cc:1403] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent bootstrapping tablet: real 0.076s	user 0.009s	sys 0.008s
W20260709 13:03:23.188323 13473 leader_election.cc:343] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:23.188711 13548 ts_tablet_manager.cc:1403] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.007s
I20260709 13:03:23.188898 13473 leader_election.cc:304] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81
I20260709 13:03:23.189818 13587 raft_consensus.cc:2749] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:23.191673 13548 raft_consensus.cc:359] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.192441 13548 raft_consensus.cc:385] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.192723 13548 raft_consensus.cc:740] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.191664 13549 raft_consensus.cc:359] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.193217 13549 raft_consensus.cc:385] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.193157 13582 raft_consensus.cc:515] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.193650 13549 raft_consensus.cc:740] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.193545 13548 consensus_queue.cc:260] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.196259 13548 ts_tablet_manager.cc:1434] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:03:23.196868 13550 tablet_bootstrap.cc:654] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.197351 13357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a95014835046778206b3d34ce5dbd9" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:23.197979 13357 raft_consensus.cc:3060] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.198419 13582 leader_election.cc:290] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.199661 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a95014835046778206b3d34ce5dbd9" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:23.200064 13549 consensus_queue.cc:260] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
W20260709 13:03:23.201553 13398 leader_election.cc:343] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:23.203012 13549 ts_tablet_manager.cc:1434] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.004s
I20260709 13:03:23.204948 13254 catalog_manager.cc:5697] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 reported cstate change: term changed from 0 to 1, leader changed from <none> to b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.207906 13549 tablet_bootstrap.cc:492] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b: Bootstrap starting.
I20260709 13:03:23.208876 13357 raft_consensus.cc:2468] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 1.
I20260709 13:03:23.197319 13548 tablet_bootstrap.cc:492] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81: Bootstrap starting.
I20260709 13:03:23.210376 13399 leader_election.cc:304] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 7a2f043eb0c34e709f91d56f7adc5d2b
I20260709 13:03:23.211855 13582 raft_consensus.cc:2804] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:23.212395 13582 raft_consensus.cc:697] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Becoming Leader. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:23.213290 13582 consensus_queue.cc:237] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.213815 13550 tablet_bootstrap.cc:492] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d: No bootstrap required, opened a new log
I20260709 13:03:23.215301 13550 ts_tablet_manager.cc:1403] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.001s
I20260709 13:03:23.217525 13548 tablet_bootstrap.cc:654] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.224843 13548 tablet_bootstrap.cc:492] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81: No bootstrap required, opened a new log
I20260709 13:03:23.225373 13548 ts_tablet_manager.cc:1403] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent bootstrapping tablet: real 0.028s	user 0.007s	sys 0.004s
I20260709 13:03:23.227290 13549 tablet_bootstrap.cc:654] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.228462 13548 raft_consensus.cc:359] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.229230 13548 raft_consensus.cc:385] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.219121 13550 raft_consensus.cc:359] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.229564 13548 raft_consensus.cc:740] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.229928 13550 raft_consensus.cc:385] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.230331 13550 raft_consensus.cc:740] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.230434 13548 consensus_queue.cc:260] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.231268 13550 consensus_queue.cc:260] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.232646 13548 ts_tablet_manager.cc:1434] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:23.232596 13255 catalog_manager.cc:5697] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 reported cstate change: term changed from 0 to 1, leader changed from <none> to b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.233748 13548 tablet_bootstrap.cc:492] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81: Bootstrap starting.
I20260709 13:03:23.238117 13550 ts_tablet_manager.cc:1434] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.004s
I20260709 13:03:23.239308 13550 tablet_bootstrap.cc:492] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d: Bootstrap starting.
I20260709 13:03:23.240357 13548 tablet_bootstrap.cc:654] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.248662 13550 tablet_bootstrap.cc:654] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.251878 13548 tablet_bootstrap.cc:492] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81: No bootstrap required, opened a new log
I20260709 13:03:23.252458 13548 ts_tablet_manager.cc:1403] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent bootstrapping tablet: real 0.019s	user 0.004s	sys 0.012s
I20260709 13:03:23.254215 13549 tablet_bootstrap.cc:492] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b: No bootstrap required, opened a new log
I20260709 13:03:23.254743 13549 ts_tablet_manager.cc:1403] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent bootstrapping tablet: real 0.051s	user 0.010s	sys 0.006s
I20260709 13:03:23.255369 13548 raft_consensus.cc:359] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.256095 13548 raft_consensus.cc:385] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.256388 13548 raft_consensus.cc:740] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.257355 13548 consensus_queue.cc:260] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.259835 13548 ts_tablet_manager.cc:1434] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:03:23.260790 13548 tablet_bootstrap.cc:492] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81: Bootstrap starting.
I20260709 13:03:23.262912 13549 raft_consensus.cc:359] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.263865 13549 raft_consensus.cc:385] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.264230 13549 raft_consensus.cc:740] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.265420 13549 consensus_queue.cc:260] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.267330 13548 tablet_bootstrap.cc:654] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.274029 13587 raft_consensus.cc:493] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.274814 13587 raft_consensus.cc:515] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.278959 13357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a95014835046778206b3d34ce5dbd9" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:23.279112 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a95014835046778206b3d34ce5dbd9" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" is_pre_election: true
I20260709 13:03:23.279356 13548 tablet_bootstrap.cc:492] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81: No bootstrap required, opened a new log
I20260709 13:03:23.279903 13548 ts_tablet_manager.cc:1403] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 13:03:23.279855 13357 raft_consensus.cc:2393] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a2f043eb0c34e709f91d56f7adc5d2b in current term 1: Already voted for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in this term.
I20260709 13:03:23.281337 13587 leader_election.cc:290] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:23.281275 13473 leader_election.cc:304] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81
I20260709 13:03:23.282202 13587 raft_consensus.cc:3060] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.282972 13548 raft_consensus.cc:359] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.283667 13548 raft_consensus.cc:385] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.283931 13548 raft_consensus.cc:740] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.284068 13550 tablet_bootstrap.cc:492] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d: No bootstrap required, opened a new log
I20260709 13:03:23.284591 13550 ts_tablet_manager.cc:1403] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.016s
I20260709 13:03:23.284869 13548 consensus_queue.cc:260] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.285409 13549 ts_tablet_manager.cc:1434] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 13:03:23.286556 13549 tablet_bootstrap.cc:492] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b: Bootstrap starting.
I20260709 13:03:23.287541 13550 raft_consensus.cc:359] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.288240 13550 raft_consensus.cc:385] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.288525 13550 raft_consensus.cc:740] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.289347 13550 consensus_queue.cc:260] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.291455 13550 ts_tablet_manager.cc:1434] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:03:23.292414 13550 tablet_bootstrap.cc:492] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d: Bootstrap starting.
I20260709 13:03:23.294000 13549 tablet_bootstrap.cc:654] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.295197 13548 ts_tablet_manager.cc:1434] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:03:23.299577 13550 tablet_bootstrap.cc:654] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.303853 13582 raft_consensus.cc:493] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.304409 13582 raft_consensus.cc:515] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.306046 13549 tablet_bootstrap.cc:492] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b: No bootstrap required, opened a new log
I20260709 13:03:23.306628 13549 ts_tablet_manager.cc:1403] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 13:03:23.308604 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:23.309321 13505 raft_consensus.cc:2468] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 0.
I20260709 13:03:23.308625 13357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:23.309892 13549 raft_consensus.cc:359] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.310393 13550 tablet_bootstrap.cc:492] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d: No bootstrap required, opened a new log
I20260709 13:03:23.310722 13549 raft_consensus.cc:385] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.310797 13587 raft_consensus.cc:2749] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:23.311165 13550 ts_tablet_manager.cc:1403] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.003s
I20260709 13:03:23.311240 13549 raft_consensus.cc:740] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.311882 13398 leader_election.cc:304] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:23.312170 13549 consensus_queue.cc:260] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.312760 13578 raft_consensus.cc:2804] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:23.313176 13578 raft_consensus.cc:493] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.313529 13578 raft_consensus.cc:3060] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.314707 13549 ts_tablet_manager.cc:1434] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:03:23.314911 13582 leader_election.cc:290] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
W20260709 13:03:23.314612 13399 leader_election.cc:343] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:23.315469 13577 raft_consensus.cc:493] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.315863 13549 tablet_bootstrap.cc:492] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b: Bootstrap starting.
I20260709 13:03:23.316072 13577 raft_consensus.cc:515] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.320789 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41a55f23c614145bfc2940f9c65974a" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" is_pre_election: true
I20260709 13:03:23.321475 13434 raft_consensus.cc:2468] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 0.
I20260709 13:03:23.321838 13549 tablet_bootstrap.cc:654] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.321947 13578 raft_consensus.cc:515] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.322963 13325 leader_election.cc:304] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:23.323894 13583 raft_consensus.cc:2804] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:23.323734 13550 raft_consensus.cc:359] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:23.324335 13583 raft_consensus.cc:493] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.324590 13550 raft_consensus.cc:385] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.325057 13357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:23.325181 13550 raft_consensus.cc:740] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.325459 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41a55f23c614145bfc2940f9c65974a" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:23.326115 13505 raft_consensus.cc:2468] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 0.
W20260709 13:03:23.326723 13399 leader_election.cc:343] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:23.324723 13583 raft_consensus.cc:3060] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.326411 13550 consensus_queue.cc:260] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:23.328994 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:23.329608 13506 raft_consensus.cc:3060] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.329787 13550 ts_tablet_manager.cc:1434] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.003s
I20260709 13:03:23.330363 13577 leader_election.cc:290] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.331728 13550 tablet_bootstrap.cc:492] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d: Bootstrap starting.
I20260709 13:03:23.333842 13577 raft_consensus.cc:493] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.334348 13577 raft_consensus.cc:515] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.336619 13577 leader_election.cc:290] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.338588 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" is_pre_election: true
I20260709 13:03:23.340924 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:23.341153 13550 tablet_bootstrap.cc:654] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:23.341825 13505 raft_consensus.cc:2393] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b3a6dbb1e52470ea290020cb1d60b2d in current term 1: Already voted for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in this term.
I20260709 13:03:23.343158 13323 leader_election.cc:304] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d; no voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81
I20260709 13:03:23.344205 13506 raft_consensus.cc:2468] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 1.
I20260709 13:03:23.344902 13578 leader_election.cc:290] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.346351 13398 leader_election.cc:304] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 5b3a6dbb1e52470ea290020cb1d60b2d
I20260709 13:03:23.350338 13578 raft_consensus.cc:2804] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:23.353372 13578 raft_consensus.cc:697] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Becoming Leader. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:23.354444 13578 consensus_queue.cc:237] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.358268 13577 raft_consensus.cc:3060] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.372898 13550 tablet_bootstrap.cc:492] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d: No bootstrap required, opened a new log
I20260709 13:03:23.357479 13583 raft_consensus.cc:515] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.372409 13254 catalog_manager.cc:5697] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 reported cstate change: term changed from 0 to 1, leader changed from <none> to b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.373467 13550 ts_tablet_manager.cc:1403] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.023s
I20260709 13:03:23.376132 13583 leader_election.cc:290] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 election: Requested vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.379714 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41a55f23c614145bfc2940f9c65974a" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:23.380445 13506 raft_consensus.cc:3060] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.381796 13550 raft_consensus.cc:359] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.382589 13550 raft_consensus.cc:385] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.382891 13550 raft_consensus.cc:740] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.377487 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41a55f23c614145bfc2940f9c65974a" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:23.383881 13550 consensus_queue.cc:260] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.384204 13434 raft_consensus.cc:3060] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.383111 13577 raft_consensus.cc:2749] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:23.389828 13550 ts_tablet_manager.cc:1434] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:03:23.397457 13434 raft_consensus.cc:2468] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 1.
I20260709 13:03:23.398540 13506 raft_consensus.cc:2468] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 1.
I20260709 13:03:23.399250 13325 leader_election.cc:304] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:23.400285 13577 raft_consensus.cc:2804] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:23.400815 13577 raft_consensus.cc:697] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Becoming Leader. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:23.401386 13549 tablet_bootstrap.cc:492] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b: No bootstrap required, opened a new log
I20260709 13:03:23.401826 13549 ts_tablet_manager.cc:1403] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent bootstrapping tablet: real 0.086s	user 0.053s	sys 0.030s
I20260709 13:03:23.404808 13549 raft_consensus.cc:359] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:23.405344 13549 raft_consensus.cc:385] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:23.405640 13549 raft_consensus.cc:740] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Initialized, Role: FOLLOWER
I20260709 13:03:23.401697 13577 consensus_queue.cc:237] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.406347 13549 consensus_queue.cc:260] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:23.408799 13549 ts_tablet_manager.cc:1434] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:23.417173 13256 catalog_manager.cc:5697] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65). New cstate: current_term: 1 leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.483345 13578 raft_consensus.cc:493] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.483901 13578 raft_consensus.cc:515] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.485914 13578 leader_election.cc:290] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.487031 13357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41974e5cf5d41d5819d2fdb1691b5d9" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:23.487031 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41974e5cf5d41d5819d2fdb1691b5d9" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:23.488030 13577 raft_consensus.cc:493] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.488097 13506 raft_consensus.cc:2468] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 0.
I20260709 13:03:23.488030 13357 raft_consensus.cc:2468] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 0.
I20260709 13:03:23.488799 13577 raft_consensus.cc:515] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:23.489514 13398 leader_election.cc:304] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:23.490418 13578 raft_consensus.cc:2804] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:23.490757 13578 raft_consensus.cc:493] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.491096 13578 raft_consensus.cc:3060] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.491134 13577 leader_election.cc:290] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:23.492044 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ff76a0fb4b540189ec875888456487d" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:23.492343 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ff76a0fb4b540189ec875888456487d" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" is_pre_election: true
I20260709 13:03:23.492837 13506 raft_consensus.cc:2468] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 0.
I20260709 13:03:23.492995 13434 raft_consensus.cc:2468] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 0.
I20260709 13:03:23.493893 13323 leader_election.cc:304] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 
I20260709 13:03:23.494796 13577 raft_consensus.cc:2804] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:23.495191 13577 raft_consensus.cc:493] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.495558 13577 raft_consensus.cc:3060] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.499059 13578 raft_consensus.cc:515] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.501461 13578 leader_election.cc:290] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.502465 13357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41974e5cf5d41d5819d2fdb1691b5d9" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:23.502844 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41974e5cf5d41d5819d2fdb1691b5d9" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:23.503216 13357 raft_consensus.cc:3060] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.503158 13577 raft_consensus.cc:515] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:23.503530 13506 raft_consensus.cc:3060] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.505177 13577 leader_election.cc:290] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 election: Requested vote from peers 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:23.506289 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ff76a0fb4b540189ec875888456487d" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:23.506400 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ff76a0fb4b540189ec875888456487d" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:23.506847 13434 raft_consensus.cc:3060] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.507016 13505 raft_consensus.cc:3060] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.512538 13506 raft_consensus.cc:2468] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 1.
I20260709 13:03:23.512964 13434 raft_consensus.cc:2468] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 1.
I20260709 13:03:23.514670 13325 leader_election.cc:304] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:23.514813 13398 leader_election.cc:304] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:23.515813 13577 raft_consensus.cc:2804] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:23.516369 13577 raft_consensus.cc:697] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Becoming Leader. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:23.516513 13357 raft_consensus.cc:2468] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 1.
I20260709 13:03:23.515813 13578 raft_consensus.cc:2804] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:23.517278 13577 consensus_queue.cc:237] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:23.518313 13578 raft_consensus.cc:697] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Becoming Leader. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:23.519181 13578 consensus_queue.cc:237] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.520006 13505 raft_consensus.cc:2468] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 1.
I20260709 13:03:23.528751 13257 catalog_manager.cc:5697] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65). New cstate: current_term: 1 leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.528753 13256 catalog_manager.cc:5697] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 reported cstate change: term changed from 0 to 1, leader changed from <none> to b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.538362 13434 raft_consensus.cc:3060] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.545912 13577 consensus_queue.cc:1048] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:23.559106 13558 consensus_queue.cc:1048] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:23.560938 13578 raft_consensus.cc:493] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.561564 13578 raft_consensus.cc:515] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.566191 13558 raft_consensus.cc:493] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.566893 13582 consensus_queue.cc:1048] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:23.567562 13558 raft_consensus.cc:515] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.571839 13578 leader_election.cc:290] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.572542 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:23.573280 13506 raft_consensus.cc:2468] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 0.
I20260709 13:03:23.574007 13358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:23.574599 13398 leader_election.cc:304] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:23.575114 13358 raft_consensus.cc:2468] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 0.
I20260709 13:03:23.575654 13578 raft_consensus.cc:2804] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:23.576117 13578 raft_consensus.cc:493] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.577376 13578 raft_consensus.cc:3060] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.576835 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" is_pre_election: true
I20260709 13:03:23.578012 13558 leader_election.cc:290] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.579545 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" is_pre_election: true
I20260709 13:03:23.580504 13506 raft_consensus.cc:2468] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 0.
I20260709 13:03:23.582450 13323 leader_election.cc:304] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 
I20260709 13:03:23.583604 13558 raft_consensus.cc:2804] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:23.584008 13558 raft_consensus.cc:493] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.584367 13558 raft_consensus.cc:3060] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.586771 13578 raft_consensus.cc:515] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.590231 13434 raft_consensus.cc:2393] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b3a6dbb1e52470ea290020cb1d60b2d in current term 1: Already voted for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in this term.
I20260709 13:03:23.594828 13582 consensus_queue.cc:1048] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:23.596041 13558 raft_consensus.cc:515] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.598650 13558 leader_election.cc:290] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 election: Requested vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.599442 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:23.596599 13578 leader_election.cc:290] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:23.600325 13358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:23.596977 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:23.601150 13505 raft_consensus.cc:3060] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.601222 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:23.600338 13434 raft_consensus.cc:2393] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b3a6dbb1e52470ea290020cb1d60b2d in current term 1: Already voted for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in this term.
I20260709 13:03:23.606208 13325 leader_election.cc:304] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d; no voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81
I20260709 13:03:23.601236 13358 raft_consensus.cc:2393] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b876f4d43c9c4bc2b673f4ef507f2e81 in current term 1: Already voted for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in this term.
I20260709 13:03:23.608295 13558 raft_consensus.cc:2749] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:23.614356 13505 raft_consensus.cc:2468] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 1.
I20260709 13:03:23.615818 13398 leader_election.cc:304] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:23.616979 13620 consensus_queue.cc:1048] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:23.618927 13582 raft_consensus.cc:2804] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:23.639704 13582 raft_consensus.cc:697] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Becoming Leader. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:23.641206 13582 consensus_queue.cc:237] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:23.649416 13621 consensus_queue.cc:1048] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:23.667337 13256 catalog_manager.cc:5697] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 reported cstate change: term changed from 0 to 1, leader changed from <none> to b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.739763 13625 consensus_queue.cc:1048] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:23.740253 13357 raft_consensus.cc:3060] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:23.746558 13625 consensus_queue.cc:1048] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:23.750254 13621 consensus_queue.cc:1048] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:23.774416 13622 consensus_queue.cc:1048] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:23.811828 13305 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:23.860146 13558 consensus_queue.cc:1048] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:23.871742 13558 consensus_queue.cc:1048] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:23.926776 13622 consensus_queue.cc:1048] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:23.938927 13628 consensus_queue.cc:1048] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:23.940661 13558 consensus_queue.cc:1048] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:23.956311 13577 consensus_queue.cc:1048] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:24.087239 13621 consensus_queue.cc:1048] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:24.099753 13582 consensus_queue.cc:1048] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:24.723563 13642 raft_consensus.cc:493] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.724150 13642 raft_consensus.cc:515] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:24.726176 13642 leader_election.cc:290] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:24.727244 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" is_pre_election: true
I20260709 13:03:24.727355 13359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" is_pre_election: true
I20260709 13:03:24.727983 13434 raft_consensus.cc:2468] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 1.
I20260709 13:03:24.728113 13359 raft_consensus.cc:2468] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 1.
I20260709 13:03:24.729347 13473 leader_election.cc:304] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 
I20260709 13:03:24.730144 13642 raft_consensus.cc:2804] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:24.730504 13642 raft_consensus.cc:493] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.730744 13642 raft_consensus.cc:3060] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:24.738971 13642 raft_consensus.cc:515] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:24.740967 13642 leader_election.cc:290] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 2 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:24.742141 13359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:24.742525 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:24.742892 13359 raft_consensus.cc:3060] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:24.743098 13434 raft_consensus.cc:3060] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:24.751328 13359 raft_consensus.cc:2468] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 2.
I20260709 13:03:24.751328 13434 raft_consensus.cc:2468] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 2.
I20260709 13:03:24.753101 13473 leader_election.cc:304] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 
I20260709 13:03:24.753925 13642 raft_consensus.cc:2804] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:24.755050 13642 raft_consensus.cc:697] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 LEADER]: Becoming Leader. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Running, Role: LEADER
I20260709 13:03:24.756001 13642 consensus_queue.cc:237] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:24.764413 13257 catalog_manager.cc:5697] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b reported cstate change: term changed from 0 to 2, leader changed from <none> to 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67). New cstate: current_term: 2 leader_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:25.028182 13434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470"
dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
mode: GRACEFUL
new_leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:49230
I20260709 13:03:25.028825 13434 raft_consensus.cc:606] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Received request to transfer leadership to TS 5b3a6dbb1e52470ea290020cb1d60b2d
I20260709 13:03:25.043823 13505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea"
dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
mode: GRACEFUL
new_leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:53444
I20260709 13:03:25.044358 13505 raft_consensus.cc:606] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 LEADER]: Received request to transfer leadership to TS 5b3a6dbb1e52470ea290020cb1d60b2d
I20260709 13:03:25.049317 13434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c"
dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
mode: GRACEFUL
new_leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:49230
I20260709 13:03:25.049817 13434 raft_consensus.cc:606] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Received request to transfer leadership to TS 5b3a6dbb1e52470ea290020cb1d60b2d
I20260709 13:03:25.070757 13359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d322d4b0355d4e8ead708eb5b89375b9"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
mode: GRACEFUL
new_leader_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
 from {username='slave'} at 127.0.0.1:53256
I20260709 13:03:25.071419 13359 raft_consensus.cc:606] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Received request to transfer leadership to TS b876f4d43c9c4bc2b673f4ef507f2e81
I20260709 13:03:25.076968 13434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b41974e5cf5d41d5819d2fdb1691b5d9"
dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
mode: GRACEFUL
new_leader_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
 from {username='slave'} at 127.0.0.1:49230
I20260709 13:03:25.077545 13434 raft_consensus.cc:606] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Received request to transfer leadership to TS 7a2f043eb0c34e709f91d56f7adc5d2b
I20260709 13:03:25.119113 13582 raft_consensus.cc:993] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81: : Instructing follower 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:25.119714 13653 raft_consensus.cc:1081] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Signalling peer 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:25.121402 13358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:53248
I20260709 13:03:25.122011 13358 raft_consensus.cc:493] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:25.122474 13358 raft_consensus.cc:3060] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:25.130467 13582 raft_consensus.cc:993] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81: : Instructing follower 7a2f043eb0c34e709f91d56f7adc5d2b to start an election
I20260709 13:03:25.130146 13358 raft_consensus.cc:515] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.131052 13653 raft_consensus.cc:1081] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Signalling peer 7a2f043eb0c34e709f91d56f7adc5d2b to start an election
I20260709 13:03:25.132351 13358 leader_election.cc:290] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 2 election: Requested vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:25.132797 13505 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b41974e5cf5d41d5819d2fdb1691b5d9"
dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
 from {username='slave'} at 127.0.0.1:53426
I20260709 13:03:25.133147 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:25.133392 13505 raft_consensus.cc:493] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:25.133502 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:25.133795 13434 raft_consensus.cc:3055] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:25.134158 13506 raft_consensus.cc:3060] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:25.134128 13434 raft_consensus.cc:740] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:25.135176 13434 consensus_queue.cc:260] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.134364 13505 raft_consensus.cc:3060] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:25.142093 13434 raft_consensus.cc:3060] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:25.143900 13506 raft_consensus.cc:2468] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 2.
I20260709 13:03:25.145572 13323 leader_election.cc:304] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 
I20260709 13:03:25.146708 13558 raft_consensus.cc:2804] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:25.147228 13558 raft_consensus.cc:697] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Becoming Leader. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:25.148128 13558 consensus_queue.cc:237] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.149353 13505 raft_consensus.cc:515] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.152802 13358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41974e5cf5d41d5819d2fdb1691b5d9" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:25.153005 13433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41974e5cf5d41d5819d2fdb1691b5d9" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:25.153496 13358 raft_consensus.cc:3060] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:25.153613 13433 raft_consensus.cc:3055] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:25.153945 13433 raft_consensus.cc:740] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:25.154907 13433 consensus_queue.cc:260] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.156088 13433 raft_consensus.cc:3060] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:25.156738 13434 raft_consensus.cc:2468] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 2.
I20260709 13:03:25.158552 13505 leader_election.cc:290] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 2 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:25.164068 13358 raft_consensus.cc:2468] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 2.
I20260709 13:03:25.165462 13473 leader_election.cc:304] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 
I20260709 13:03:25.165342 13257 catalog_manager.cc:5697] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d reported cstate change: term changed from 1 to 2, leader changed from b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66) to 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65). New cstate: current_term: 2 leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:25.167575 13642 raft_consensus.cc:2804] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:25.168071 13642 raft_consensus.cc:697] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 LEADER]: Becoming Leader. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Running, Role: LEADER
I20260709 13:03:25.169140 13642 consensus_queue.cc:237] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.179055 13433 raft_consensus.cc:2468] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 2.
I20260709 13:03:25.178411 13256 catalog_manager.cc:5697] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b reported cstate change: term changed from 1 to 2, leader changed from b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66) to 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67). New cstate: current_term: 2 leader_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:25.277078 13582 raft_consensus.cc:993] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81: : Instructing follower 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:25.277622 13653 raft_consensus.cc:1081] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Signalling peer 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:25.279286 13358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:53248
I20260709 13:03:25.279914 13358 raft_consensus.cc:493] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:25.280320 13358 raft_consensus.cc:3060] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:25.286670 13358 raft_consensus.cc:515] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.288904 13358 leader_election.cc:290] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 2 election: Requested vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:25.290045 13433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:25.290227 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:25.290797 13505 raft_consensus.cc:3060] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:25.290757 13433 raft_consensus.cc:3055] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:25.291347 13433 raft_consensus.cc:740] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:25.292268 13433 consensus_queue.cc:260] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.293262 13433 raft_consensus.cc:3060] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:25.296880 13505 raft_consensus.cc:2468] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 2.
I20260709 13:03:25.298395 13323 leader_election.cc:304] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 
I20260709 13:03:25.298880 13433 raft_consensus.cc:2468] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 2.
I20260709 13:03:25.299208 13558 raft_consensus.cc:2804] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:25.299748 13558 raft_consensus.cc:697] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Becoming Leader. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:25.300679 13558 consensus_queue.cc:237] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.308022 13256 catalog_manager.cc:5697] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d reported cstate change: term changed from 1 to 2, leader changed from b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66) to 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65). New cstate: current_term: 2 leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:25.370649 13358 raft_consensus.cc:1275] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Refusing update from remote peer 7a2f043eb0c34e709f91d56f7adc5d2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:25.371253 13433 raft_consensus.cc:1275] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Refusing update from remote peer 7a2f043eb0c34e709f91d56f7adc5d2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:25.372092 13642 consensus_queue.cc:1048] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:25.372783 13643 consensus_queue.cc:1048] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:25.394903 13642 raft_consensus.cc:993] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b: : Instructing follower 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:25.395619 13642 raft_consensus.cc:1081] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 LEADER]: Signalling peer 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:25.402899 13359 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:53252
I20260709 13:03:25.404878 13359 raft_consensus.cc:493] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:25.405282 13359 raft_consensus.cc:3060] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:25.410674 13359 raft_consensus.cc:515] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.414197 13359 leader_election.cc:290] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 3 election: Requested vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:25.415210 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:25.415270 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52a2af2ac34f4687853ed638c08aa2ea" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:25.416024 13434 raft_consensus.cc:3060] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:25.416088 13505 raft_consensus.cc:3055] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:25.416481 13505 raft_consensus.cc:740] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Running, Role: LEADER
I20260709 13:03:25.417361 13505 consensus_queue.cc:260] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.418535 13505 raft_consensus.cc:3060] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:25.422698 13434 raft_consensus.cc:2468] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 3.
I20260709 13:03:25.423906 13325 leader_election.cc:304] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:25.424760 13558 raft_consensus.cc:2804] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:25.425258 13558 raft_consensus.cc:697] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 LEADER]: Becoming Leader. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:25.425398 13505 raft_consensus.cc:2468] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 3.
I20260709 13:03:25.426275 13558 consensus_queue.cc:237] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.433243 13257 catalog_manager.cc:5697] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d reported cstate change: term changed from 2 to 3, leader changed from 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67) to 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65). New cstate: current_term: 3 leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:25.448850 13577 raft_consensus.cc:993] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d: : Instructing follower b876f4d43c9c4bc2b673f4ef507f2e81 to start an election
I20260709 13:03:25.449396 13558 raft_consensus.cc:1081] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Signalling peer b876f4d43c9c4bc2b673f4ef507f2e81 to start an election
I20260709 13:03:25.450894 13434 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d322d4b0355d4e8ead708eb5b89375b9"
dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
 from {username='slave'} at 127.0.0.1:49220
I20260709 13:03:25.451414 13434 raft_consensus.cc:493] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:25.451741 13434 raft_consensus.cc:3060] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:25.457149 13434 raft_consensus.cc:515] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.458937 13434 leader_election.cc:290] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 2 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:25.459900 13359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d322d4b0355d4e8ead708eb5b89375b9" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:25.460117 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d322d4b0355d4e8ead708eb5b89375b9" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:25.460593 13359 raft_consensus.cc:3055] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:25.460680 13505 raft_consensus.cc:3060] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:25.460916 13359 raft_consensus.cc:740] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:25.461774 13359 consensus_queue.cc:260] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.462845 13359 raft_consensus.cc:3060] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:25.466252 13505 raft_consensus.cc:2468] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 2.
I20260709 13:03:25.467396 13398 leader_election.cc:304] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:25.468117 13359 raft_consensus.cc:2468] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 2.
I20260709 13:03:25.468286 13653 raft_consensus.cc:2804] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:25.468837 13653 raft_consensus.cc:697] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 LEADER]: Becoming Leader. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:25.469661 13653 consensus_queue.cc:237] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:25.477874 13257 catalog_manager.cc:5697] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 reported cstate change: term changed from 1 to 2, leader changed from 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65) to b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:25.593961 13505 raft_consensus.cc:1275] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:25.595502 13558 consensus_queue.cc:1048] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:25.611423 13434 raft_consensus.cc:1275] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:25.613353 13433 raft_consensus.cc:1275] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Refusing update from remote peer 7a2f043eb0c34e709f91d56f7adc5d2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:25.613199 13558 consensus_queue.cc:1048] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:25.615131 13642 consensus_queue.cc:1048] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:25.628110 13359 raft_consensus.cc:1275] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Refusing update from remote peer 7a2f043eb0c34e709f91d56f7adc5d2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:25.629892 13660 consensus_queue.cc:1048] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:25.720641 13434 raft_consensus.cc:1275] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:25.722289 13558 consensus_queue.cc:1048] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:25.732368 13505 raft_consensus.cc:1275] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:25.734563 13577 consensus_queue.cc:1048] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:25.838644 13505 raft_consensus.cc:1275] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:03:25.840291 13577 consensus_queue.cc:1048] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:25.849368 13434 raft_consensus.cc:1275] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:03:25.851145 13577 consensus_queue.cc:1048] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:25.891755 13505 raft_consensus.cc:1275] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Refusing update from remote peer b876f4d43c9c4bc2b673f4ef507f2e81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:25.893298 13582 consensus_queue.cc:1048] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:25.903316 13359 raft_consensus.cc:1275] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Refusing update from remote peer b876f4d43c9c4bc2b673f4ef507f2e81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:25.905912 13673 consensus_queue.cc:1048] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:28.112340 12725 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:03:28.113358 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:28.139523 13359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
mode: GRACEFUL
new_leader_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
 from {username='slave'} at 127.0.0.1:37212
I20260709 13:03:28.140223 13359 raft_consensus.cc:606] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Received request to transfer leadership to TS 7a2f043eb0c34e709f91d56f7adc5d2b
I20260709 13:03:28.143059 13359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
mode: GRACEFUL
new_leader_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
 from {username='slave'} at 127.0.0.1:37212
I20260709 13:03:28.143659 13359 raft_consensus.cc:606] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Received request to transfer leadership to TS 7a2f043eb0c34e709f91d56f7adc5d2b
I20260709 13:03:28.144884 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:03:28.145495 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:28.179605 13558 raft_consensus.cc:993] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d: : Instructing follower 7a2f043eb0c34e709f91d56f7adc5d2b to start an election
I20260709 13:03:28.180186 13680 raft_consensus.cc:1081] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Signalling peer 7a2f043eb0c34e709f91d56f7adc5d2b to start an election
I20260709 13:03:28.182149 13505 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470"
dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
 from {username='slave'} at 127.0.0.1:53434
I20260709 13:03:28.182703 13505 raft_consensus.cc:493] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:28.183005 13505 raft_consensus.cc:3060] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:28.187948 13505 raft_consensus.cc:515] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:28.189661 13505 leader_election.cc:290] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 3 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:28.190614 13358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:28.190891 13433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:28.191181 13358 raft_consensus.cc:3055] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:28.191449 13433 raft_consensus.cc:3060] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:28.191541 13358 raft_consensus.cc:740] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:28.192427 13358 consensus_queue.cc:260] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:28.193424 13358 raft_consensus.cc:3060] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:28.197084 13433 raft_consensus.cc:2468] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 3.
I20260709 13:03:28.198436 13474 leader_election.cc:304] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:28.198987 13358 raft_consensus.cc:2468] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 3.
I20260709 13:03:28.199322 13660 raft_consensus.cc:2804] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:28.200736 13660 raft_consensus.cc:697] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 LEADER]: Becoming Leader. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Running, Role: LEADER
I20260709 13:03:28.201493 13660 consensus_queue.cc:237] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:28.209133 13257 catalog_manager.cc:5697] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b reported cstate change: term changed from 2 to 3, leader changed from 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65) to 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67). New cstate: current_term: 3 leader_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:28.454762 13558 raft_consensus.cc:993] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d: : Instructing follower 7a2f043eb0c34e709f91d56f7adc5d2b to start an election
I20260709 13:03:28.455400 13680 raft_consensus.cc:1081] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Signalling peer 7a2f043eb0c34e709f91d56f7adc5d2b to start an election
I20260709 13:03:28.457552 13505 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c"
dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
 from {username='slave'} at 127.0.0.1:53434
I20260709 13:03:28.458235 13505 raft_consensus.cc:493] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:28.458572 13505 raft_consensus.cc:3060] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:28.464939 13505 raft_consensus.cc:515] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:28.466985 13505 leader_election.cc:290] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 3 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:28.468945 13433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:28.468945 13358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:28.469888 13433 raft_consensus.cc:3060] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:28.469925 13358 raft_consensus.cc:3055] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:28.470544 13358 raft_consensus.cc:740] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:28.471607 13358 consensus_queue.cc:260] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:28.473263 13358 raft_consensus.cc:3060] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:28.478251 13433 raft_consensus.cc:2468] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 3.
I20260709 13:03:28.480189 13474 leader_election.cc:304] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:28.481289 13660 raft_consensus.cc:2804] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:28.481251 13358 raft_consensus.cc:2468] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 3.
I20260709 13:03:28.482156 13660 raft_consensus.cc:697] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 LEADER]: Becoming Leader. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Running, Role: LEADER
I20260709 13:03:28.483209 13660 consensus_queue.cc:237] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:28.493224 13257 catalog_manager.cc:5697] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b reported cstate change: term changed from 2 to 3, leader changed from 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65) to 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67). New cstate: current_term: 3 leader_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:28.595311 13433 raft_consensus.cc:1275] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Refusing update from remote peer 7a2f043eb0c34e709f91d56f7adc5d2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:28.596837 13660 consensus_queue.cc:1048] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:28.607219 13358 raft_consensus.cc:1275] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Refusing update from remote peer 7a2f043eb0c34e709f91d56f7adc5d2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:28.608934 13660 consensus_queue.cc:1048] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:28.876106 13433 raft_consensus.cc:1275] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Refusing update from remote peer 7a2f043eb0c34e709f91d56f7adc5d2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:28.877666 13660 consensus_queue.cc:1048] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:28.888338 13358 raft_consensus.cc:1275] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Refusing update from remote peer 7a2f043eb0c34e709f91d56f7adc5d2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:28.895205 13695 consensus_queue.cc:1048] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:29.147208 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:29.170167 13505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470"
dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
mode: GRACEFUL
new_leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:53444
I20260709 13:03:29.171170 13505 raft_consensus.cc:606] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 LEADER]: Received request to transfer leadership to TS 5b3a6dbb1e52470ea290020cb1d60b2d
I20260709 13:03:29.180076 13505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c"
dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
mode: GRACEFUL
new_leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:53444
I20260709 13:03:29.180769 13505 raft_consensus.cc:606] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 LEADER]: Received request to transfer leadership to TS 5b3a6dbb1e52470ea290020cb1d60b2d
I20260709 13:03:29.186483 13433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2898be53627a492a80cdd85831887325"
dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
mode: GRACEFUL
new_leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:49230
I20260709 13:03:29.187069 13433 raft_consensus.cc:606] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Received request to transfer leadership to TS 5b3a6dbb1e52470ea290020cb1d60b2d
I20260709 13:03:29.192843 13433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201"
dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
mode: GRACEFUL
new_leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:49230
I20260709 13:03:29.193430 13433 raft_consensus.cc:606] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Received request to transfer leadership to TS 5b3a6dbb1e52470ea290020cb1d60b2d
I20260709 13:03:29.199168 13433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14a95014835046778206b3d34ce5dbd9"
dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
mode: GRACEFUL
new_leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:49230
I20260709 13:03:29.200089 13433 raft_consensus.cc:606] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Received request to transfer leadership to TS 5b3a6dbb1e52470ea290020cb1d60b2d
I20260709 13:03:29.315399 13660 raft_consensus.cc:993] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b: : Instructing follower 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:29.316118 13695 raft_consensus.cc:1081] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 LEADER]: Signalling peer 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:29.317884 13358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:53252
I20260709 13:03:29.318646 13358 raft_consensus.cc:493] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:29.318986 13358 raft_consensus.cc:3060] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.326824 13358 raft_consensus.cc:515] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.329066 13358 leader_election.cc:290] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 4 election: Requested vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:29.330036 13433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:29.330441 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8edd59f2f96343fa8009f3b79a2c9e1c" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:29.331156 13505 raft_consensus.cc:3055] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:29.331182 13433 raft_consensus.cc:3060] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.331601 13505 raft_consensus.cc:740] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Running, Role: LEADER
I20260709 13:03:29.332571 13505 consensus_queue.cc:260] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.334144 13505 raft_consensus.cc:3060] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.340379 13433 raft_consensus.cc:2468] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 4.
I20260709 13:03:29.341974 13325 leader_election.cc:304] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:29.342275 13505 raft_consensus.cc:2468] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 4.
I20260709 13:03:29.343257 13680 raft_consensus.cc:2804] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:29.344470 13680 raft_consensus.cc:697] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 4 LEADER]: Becoming Leader. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:29.345606 13680 consensus_queue.cc:237] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.353400 13254 catalog_manager.cc:5697] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d reported cstate change: term changed from 3 to 4, leader changed from 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67) to 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65). New cstate: current_term: 4 leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:29.474749 13688 raft_consensus.cc:993] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81: : Instructing follower 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:29.475366 13702 raft_consensus.cc:1081] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Signalling peer 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:29.477334 13358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:53248
I20260709 13:03:29.477944 13358 raft_consensus.cc:493] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:29.478353 13358 raft_consensus.cc:3060] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:29.484403 13358 raft_consensus.cc:515] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.486357 13358 leader_election.cc:290] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 2 election: Requested vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:29.487505 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:29.487506 13433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:29.488346 13505 raft_consensus.cc:3060] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:29.488379 13433 raft_consensus.cc:3055] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:29.488906 13433 raft_consensus.cc:740] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:29.489897 13433 consensus_queue.cc:260] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.491420 13433 raft_consensus.cc:3060] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:29.496917 13505 raft_consensus.cc:2468] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 2.
I20260709 13:03:29.498416 13323 leader_election.cc:304] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 
I20260709 13:03:29.499313 13680 raft_consensus.cc:2804] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:29.499894 13680 raft_consensus.cc:697] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Becoming Leader. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:29.500102 13433 raft_consensus.cc:2468] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 2.
I20260709 13:03:29.501051 13680 consensus_queue.cc:237] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.509279 13254 catalog_manager.cc:5697] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d reported cstate change: term changed from 1 to 2, leader changed from b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66) to 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65). New cstate: current_term: 2 leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:29.533406 13688 raft_consensus.cc:993] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81: : Instructing follower 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:29.534014 13702 raft_consensus.cc:1081] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Signalling peer 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:29.535657 13358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2898be53627a492a80cdd85831887325"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:53248
I20260709 13:03:29.536114 13358 raft_consensus.cc:493] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:29.536381 13358 raft_consensus.cc:3060] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:29.539994 13660 raft_consensus.cc:993] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b: : Instructing follower 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:29.540501 13695 raft_consensus.cc:1081] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 LEADER]: Signalling peer 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:29.542480 13359 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:53252
I20260709 13:03:29.543100 13359 raft_consensus.cc:493] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:29.543433 13359 raft_consensus.cc:3060] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.543304 13358 raft_consensus.cc:515] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.545630 13358 leader_election.cc:290] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 2 election: Requested vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:29.546798 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:29.547104 13688 raft_consensus.cc:993] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81: : Instructing follower 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:29.547446 13505 raft_consensus.cc:3060] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:29.547606 13702 raft_consensus.cc:1081] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Signalling peer 5b3a6dbb1e52470ea290020cb1d60b2d to start an election
I20260709 13:03:29.547482 13433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:29.548344 13433 raft_consensus.cc:3055] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:29.549176 13433 raft_consensus.cc:740] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:29.550184 13433 consensus_queue.cc:260] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.551661 13433 raft_consensus.cc:3060] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:29.554207 13505 raft_consensus.cc:2468] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 2.
I20260709 13:03:29.554279 13359 raft_consensus.cc:515] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.554957 13358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14a95014835046778206b3d34ce5dbd9"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
 from {username='slave'} at 127.0.0.1:53248
I20260709 13:03:29.555867 13358 raft_consensus.cc:493] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:29.556258 13358 raft_consensus.cc:3060] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:29.556416 13323 leader_election.cc:304] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 
I20260709 13:03:29.557438 13680 raft_consensus.cc:2804] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:29.557891 13680 raft_consensus.cc:697] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Becoming Leader. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:29.558718 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:29.559378 13434 raft_consensus.cc:3060] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.559394 13433 raft_consensus.cc:2468] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 2.
I20260709 13:03:29.559221 13680 consensus_queue.cc:237] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.564699 13358 raft_consensus.cc:515] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.567302 13358 leader_election.cc:290] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 2 election: Requested vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:29.568818 13505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a95014835046778206b3d34ce5dbd9" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:29.569619 13505 raft_consensus.cc:3060] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:29.568780 13433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a95014835046778206b3d34ce5dbd9" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:29.569283 13254 catalog_manager.cc:5697] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d reported cstate change: term changed from 1 to 2, leader changed from b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66) to 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65). New cstate: current_term: 2 leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:29.570312 13433 raft_consensus.cc:3055] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:29.570408 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe14de2a972c421baeca97a332ae0470" candidate_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:29.570776 13433 raft_consensus.cc:740] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:29.571076 13506 raft_consensus.cc:3055] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:29.571380 13506 raft_consensus.cc:740] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Running, Role: LEADER
I20260709 13:03:29.571765 13433 consensus_queue.cc:260] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.572397 13506 consensus_queue.cc:260] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.573601 13506 raft_consensus.cc:3060] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.573189 13433 raft_consensus.cc:3060] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:29.576900 13505 raft_consensus.cc:2468] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 2.
I20260709 13:03:29.578236 13323 leader_election.cc:304] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 
I20260709 13:03:29.580865 13698 raft_consensus.cc:2804] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:29.581382 13698 raft_consensus.cc:697] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Becoming Leader. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:29.581905 13433 raft_consensus.cc:2468] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 2.
I20260709 13:03:29.582350 13506 raft_consensus.cc:2468] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 4.
I20260709 13:03:29.583518 13323 leader_election.cc:304] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3a6dbb1e52470ea290020cb1d60b2d, 7a2f043eb0c34e709f91d56f7adc5d2b; no voters: 
I20260709 13:03:29.583854 13698 consensus_queue.cc:237] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.585337 13359 leader_election.cc:290] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [CANDIDATE]: Term 4 election: Requested vote from peers b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:29.585817 13705 raft_consensus.cc:2804] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:29.586374 13705 raft_consensus.cc:697] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 4 LEADER]: Becoming Leader. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:29.567618 13434 raft_consensus.cc:2468] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3a6dbb1e52470ea290020cb1d60b2d in term 4.
I20260709 13:03:29.587353 13705 consensus_queue.cc:237] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:29.600646 13254 catalog_manager.cc:5697] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d reported cstate change: term changed from 1 to 2, leader changed from b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66) to 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65). New cstate: current_term: 2 leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:29.613885 13254 catalog_manager.cc:5697] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d reported cstate change: term changed from 3 to 4, leader changed from 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67) to 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65). New cstate: current_term: 4 leader_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:29.724689 13434 raft_consensus.cc:1275] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 4 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:29.726202 13705 consensus_queue.cc:1048] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:29.736416 13506 raft_consensus.cc:1275] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 4 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:29.738211 13706 consensus_queue.cc:1048] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:29.900192 13434 raft_consensus.cc:1275] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:29.901679 13706 consensus_queue.cc:1048] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:29.912287 13506 raft_consensus.cc:1275] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:29.914163 13698 consensus_queue.cc:1048] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:30.009305 13434 raft_consensus.cc:1275] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:30.011046 13698 consensus_queue.cc:1048] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:30.021570 13506 raft_consensus.cc:1275] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:30.023763 13698 consensus_queue.cc:1048] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:30.155304 13506 raft_consensus.cc:1275] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:30.156050 13434 raft_consensus.cc:1275] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:30.156869 13705 consensus_queue.cc:1048] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:30.157546 13706 consensus_queue.cc:1048] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:30.175125 13506 raft_consensus.cc:1275] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 4 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:30.176934 13680 consensus_queue.cc:1048] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:30.187896 13432 raft_consensus.cc:1275] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 4 FOLLOWER]: Refusing update from remote peer 5b3a6dbb1e52470ea290020cb1d60b2d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:30.189752 13698 consensus_queue.cc:1048] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:32.209939 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:32.218961 13359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2898be53627a492a80cdd85831887325"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
mode: GRACEFUL
new_leader_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
 from {username='slave'} at 127.0.0.1:37212
I20260709 13:03:32.219616 13359 raft_consensus.cc:606] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Received request to transfer leadership to TS b876f4d43c9c4bc2b673f4ef507f2e81
I20260709 13:03:32.222662 13359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ff76a0fb4b540189ec875888456487d"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
mode: GRACEFUL
new_leader_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
 from {username='slave'} at 127.0.0.1:37212
I20260709 13:03:32.223276 13359 raft_consensus.cc:606] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Received request to transfer leadership to TS b876f4d43c9c4bc2b673f4ef507f2e81
I20260709 13:03:32.225754 13359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
mode: GRACEFUL
new_leader_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
 from {username='slave'} at 127.0.0.1:37212
I20260709 13:03:32.226387 13359 raft_consensus.cc:606] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Received request to transfer leadership to TS 7a2f043eb0c34e709f91d56f7adc5d2b
I20260709 13:03:32.228760 13359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d41a55f23c614145bfc2940f9c65974a"
dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
mode: GRACEFUL
new_leader_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
 from {username='slave'} at 127.0.0.1:37212
I20260709 13:03:32.229242 13359 raft_consensus.cc:606] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Received request to transfer leadership to TS 7a2f043eb0c34e709f91d56f7adc5d2b
I20260709 13:03:32.230613 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:03:32.231264 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:32.284582 13720 raft_consensus.cc:993] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d: : Instructing follower b876f4d43c9c4bc2b673f4ef507f2e81 to start an election
I20260709 13:03:32.285115 13721 raft_consensus.cc:1081] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Signalling peer b876f4d43c9c4bc2b673f4ef507f2e81 to start an election
I20260709 13:03:32.286744 13434 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2898be53627a492a80cdd85831887325"
dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
 from {username='slave'} at 127.0.0.1:49220
I20260709 13:03:32.287180 13434 raft_consensus.cc:493] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:32.287406 13434 raft_consensus.cc:3060] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:32.292167 13434 raft_consensus.cc:515] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:32.293771 13434 leader_election.cc:290] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 3 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829)
I20260709 13:03:32.295017 13359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:32.295183 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2898be53627a492a80cdd85831887325" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:32.295593 13359 raft_consensus.cc:3055] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:32.295769 13506 raft_consensus.cc:3060] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:32.295898 13359 raft_consensus.cc:740] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:32.296731 13359 consensus_queue.cc:260] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:32.297739 13359 raft_consensus.cc:3060] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:32.301610 13506 raft_consensus.cc:2468] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 3.
I20260709 13:03:32.303244 13359 raft_consensus.cc:2468] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 3.
I20260709 13:03:32.303189 13398 leader_election.cc:304] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:32.304224 13719 raft_consensus.cc:2804] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:32.305145 13719 raft_consensus.cc:697] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 LEADER]: Becoming Leader. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:32.305994 13719 consensus_queue.cc:237] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:32.313421 13254 catalog_manager.cc:5697] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 reported cstate change: term changed from 2 to 3, leader changed from 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65) to b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66). New cstate: current_term: 3 leader_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:32.449184 13720 raft_consensus.cc:993] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d: : Instructing follower 7a2f043eb0c34e709f91d56f7adc5d2b to start an election
I20260709 13:03:32.449733 13721 raft_consensus.cc:1081] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Signalling peer 7a2f043eb0c34e709f91d56f7adc5d2b to start an election
I20260709 13:03:32.451435 13506 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d41a55f23c614145bfc2940f9c65974a"
dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
 from {username='slave'} at 127.0.0.1:53434
I20260709 13:03:32.452009 13506 raft_consensus.cc:493] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:32.452343 13506 raft_consensus.cc:3060] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:32.457710 13506 raft_consensus.cc:515] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:32.459733 13506 leader_election.cc:290] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 2 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:32.460893 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41a55f23c614145bfc2940f9c65974a" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:32.460738 13359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41a55f23c614145bfc2940f9c65974a" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:32.461436 13434 raft_consensus.cc:3060] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:32.461532 13359 raft_consensus.cc:3055] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:32.461921 13359 raft_consensus.cc:740] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:32.462792 13359 consensus_queue.cc:260] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:32.463979 13359 raft_consensus.cc:3060] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:32.466730 13434 raft_consensus.cc:2468] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 2.
I20260709 13:03:32.468087 13474 leader_election.cc:304] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:32.469082 13695 raft_consensus.cc:2804] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:32.470435 13695 raft_consensus.cc:697] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 LEADER]: Becoming Leader. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Running, Role: LEADER
I20260709 13:03:32.470580 13359 raft_consensus.cc:2468] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 2.
I20260709 13:03:32.471417 13695 consensus_queue.cc:237] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:32.478719 13254 catalog_manager.cc:5697] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b reported cstate change: term changed from 1 to 2, leader changed from 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65) to 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67). New cstate: current_term: 2 leader_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:32.663503 13720 raft_consensus.cc:993] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d: : Instructing follower b876f4d43c9c4bc2b673f4ef507f2e81 to start an election
I20260709 13:03:32.664110 13720 raft_consensus.cc:1081] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Signalling peer b876f4d43c9c4bc2b673f4ef507f2e81 to start an election
I20260709 13:03:32.665871 13434 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ff76a0fb4b540189ec875888456487d"
dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
 from {username='slave'} at 127.0.0.1:49220
I20260709 13:03:32.666455 13434 raft_consensus.cc:493] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:32.666923 13434 raft_consensus.cc:3060] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:32.672137 13434 raft_consensus.cc:515] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:32.673933 13434 leader_election.cc:290] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 2 election: Requested vote from peers 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67:38829), 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659)
I20260709 13:03:32.675206 13359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ff76a0fb4b540189ec875888456487d" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:32.675244 13506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ff76a0fb4b540189ec875888456487d" candidate_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
I20260709 13:03:32.675851 13359 raft_consensus.cc:3055] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:32.675968 13506 raft_consensus.cc:3060] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:32.676138 13359 raft_consensus.cc:740] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:32.676985 13359 consensus_queue.cc:260] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:32.678013 13359 raft_consensus.cc:3060] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:32.681998 13506 raft_consensus.cc:2468] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 2.
I20260709 13:03:32.683251 13398 leader_election.cc:304] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:32.684149 13719 raft_consensus.cc:2804] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:32.684437 13359 raft_consensus.cc:2468] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b876f4d43c9c4bc2b673f4ef507f2e81 in term 2.
I20260709 13:03:32.684649 13719 raft_consensus.cc:697] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 LEADER]: Becoming Leader. State: Replica: b876f4d43c9c4bc2b673f4ef507f2e81, State: Running, Role: LEADER
I20260709 13:03:32.685707 13719 consensus_queue.cc:237] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } }
I20260709 13:03:32.696597 13254 catalog_manager.cc:5697] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 reported cstate change: term changed from 1 to 2, leader changed from 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65) to b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:32.731325 13721 raft_consensus.cc:993] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d: : Instructing follower 7a2f043eb0c34e709f91d56f7adc5d2b to start an election
I20260709 13:03:32.731868 13720 raft_consensus.cc:1081] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Signalling peer 7a2f043eb0c34e709f91d56f7adc5d2b to start an election
I20260709 13:03:32.733652 13506 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201"
dest_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b"
 from {username='slave'} at 127.0.0.1:53434
I20260709 13:03:32.734198 13506 raft_consensus.cc:493] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:32.734473 13506 raft_consensus.cc:3060] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:32.739463 13506 raft_consensus.cc:515] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:32.741268 13506 leader_election.cc:290] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 3 election: Requested vote from peers 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659), b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873)
I20260709 13:03:32.742261 13359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d"
I20260709 13:03:32.742622 13434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0095d3e66c140d68b6b02999b2fe201" candidate_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81"
I20260709 13:03:32.742892 13359 raft_consensus.cc:3055] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:32.743216 13434 raft_consensus.cc:3060] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:32.743320 13359 raft_consensus.cc:740] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b3a6dbb1e52470ea290020cb1d60b2d, State: Running, Role: LEADER
I20260709 13:03:32.744205 13359 consensus_queue.cc:260] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:32.745334 13359 raft_consensus.cc:3060] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:32.749364 13434 raft_consensus.cc:2468] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 3.
I20260709 13:03:32.750677 13474 leader_election.cc:304] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2f043eb0c34e709f91d56f7adc5d2b, b876f4d43c9c4bc2b673f4ef507f2e81; no voters: 
I20260709 13:03:32.751591 13695 raft_consensus.cc:2804] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:32.752094 13695 raft_consensus.cc:697] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 LEADER]: Becoming Leader. State: Replica: 7a2f043eb0c34e709f91d56f7adc5d2b, State: Running, Role: LEADER
I20260709 13:03:32.752933 13359 raft_consensus.cc:2468] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2f043eb0c34e709f91d56f7adc5d2b in term 3.
I20260709 13:03:32.752980 13695 consensus_queue.cc:237] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } }
I20260709 13:03:32.761278 13254 catalog_manager.cc:5697] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b reported cstate change: term changed from 2 to 3, leader changed from 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65) to 7a2f043eb0c34e709f91d56f7adc5d2b (127.12.109.67). New cstate: current_term: 3 leader_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:32.823115 13505 raft_consensus.cc:1275] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Refusing update from remote peer b876f4d43c9c4bc2b673f4ef507f2e81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:32.824608 13719 consensus_queue.cc:1048] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:32.834803 13359 raft_consensus.cc:1275] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Refusing update from remote peer b876f4d43c9c4bc2b673f4ef507f2e81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:32.836910 13719 consensus_queue.cc:1048] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:32.868584 13434 raft_consensus.cc:1275] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Refusing update from remote peer 7a2f043eb0c34e709f91d56f7adc5d2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:32.869794 13695 consensus_queue.cc:1048] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:32.881956 13359 raft_consensus.cc:1275] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Refusing update from remote peer 7a2f043eb0c34e709f91d56f7adc5d2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:32.883935 13739 consensus_queue.cc:1048] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:33.164381 13359 raft_consensus.cc:1275] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Refusing update from remote peer b876f4d43c9c4bc2b673f4ef507f2e81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:33.165830 13719 consensus_queue.cc:1048] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3a6dbb1e52470ea290020cb1d60b2d" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:33.176097 13505 raft_consensus.cc:1275] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Refusing update from remote peer b876f4d43c9c4bc2b673f4ef507f2e81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:33.177883 13722 consensus_queue.cc:1048] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2f043eb0c34e709f91d56f7adc5d2b" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:33.232342 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:33.240849 12725 tablet_server.cc:179] TabletServer@127.12.109.65:0 shutting down...
I20260709 13:03:33.272696 13434 raft_consensus.cc:1275] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Refusing update from remote peer 7a2f043eb0c34e709f91d56f7adc5d2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:33.274494 13739 consensus_queue.cc:1048] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876f4d43c9c4bc2b673f4ef507f2e81" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:33.279771 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:33.280892 12725 tablet_replica.cc:333] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d: stopping tablet replica
I20260709 13:03:33.281885 12725 raft_consensus.cc:2243] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.282971 12725 raft_consensus.cc:2272] T fe14de2a972c421baeca97a332ae0470 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:33.287642 13473 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
I20260709 13:03:33.291335 12725 tablet_replica.cc:333] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d: stopping tablet replica
W20260709 13:03:33.291708 13473 consensus_peers.cc:597] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b -> Peer 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659): Couldn't send request to peer 5b3a6dbb1e52470ea290020cb1d60b2d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.292397 12725 raft_consensus.cc:2243] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.293042 12725 raft_consensus.cc:2272] T 6ff76a0fb4b540189ec875888456487d P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.296119 12725 tablet_replica.cc:333] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d: stopping tablet replica
I20260709 13:03:33.296978 12725 raft_consensus.cc:2243] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.297418 12725 raft_consensus.cc:2272] T c0095d3e66c140d68b6b02999b2fe201 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.299817 12725 tablet_replica.cc:333] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d: stopping tablet replica
I20260709 13:03:33.300372 12725 raft_consensus.cc:2243] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.300819 12725 raft_consensus.cc:2272] T b41974e5cf5d41d5819d2fdb1691b5d9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.302722 12725 tablet_replica.cc:333] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d: stopping tablet replica
I20260709 13:03:33.303303 12725 raft_consensus.cc:2243] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.303736 12725 raft_consensus.cc:2272] T d41a55f23c614145bfc2940f9c65974a P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.305912 12725 tablet_replica.cc:333] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d: stopping tablet replica
I20260709 13:03:33.306528 12725 raft_consensus.cc:2243] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.306972 12725 raft_consensus.cc:2272] T d322d4b0355d4e8ead708eb5b89375b9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.308815 12725 tablet_replica.cc:333] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d: stopping tablet replica
I20260709 13:03:33.309340 12725 raft_consensus.cc:2243] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.309769 12725 raft_consensus.cc:2272] T 2898be53627a492a80cdd85831887325 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.312021 12725 tablet_replica.cc:333] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d: stopping tablet replica
I20260709 13:03:33.312556 12725 raft_consensus.cc:2243] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.313376 12725 raft_consensus.cc:2272] T 8edd59f2f96343fa8009f3b79a2c9e1c P 5b3a6dbb1e52470ea290020cb1d60b2d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.315392 12725 tablet_replica.cc:333] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d: stopping tablet replica
I20260709 13:03:33.315920 12725 raft_consensus.cc:2243] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.316733 12725 raft_consensus.cc:2272] T 14a95014835046778206b3d34ce5dbd9 P 5b3a6dbb1e52470ea290020cb1d60b2d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.318502 12725 tablet_replica.cc:333] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d: stopping tablet replica
I20260709 13:03:33.318989 12725 raft_consensus.cc:2243] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.319787 12725 raft_consensus.cc:2272] T 52a2af2ac34f4687853ed638c08aa2ea P 5b3a6dbb1e52470ea290020cb1d60b2d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.342794 12725 tablet_server.cc:196] TabletServer@127.12.109.65:0 shutdown complete.
I20260709 13:03:33.368963 12725 tablet_server.cc:179] TabletServer@127.12.109.66:0 shutting down...
W20260709 13:03:33.382354 13474 consensus_peers.cc:597] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b -> Peer b876f4d43c9c4bc2b673f4ef507f2e81 (127.12.109.66:40873): Couldn't send request to peer b876f4d43c9c4bc2b673f4ef507f2e81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.403743 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:33.404646 12725 tablet_replica.cc:333] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81: stopping tablet replica
I20260709 13:03:33.405471 12725 raft_consensus.cc:2243] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.406103 12725 raft_consensus.cc:2272] T b41974e5cf5d41d5819d2fdb1691b5d9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.408174 12725 tablet_replica.cc:333] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81: stopping tablet replica
I20260709 13:03:33.408721 12725 raft_consensus.cc:2243] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.409189 12725 raft_consensus.cc:2272] T d41a55f23c614145bfc2940f9c65974a P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.412271 12725 tablet_replica.cc:333] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81: stopping tablet replica
I20260709 13:03:33.412952 12725 raft_consensus.cc:2243] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.413889 12725 raft_consensus.cc:2272] T d322d4b0355d4e8ead708eb5b89375b9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.415961 12725 tablet_replica.cc:333] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81: stopping tablet replica
I20260709 13:03:33.416541 12725 raft_consensus.cc:2243] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.417287 12725 raft_consensus.cc:2272] T 2898be53627a492a80cdd85831887325 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.421116 12725 tablet_replica.cc:333] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81: stopping tablet replica
I20260709 13:03:33.421626 12725 raft_consensus.cc:2243] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.422016 12725 raft_consensus.cc:2272] T fe14de2a972c421baeca97a332ae0470 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.423804 12725 tablet_replica.cc:333] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81: stopping tablet replica
I20260709 13:03:33.424260 12725 raft_consensus.cc:2243] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.424618 12725 raft_consensus.cc:2272] T c0095d3e66c140d68b6b02999b2fe201 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.426826 12725 tablet_replica.cc:333] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81: stopping tablet replica
I20260709 13:03:33.427649 12725 raft_consensus.cc:2243] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.428534 12725 raft_consensus.cc:2272] T 6ff76a0fb4b540189ec875888456487d P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.430757 12725 tablet_replica.cc:333] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81: stopping tablet replica
I20260709 13:03:33.431247 12725 raft_consensus.cc:2243] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.431705 12725 raft_consensus.cc:2272] T 8edd59f2f96343fa8009f3b79a2c9e1c P b876f4d43c9c4bc2b673f4ef507f2e81 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.433431 12725 tablet_replica.cc:333] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81: stopping tablet replica
I20260709 13:03:33.433926 12725 raft_consensus.cc:2243] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.434372 12725 raft_consensus.cc:2272] T 14a95014835046778206b3d34ce5dbd9 P b876f4d43c9c4bc2b673f4ef507f2e81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.435978 12725 tablet_replica.cc:333] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81: stopping tablet replica
I20260709 13:03:33.436439 12725 raft_consensus.cc:2243] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.436841 12725 raft_consensus.cc:2272] T 52a2af2ac34f4687853ed638c08aa2ea P b876f4d43c9c4bc2b673f4ef507f2e81 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.466589 12725 tablet_server.cc:196] TabletServer@127.12.109.66:0 shutdown complete.
W20260709 13:03:33.488339 13473 consensus_peers.cc:597] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b -> Peer 5b3a6dbb1e52470ea290020cb1d60b2d (127.12.109.65:36659): Couldn't send request to peer 5b3a6dbb1e52470ea290020cb1d60b2d. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:36659: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.491719 12725 tablet_server.cc:179] TabletServer@127.12.109.67:0 shutting down...
I20260709 13:03:33.518872 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:33.519753 12725 tablet_replica.cc:333] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b: stopping tablet replica
I20260709 13:03:33.520752 12725 raft_consensus.cc:2243] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.521406 12725 raft_consensus.cc:2272] T 6ff76a0fb4b540189ec875888456487d P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.524121 12725 tablet_replica.cc:333] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b: stopping tablet replica
I20260709 13:03:33.524698 12725 raft_consensus.cc:2243] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.525571 12725 raft_consensus.cc:2272] T d41a55f23c614145bfc2940f9c65974a P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.527693 12725 tablet_replica.cc:333] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b: stopping tablet replica
I20260709 13:03:33.528198 12725 raft_consensus.cc:2243] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.528573 12725 raft_consensus.cc:2272] T 14a95014835046778206b3d34ce5dbd9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.530287 12725 tablet_replica.cc:333] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b: stopping tablet replica
I20260709 13:03:33.530740 12725 raft_consensus.cc:2243] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.531095 12725 raft_consensus.cc:2272] T fe14de2a972c421baeca97a332ae0470 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.532683 12725 tablet_replica.cc:333] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b: stopping tablet replica
I20260709 13:03:33.533277 12725 raft_consensus.cc:2243] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.534036 12725 raft_consensus.cc:2272] T b41974e5cf5d41d5819d2fdb1691b5d9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.535742 12725 tablet_replica.cc:333] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b: stopping tablet replica
I20260709 13:03:33.536207 12725 raft_consensus.cc:2243] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.536906 12725 raft_consensus.cc:2272] T c0095d3e66c140d68b6b02999b2fe201 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.539062 12725 tablet_replica.cc:333] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b: stopping tablet replica
I20260709 13:03:33.539512 12725 raft_consensus.cc:2243] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.539860 12725 raft_consensus.cc:2272] T 52a2af2ac34f4687853ed638c08aa2ea P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.541422 12725 tablet_replica.cc:333] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b: stopping tablet replica
I20260709 13:03:33.541973 12725 raft_consensus.cc:2243] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.542479 12725 raft_consensus.cc:2272] T 2898be53627a492a80cdd85831887325 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.544478 12725 tablet_replica.cc:333] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b: stopping tablet replica
I20260709 13:03:33.544994 12725 raft_consensus.cc:2243] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.545557 12725 raft_consensus.cc:2272] T d322d4b0355d4e8ead708eb5b89375b9 P 7a2f043eb0c34e709f91d56f7adc5d2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.547293 12725 tablet_replica.cc:333] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b: stopping tablet replica
I20260709 13:03:33.547736 12725 raft_consensus.cc:2243] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.548127 12725 raft_consensus.cc:2272] T 8edd59f2f96343fa8009f3b79a2c9e1c P 7a2f043eb0c34e709f91d56f7adc5d2b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.569818 12725 tablet_server.cc:196] TabletServer@127.12.109.67:0 shutdown complete.
I20260709 13:03:33.591740 12725 master.cc:562] Master@127.12.109.126:34033 shutting down...
I20260709 13:03:33.617270 12725 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.617838 12725 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.618235 12725 tablet_replica.cc:333] T 00000000000000000000000000000000 P 22d20bbb75a744f286e9c9ee28e45a9c: stopping tablet replica
I20260709 13:03:33.639191 12725 master.cc:584] Master@127.12.109.126:34033 shutdown complete.
5b3a6dbb1e52470ea290020cb1d60b2d  5
7a2f043eb0c34e709f91d56f7adc5d2b  1
b876f4d43c9c4bc2b673f4ef507f2e81  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (12039 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:03:33.688851 12725 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.109.126:43315
I20260709 13:03:33.690349 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:33.697049 13748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:33.697235 13749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:33.697901 13751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:33.700421 12725 server_base.cc:1061] running on GCE node
I20260709 13:03:33.701417 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:33.701637 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:33.701794 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602213701777 us; error 0 us; skew 500 ppm
I20260709 13:03:33.702533 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:33.705456 12725 webserver.cc:533] Webserver started at http://127.12.109.126:35905/ using document root <none> and password file <none>
I20260709 13:03:33.706000 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:33.706259 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:33.706575 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:33.707857 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/master-0-root/instance:
uuid: "91c63734839347c4aee340a59c8e3221"
format_stamp: "Formatted at 2026-07-09 13:03:33 on dist-test-slave-zhdc"
I20260709 13:03:33.712819 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:03:33.716677 13756 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:33.717514 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:03:33.717831 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/master-0-root
uuid: "91c63734839347c4aee340a59c8e3221"
format_stamp: "Formatted at 2026-07-09 13:03:33 on dist-test-slave-zhdc"
I20260709 13:03:33.718204 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:33.733650 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:33.735208 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:33.779572 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.126:43315
I20260709 13:03:33.779696 13807 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.126:43315 every 8 connection(s)
I20260709 13:03:33.784328 13808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:33.795853 13808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Bootstrap starting.
I20260709 13:03:33.801383 13808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.806332 13808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: No bootstrap required, opened a new log
I20260709 13:03:33.808787 13808 raft_consensus.cc:359] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c63734839347c4aee340a59c8e3221" member_type: VOTER }
I20260709 13:03:33.809268 13808 raft_consensus.cc:385] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.809521 13808 raft_consensus.cc:740] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c63734839347c4aee340a59c8e3221, State: Initialized, Role: FOLLOWER
I20260709 13:03:33.810212 13808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c63734839347c4aee340a59c8e3221" member_type: VOTER }
I20260709 13:03:33.810725 13808 raft_consensus.cc:399] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:03:33.810982 13808 raft_consensus.cc:493] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:03:33.811264 13808 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:33.816027 13808 raft_consensus.cc:515] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c63734839347c4aee340a59c8e3221" member_type: VOTER }
I20260709 13:03:33.816604 13808 leader_election.cc:304] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91c63734839347c4aee340a59c8e3221; no voters: 
I20260709 13:03:33.817901 13808 leader_election.cc:290] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:03:33.818357 13811 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:33.820173 13811 raft_consensus.cc:697] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [term 1 LEADER]: Becoming Leader. State: Replica: 91c63734839347c4aee340a59c8e3221, State: Running, Role: LEADER
I20260709 13:03:33.820881 13811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c63734839347c4aee340a59c8e3221" member_type: VOTER }
I20260709 13:03:33.821604 13808 sys_catalog.cc:565] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:03:33.826260 13812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91c63734839347c4aee340a59c8e3221" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c63734839347c4aee340a59c8e3221" member_type: VOTER } }
I20260709 13:03:33.826469 13813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91c63734839347c4aee340a59c8e3221. Latest consensus state: current_term: 1 leader_uuid: "91c63734839347c4aee340a59c8e3221" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c63734839347c4aee340a59c8e3221" member_type: VOTER } }
I20260709 13:03:33.827029 13812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:33.827055 13813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:33.829967 13818 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:03:33.835805 13818 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:03:33.839404 12725 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:03:33.845636 13818 catalog_manager.cc:1383] Generated new cluster ID: 1d51c7391c0741bebfefee09dbc7b657
I20260709 13:03:33.845948 13818 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:03:33.862459 13818 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:03:33.864243 13818 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:03:33.883502 13818 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Generated new TSK 0
I20260709 13:03:33.884207 13818 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:03:33.906836 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:33.913547 13829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:33.914624 13830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:33.916220 13832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:33.916960 12725 server_base.cc:1061] running on GCE node
I20260709 13:03:33.917840 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:33.918048 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:33.918246 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602213918208 us; error 0 us; skew 500 ppm
I20260709 13:03:33.918879 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:33.921298 12725 webserver.cc:533] Webserver started at http://127.12.109.65:39323/ using document root <none> and password file <none>
I20260709 13:03:33.921895 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:33.922150 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:33.922465 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:33.923704 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/instance:
uuid: "e1649107c781428692458f10597c5742"
format_stamp: "Formatted at 2026-07-09 13:03:33 on dist-test-slave-zhdc"
I20260709 13:03:33.928660 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.005s
I20260709 13:03:33.932466 13837 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:33.933468 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:03:33.933871 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root
uuid: "e1649107c781428692458f10597c5742"
format_stamp: "Formatted at 2026-07-09 13:03:33 on dist-test-slave-zhdc"
I20260709 13:03:33.934268 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:33.956240 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:33.957726 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:33.959591 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:33.962464 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:33.962662 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:33.962994 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:33.963167 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:34.011183 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.65:43607
I20260709 13:03:34.011291 13899 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.65:43607 every 8 connection(s)
I20260709 13:03:34.016744 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:34.024904 13904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:34.027173 13905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:34.030596 13907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:34.031028 12725 server_base.cc:1061] running on GCE node
I20260709 13:03:34.032078 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:34.032325 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:34.032536 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602214032519 us; error 0 us; skew 500 ppm
I20260709 13:03:34.033111 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:34.035763 12725 webserver.cc:533] Webserver started at http://127.12.109.66:35673/ using document root <none> and password file <none>
I20260709 13:03:34.036293 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:34.036526 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:34.036813 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:34.038131 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-1-root/instance:
uuid: "3779649036364744823fc035dac892c7"
format_stamp: "Formatted at 2026-07-09 13:03:34 on dist-test-slave-zhdc"
I20260709 13:03:34.042578 13900 heartbeater.cc:344] Connected to a master server at 127.12.109.126:43315
I20260709 13:03:34.043048 13900 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:34.044127 13900 heartbeater.cc:507] Master 127.12.109.126:43315 requested a full tablet report, sending...
I20260709 13:03:34.045225 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:03:34.046763 13773 ts_manager.cc:194] Registered new tserver with Master: e1649107c781428692458f10597c5742 (127.12.109.65:43607)
I20260709 13:03:34.049315 13773 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57626
I20260709 13:03:34.049881 13912 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:34.050841 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:03:34.051151 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-1-root
uuid: "3779649036364744823fc035dac892c7"
format_stamp: "Formatted at 2026-07-09 13:03:34 on dist-test-slave-zhdc"
I20260709 13:03:34.051452 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:34.068789 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:34.070276 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:34.071954 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:34.074470 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:34.074656 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:34.074947 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:34.075120 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:34.123208 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.66:43953
I20260709 13:03:34.123371 13974 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.66:43953 every 8 connection(s)
I20260709 13:03:34.128424 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:34.136718 13978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:34.138298 13979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:34.143759 13981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:34.146123 13975 heartbeater.cc:344] Connected to a master server at 127.12.109.126:43315
I20260709 13:03:34.146260 12725 server_base.cc:1061] running on GCE node
I20260709 13:03:34.146718 13975 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:34.147509 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:34.147804 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:34.147778 13975 heartbeater.cc:507] Master 127.12.109.126:43315 requested a full tablet report, sending...
I20260709 13:03:34.148169 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602214148143 us; error 0 us; skew 500 ppm
I20260709 13:03:34.148990 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:34.150625 13773 ts_manager.cc:194] Registered new tserver with Master: 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:34.152166 12725 webserver.cc:533] Webserver started at http://127.12.109.67:32947/ using document root <none> and password file <none>
I20260709 13:03:34.152631 13773 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57642
I20260709 13:03:34.152885 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:34.153172 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:34.153503 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:34.155220 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root/instance:
uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
format_stamp: "Formatted at 2026-07-09 13:03:34 on dist-test-slave-zhdc"
I20260709 13:03:34.160254 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:03:34.163901 13986 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:34.164757 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:03:34.165171 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root
uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
format_stamp: "Formatted at 2026-07-09 13:03:34 on dist-test-slave-zhdc"
I20260709 13:03:34.165493 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:34.178464 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:34.179952 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:34.181684 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:34.184288 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:34.184541 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:34.184806 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:34.184994 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:34.232242 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.67:46599
I20260709 13:03:34.232335 14048 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.67:46599 every 8 connection(s)
I20260709 13:03:34.248327 14049 heartbeater.cc:344] Connected to a master server at 127.12.109.126:43315
I20260709 13:03:34.248740 14049 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:34.249610 14049 heartbeater.cc:507] Master 127.12.109.126:43315 requested a full tablet report, sending...
I20260709 13:03:34.251880 12725 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01617977s
I20260709 13:03:34.251801 13773 ts_manager.cc:194] Registered new tserver with Master: d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:34.253989 13773 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57648
I20260709 13:03:34.282224 13773 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57656:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:03:34.288926 13773 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:03:34.508674 14014 tablet_service.cc:1511] Processing CreateTablet for tablet f5c93fb008bd4a4f8db94cdc59c65d93 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:34.509487 14013 tablet_service.cc:1511] Processing CreateTablet for tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:34.510383 14014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c93fb008bd4a4f8db94cdc59c65d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.511461 14012 tablet_service.cc:1511] Processing CreateTablet for tablet 17624553356541c69c1644ca409df1e9 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:34.512918 14012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17624553356541c69c1644ca409df1e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.513612 13940 tablet_service.cc:1511] Processing CreateTablet for tablet f5c93fb008bd4a4f8db94cdc59c65d93 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:34.514585 13863 tablet_service.cc:1511] Processing CreateTablet for tablet 17624553356541c69c1644ca409df1e9 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:34.515077 13862 tablet_service.cc:1511] Processing CreateTablet for tablet 435cf49d11154354987bf559d46d3c3a (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:34.515506 13938 tablet_service.cc:1511] Processing CreateTablet for tablet 17624553356541c69c1644ca409df1e9 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:34.516197 13863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17624553356541c69c1644ca409df1e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.516450 13864 tablet_service.cc:1511] Processing CreateTablet for tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:34.517024 13862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435cf49d11154354987bf559d46d3c3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.517864 13864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.515108 13940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c93fb008bd4a4f8db94cdc59c65d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.517961 13939 tablet_service.cc:1511] Processing CreateTablet for tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:34.518666 13865 tablet_service.cc:1511] Processing CreateTablet for tablet f5c93fb008bd4a4f8db94cdc59c65d93 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:34.519459 13939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.520112 13865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c93fb008bd4a4f8db94cdc59c65d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.520324 13938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17624553356541c69c1644ca409df1e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.523157 14013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.544685 14064 tablet_bootstrap.cc:492] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:34.553987 13937 tablet_service.cc:1511] Processing CreateTablet for tablet 435cf49d11154354987bf559d46d3c3a (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:34.554951 14011 tablet_service.cc:1511] Processing CreateTablet for tablet 435cf49d11154354987bf559d46d3c3a (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:34.555644 13937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435cf49d11154354987bf559d46d3c3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.556412 14011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435cf49d11154354987bf559d46d3c3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.557710 13864 tablet_service.cc:1511] Processing CreateTablet for tablet 693a6579cdaa4e2a977368c2c6d5267a (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:34.559158 13864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 693a6579cdaa4e2a977368c2c6d5267a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.558843 14010 tablet_service.cc:1511] Processing CreateTablet for tablet 693a6579cdaa4e2a977368c2c6d5267a (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:34.560199 14010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 693a6579cdaa4e2a977368c2c6d5267a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.563524 13861 tablet_service.cc:1511] Processing CreateTablet for tablet 545fc4aeb71d491985e296a0721d905d (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:34.564577 13936 tablet_service.cc:1511] Processing CreateTablet for tablet 693a6579cdaa4e2a977368c2c6d5267a (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:34.565136 13861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 545fc4aeb71d491985e296a0721d905d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.565999 13936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 693a6579cdaa4e2a977368c2c6d5267a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.579770 13935 tablet_service.cc:1511] Processing CreateTablet for tablet 545fc4aeb71d491985e296a0721d905d (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:34.581673 13935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 545fc4aeb71d491985e296a0721d905d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.583529 14009 tablet_service.cc:1511] Processing CreateTablet for tablet 545fc4aeb71d491985e296a0721d905d (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:34.585311 14009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 545fc4aeb71d491985e296a0721d905d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.587545 13860 tablet_service.cc:1511] Processing CreateTablet for tablet 76071b78766a43e0b5097f615de4c569 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:34.589200 13860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76071b78766a43e0b5097f615de4c569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.583406 13934 tablet_service.cc:1511] Processing CreateTablet for tablet 76071b78766a43e0b5097f615de4c569 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:34.591825 14013 tablet_service.cc:1511] Processing CreateTablet for tablet 76071b78766a43e0b5097f615de4c569 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:34.593583 14013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76071b78766a43e0b5097f615de4c569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.594326 13934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76071b78766a43e0b5097f615de4c569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.595810 14064 tablet_bootstrap.cc:654] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.606410 13862 tablet_service.cc:1511] Processing CreateTablet for tablet 0768fb01770349dfb259ab0db91394ba (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:34.607872 13862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0768fb01770349dfb259ab0db91394ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.609098 13938 tablet_service.cc:1511] Processing CreateTablet for tablet 0768fb01770349dfb259ab0db91394ba (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:34.610752 13938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0768fb01770349dfb259ab0db91394ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.610392 14008 tablet_service.cc:1511] Processing CreateTablet for tablet 0768fb01770349dfb259ab0db91394ba (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:34.611922 14008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0768fb01770349dfb259ab0db91394ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.612381 13859 tablet_service.cc:1511] Processing CreateTablet for tablet efeae9bc4fd84c4487b09362929c2fa6 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:34.614459 13859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efeae9bc4fd84c4487b09362929c2fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.644665 14066 tablet_bootstrap.cc:492] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:34.658002 14065 tablet_bootstrap.cc:492] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:34.658713 14014 tablet_service.cc:1511] Processing CreateTablet for tablet efeae9bc4fd84c4487b09362929c2fa6 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:34.660140 14014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efeae9bc4fd84c4487b09362929c2fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.678434 13864 tablet_service.cc:1511] Processing CreateTablet for tablet 1ec9c4d3cc3a47bd838d18fe7da650fc (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:34.680128 13864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ec9c4d3cc3a47bd838d18fe7da650fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.680222 13936 tablet_service.cc:1511] Processing CreateTablet for tablet 1ec9c4d3cc3a47bd838d18fe7da650fc (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:34.681774 13936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ec9c4d3cc3a47bd838d18fe7da650fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.682904 13940 tablet_service.cc:1511] Processing CreateTablet for tablet efeae9bc4fd84c4487b09362929c2fa6 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:34.691231 13858 tablet_service.cc:1511] Processing CreateTablet for tablet c3f6783295084924801a104ff3faf10e (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:34.692867 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3f6783295084924801a104ff3faf10e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.703830 14065 tablet_bootstrap.cc:654] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.713060 13939 tablet_service.cc:1511] Processing CreateTablet for tablet c3f6783295084924801a104ff3faf10e (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:34.714648 14012 tablet_service.cc:1511] Processing CreateTablet for tablet 1ec9c4d3cc3a47bd838d18fe7da650fc (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:34.716107 14064 tablet_bootstrap.cc:492] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:34.716796 14064 ts_tablet_manager.cc:1403] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.173s	user 0.014s	sys 0.017s
I20260709 13:03:34.718564 14066 tablet_bootstrap.cc:654] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.719018 13940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efeae9bc4fd84c4487b09362929c2fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.719884 13939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3f6783295084924801a104ff3faf10e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.716097 14012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ec9c4d3cc3a47bd838d18fe7da650fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.731994 14064 raft_consensus.cc:359] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:34.735698 14064 raft_consensus.cc:385] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.736158 14064 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.739446 14008 tablet_service.cc:1511] Processing CreateTablet for tablet c3f6783295084924801a104ff3faf10e (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:34.740971 14008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3f6783295084924801a104ff3faf10e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.748888 13859 tablet_service.cc:1511] Processing CreateTablet for tablet a3d53d5ea0e8450e97d9e321677e63d5 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:34.753466 13859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d53d5ea0e8450e97d9e321677e63d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.762264 14064 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:34.766487 13937 tablet_service.cc:1511] Processing CreateTablet for tablet a3d53d5ea0e8450e97d9e321677e63d5 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:34.768736 13937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d53d5ea0e8450e97d9e321677e63d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.768975 14013 tablet_service.cc:1511] Processing CreateTablet for tablet a3d53d5ea0e8450e97d9e321677e63d5 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:34.771792 13938 tablet_service.cc:1511] Processing CreateTablet for tablet ef6c4cc997504124a8bb7fa9c7f10e34 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:34.781968 13938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef6c4cc997504124a8bb7fa9c7f10e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.782302 14065 tablet_bootstrap.cc:492] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:34.782394 13863 tablet_service.cc:1511] Processing CreateTablet for tablet ef6c4cc997504124a8bb7fa9c7f10e34 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:34.782927 14065 ts_tablet_manager.cc:1403] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.125s	user 0.016s	sys 0.000s
I20260709 13:03:34.783785 13863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef6c4cc997504124a8bb7fa9c7f10e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.786645 14013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d53d5ea0e8450e97d9e321677e63d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.788832 14065 raft_consensus.cc:359] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } }
I20260709 13:03:34.789655 14065 raft_consensus.cc:385] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.801342 13861 tablet_service.cc:1511] Processing CreateTablet for tablet be9dcc1a27af4a9a91a283a728d4132d (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:34.802377 13934 tablet_service.cc:1511] Processing CreateTablet for tablet be9dcc1a27af4a9a91a283a728d4132d (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:34.803324 13861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9dcc1a27af4a9a91a283a728d4132d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.803649 13934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9dcc1a27af4a9a91a283a728d4132d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.789961 14065 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.805908 13858 tablet_service.cc:1511] Processing CreateTablet for tablet fdef734205364543b02ed4158216bf36 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:34.807328 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdef734205364543b02ed4158216bf36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.807037 14065 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } }
I20260709 13:03:34.811388 14010 tablet_service.cc:1511] Processing CreateTablet for tablet be9dcc1a27af4a9a91a283a728d4132d (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:34.812847 14010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9dcc1a27af4a9a91a283a728d4132d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.824445 13940 tablet_service.cc:1511] Processing CreateTablet for tablet fdef734205364543b02ed4158216bf36 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
W20260709 13:03:34.834781 13821 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:34.827561 14064 ts_tablet_manager.cc:1434] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.110s	user 0.017s	sys 0.028s
I20260709 13:03:34.835968 13900 heartbeater.cc:499] Master 127.12.109.126:43315 was elected leader, sending a full tablet report...
I20260709 13:03:34.834849 13940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdef734205364543b02ed4158216bf36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.830330 14011 tablet_service.cc:1511] Processing CreateTablet for tablet fdef734205364543b02ed4158216bf36 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:34.831686 13862 tablet_service.cc:1511] Processing CreateTablet for tablet 403f6d16105d4532ae1de441c32f632a (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:34.843070 13862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 403f6d16105d4532ae1de441c32f632a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.827605 14065 ts_tablet_manager.cc:1434] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.004s
I20260709 13:03:34.832628 13935 tablet_service.cc:1511] Processing CreateTablet for tablet 403f6d16105d4532ae1de441c32f632a (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:34.847158 14008 tablet_service.cc:1511] Processing CreateTablet for tablet ef6c4cc997504124a8bb7fa9c7f10e34 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:34.848863 13935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 403f6d16105d4532ae1de441c32f632a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.849787 14066 tablet_bootstrap.cc:492] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:34.850298 14066 ts_tablet_manager.cc:1403] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.206s	user 0.023s	sys 0.039s
I20260709 13:03:34.853111 14049 heartbeater.cc:499] Master 127.12.109.126:43315 was elected leader, sending a full tablet report...
I20260709 13:03:34.862740 14064 tablet_bootstrap.cc:492] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:34.862514 14066 raft_consensus.cc:359] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } }
I20260709 13:03:34.863541 14066 raft_consensus.cc:385] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.863911 14066 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.871951 14064 tablet_bootstrap.cc:654] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.879799 14065 tablet_bootstrap.cc:492] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:34.881682 14008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef6c4cc997504124a8bb7fa9c7f10e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.886577 14011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdef734205364543b02ed4158216bf36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.896384 14014 tablet_service.cc:1511] Processing CreateTablet for tablet 403f6d16105d4532ae1de441c32f632a (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:34.897883 14014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 403f6d16105d4532ae1de441c32f632a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.905328 14064 tablet_bootstrap.cc:492] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:34.905692 14065 tablet_bootstrap.cc:654] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.905903 14064 ts_tablet_manager.cc:1403] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.009s
I20260709 13:03:34.919021 14066 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } }
I20260709 13:03:34.957648 14064 raft_consensus.cc:359] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:34.958628 14064 raft_consensus.cc:385] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.958979 14064 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
W20260709 13:03:34.959978 13757 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 49199.2us
I20260709 13:03:34.959858 14064 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:34.962740 14064 ts_tablet_manager.cc:1434] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.056s	user 0.023s	sys 0.033s
I20260709 13:03:34.963943 14064 tablet_bootstrap.cc:492] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:34.964636 13975 heartbeater.cc:499] Master 127.12.109.126:43315 was elected leader, sending a full tablet report...
I20260709 13:03:34.971488 14064 tablet_bootstrap.cc:654] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.971618 14070 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.972219 14070 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:34.975471 14066 ts_tablet_manager.cc:1434] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.125s	user 0.013s	sys 0.002s
I20260709 13:03:34.976688 14066 tablet_bootstrap.cc:492] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:34.983718 14064 tablet_bootstrap.cc:492] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:34.983826 14070 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:34.984488 14064 ts_tablet_manager.cc:1403] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.008s
I20260709 13:03:34.988010 14064 raft_consensus.cc:359] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:34.988792 14064 raft_consensus.cc:385] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.989121 14064 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.988910 13938 tablet_service.cc:1511] Processing CreateTablet for tablet 5d6ce914003a44f2b5a485ae7962383a (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:34.989984 14064 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:34.991140 13938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d6ce914003a44f2b5a485ae7962383a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:34.992726 14064 ts_tablet_manager.cc:1434] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:03:34.993822 14064 tablet_bootstrap.cc:492] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:35.000962 14064 tablet_bootstrap.cc:654] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.002535 13862 tablet_service.cc:1511] Processing CreateTablet for tablet 5d6ce914003a44f2b5a485ae7962383a (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:35.011652 13862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d6ce914003a44f2b5a485ae7962383a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.009050 13861 tablet_service.cc:1511] Processing CreateTablet for tablet 4eeee4c6df564edcb3c96e2887208cc4 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:35.008700 14010 tablet_service.cc:1511] Processing CreateTablet for tablet 5d6ce914003a44f2b5a485ae7962383a (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:35.014143 13934 tablet_service.cc:1511] Processing CreateTablet for tablet 4eeee4c6df564edcb3c96e2887208cc4 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:35.015228 14010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d6ce914003a44f2b5a485ae7962383a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.015121 14013 tablet_service.cc:1511] Processing CreateTablet for tablet 4eeee4c6df564edcb3c96e2887208cc4 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:35.015707 13934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eeee4c6df564edcb3c96e2887208cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.016472 14013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eeee4c6df564edcb3c96e2887208cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.017545 13861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eeee4c6df564edcb3c96e2887208cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.025187 14073 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.025931 14073 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } }
I20260709 13:03:35.035338 14073 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:35.037868 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:03:35.040262 13838 leader_election.cc:343] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.088119 14065 tablet_bootstrap.cc:492] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:35.088802 14065 ts_tablet_manager.cc:1403] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.209s	user 0.031s	sys 0.025s
I20260709 13:03:35.092406 14065 raft_consensus.cc:359] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.093215 14065 raft_consensus.cc:385] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.093566 14065 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.094537 14065 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.097080 14065 ts_tablet_manager.cc:1434] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:03:35.098207 14065 tablet_bootstrap.cc:492] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:35.100646 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:35.101436 14024 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:35.103420 13840 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:03:35.104451 14070 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:35.104820 14070 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.105167 14070 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.105455 14065 tablet_bootstrap.cc:654] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.114423 13858 tablet_service.cc:1511] Processing CreateTablet for tablet dcd5a97e61554ed698b00fdf13294a7b (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:35.116412 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcd5a97e61554ed698b00fdf13294a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.117522 14065 tablet_bootstrap.cc:492] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:35.118093 14065 ts_tablet_manager.cc:1403] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 13:03:35.119925 14066 tablet_bootstrap.cc:654] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.121421 14065 raft_consensus.cc:359] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.122475 14065 raft_consensus.cc:385] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.122870 14065 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.123761 14065 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.124169 14070 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.125461 13935 tablet_service.cc:1511] Processing CreateTablet for tablet dcd5a97e61554ed698b00fdf13294a7b (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:35.126485 14065 ts_tablet_manager.cc:1434] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:03:35.127563 14065 tablet_bootstrap.cc:492] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:35.127390 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:03:35.130543 13838 leader_election.cc:343] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.127351 13935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcd5a97e61554ed698b00fdf13294a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.135052 14065 tablet_bootstrap.cc:654] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.151628 14066 tablet_bootstrap.cc:492] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:35.152185 14066 ts_tablet_manager.cc:1403] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.176s	user 0.016s	sys 0.005s
I20260709 13:03:35.155791 14064 tablet_bootstrap.cc:492] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:35.156384 14064 ts_tablet_manager.cc:1403] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.163s	user 0.074s	sys 0.071s
I20260709 13:03:35.159901 14064 raft_consensus.cc:359] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.160650 14064 raft_consensus.cc:385] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.160946 14064 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.163477 14014 tablet_service.cc:1511] Processing CreateTablet for tablet dcd5a97e61554ed698b00fdf13294a7b (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:35.164027 14066 raft_consensus.cc:359] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.164844 14066 raft_consensus.cc:385] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.165218 14066 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.166145 14066 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.168537 14066 ts_tablet_manager.cc:1434] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.003s
I20260709 13:03:35.169575 14066 tablet_bootstrap.cc:492] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:35.170779 14008 tablet_service.cc:1511] Processing CreateTablet for tablet 208a6eaac64647a59ee173ef6726ae8b (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:35.164928 14014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcd5a97e61554ed698b00fdf13294a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.174409 14064 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.174908 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:35.178797 13861 tablet_service.cc:1511] Processing CreateTablet for tablet 208a6eaac64647a59ee173ef6726ae8b (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:35.179037 13859 tablet_service.cc:1511] Processing CreateTablet for tablet 6d6d5ae5feb04bda8f546e7e4ca80503 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:35.180215 13861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208a6eaac64647a59ee173ef6726ae8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.175529 14024 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.182592 14066 tablet_bootstrap.cc:654] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.184867 13859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d6d5ae5feb04bda8f546e7e4ca80503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.186619 13934 tablet_service.cc:1511] Processing CreateTablet for tablet 208a6eaac64647a59ee173ef6726ae8b (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:35.187466 14008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208a6eaac64647a59ee173ef6726ae8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.188071 13934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208a6eaac64647a59ee173ef6726ae8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.176360 14070 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:35.203136 14085 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.207875 13939 tablet_service.cc:1511] Processing CreateTablet for tablet 6d6d5ae5feb04bda8f546e7e4ca80503 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:35.209408 13939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d6d5ae5feb04bda8f546e7e4ca80503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.210976 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:35.211969 14013 tablet_service.cc:1511] Processing CreateTablet for tablet 6d6d5ae5feb04bda8f546e7e4ca80503 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:35.214250 14013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d6d5ae5feb04bda8f546e7e4ca80503. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:35.214001 13915 leader_election.cc:343] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.226293 14085 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.230299 14085 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:35.231700 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:35.232520 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:03:35.233466 13838 leader_election.cc:343] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:35.234272 13840 leader_election.cc:343] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.234740 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:35.236616 14085 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:35.238588 13858 tablet_service.cc:1511] Processing CreateTablet for tablet 9a37812504ce4bdb94c7eafc67fa5aee (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:35.240288 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a37812504ce4bdb94c7eafc67fa5aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.246817 14024 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:35.246929 13938 tablet_service.cc:1511] Processing CreateTablet for tablet 9a37812504ce4bdb94c7eafc67fa5aee (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:35.247773 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:35.248492 14023 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:35.248764 13938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a37812504ce4bdb94c7eafc67fa5aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.249560 13840 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:03:35.250483 14085 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:35.251703 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:35.251941 14085 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:03:35.253100 14073 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:35.253552 14073 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.253046 14085 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.253885 14073 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.256984 14010 tablet_service.cc:1511] Processing CreateTablet for tablet 9a37812504ce4bdb94c7eafc67fa5aee (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:35.258559 14010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a37812504ce4bdb94c7eafc67fa5aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.259887 14066 tablet_bootstrap.cc:492] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:35.260418 14066 ts_tablet_manager.cc:1403] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.091s	user 0.013s	sys 0.018s
I20260709 13:03:35.263556 14064 ts_tablet_manager.cc:1434] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.107s	user 0.008s	sys 0.003s
I20260709 13:03:35.264665 14064 tablet_bootstrap.cc:492] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:35.267373 14066 raft_consensus.cc:359] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.268168 14066 raft_consensus.cc:385] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.268461 14066 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.278596 13773 catalog_manager.cc:5697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:35.236867 14065 tablet_bootstrap.cc:492] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:35.269346 14066 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.290289 14065 ts_tablet_manager.cc:1403] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.163s	user 0.015s	sys 0.015s
I20260709 13:03:35.290550 13863 tablet_service.cc:1511] Processing CreateTablet for tablet 82051b9bd85845d6b73773b21ccc6d6c (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:35.292091 13863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82051b9bd85845d6b73773b21ccc6d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.295557 14073 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } }
I20260709 13:03:35.298588 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:35.299506 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:35.300125 14023 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.301276 13935 tablet_service.cc:1511] Processing CreateTablet for tablet 82051b9bd85845d6b73773b21ccc6d6c (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
W20260709 13:03:35.300137 13915 leader_election.cc:343] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.303431 13935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82051b9bd85845d6b73773b21ccc6d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.305972 14011 tablet_service.cc:1511] Processing CreateTablet for tablet 82051b9bd85845d6b73773b21ccc6d6c (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:35.307356 14011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82051b9bd85845d6b73773b21ccc6d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.310204 14073 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:35.312621 14023 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:35.314122 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:35.301085 14065 raft_consensus.cc:359] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.366245 13858 tablet_service.cc:1511] Processing CreateTablet for tablet 3730fc366f4349dea87ae85e1aed546c (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:35.390220 14065 raft_consensus.cc:385] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.390679 14065 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.390859 14064 tablet_bootstrap.cc:654] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.391057 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3730fc366f4349dea87ae85e1aed546c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.338840 14085 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.368584 13934 tablet_service.cc:1511] Processing CreateTablet for tablet 3730fc366f4349dea87ae85e1aed546c (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:35.375326 14070 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.377862 14066 ts_tablet_manager.cc:1434] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.117s	user 0.022s	sys 0.018s
I20260709 13:03:35.346858 14073 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:35.392184 14070 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.406270 14070 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:35.406698 14010 tablet_service.cc:1511] Processing CreateTablet for tablet 3730fc366f4349dea87ae85e1aed546c (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:35.408151 14010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3730fc366f4349dea87ae85e1aed546c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.407951 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:35.408773 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:03:35.410293 13838 leader_election.cc:343] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.407969 13934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3730fc366f4349dea87ae85e1aed546c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:35.411180 13840 leader_election.cc:343] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.411754 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:35.399390 14085 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.413249 14070 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:35.414625 14085 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:35.423364 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:35.424772 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:03:35.425381 13840 leader_election.cc:343] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.430527 14066 tablet_bootstrap.cc:492] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Bootstrap starting.
W20260709 13:03:35.431070 13838 leader_election.cc:343] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.431905 13838 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:35.433043 14085 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:35.401716 14065 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.442826 14065 ts_tablet_manager.cc:1434] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.152s	user 0.010s	sys 0.002s
I20260709 13:03:35.446722 14089 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.447312 14089 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.404338 14073 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:35.451115 14073 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } }
I20260709 13:03:35.456478 14065 tablet_bootstrap.cc:492] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:35.465919 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:35.466823 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:35.467690 14023 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:03:35.469038 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:35.488673 13938 tablet_service.cc:1511] Processing CreateTablet for tablet f50263f80db844239948e5e67ef3b987 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:35.492766 13938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50263f80db844239948e5e67ef3b987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.470453 14089 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:35.481164 14066 tablet_bootstrap.cc:654] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.497908 14071 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.499694 14071 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.503310 14071 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:35.498973 14065 tablet_bootstrap.cc:654] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.504101 14089 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.515077 14014 tablet_service.cc:1511] Processing CreateTablet for tablet f50263f80db844239948e5e67ef3b987 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:35.516480 14014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50263f80db844239948e5e67ef3b987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.517160 13858 tablet_service.cc:1511] Processing CreateTablet for tablet 0e8d34f260fb43229460064c72d795e8 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:35.518505 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e8d34f260fb43229460064c72d795e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.521857 13934 tablet_service.cc:1511] Processing CreateTablet for tablet 0e8d34f260fb43229460064c72d795e8 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:35.514729 13861 tablet_service.cc:1511] Processing CreateTablet for tablet f50263f80db844239948e5e67ef3b987 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:35.523324 13934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e8d34f260fb43229460064c72d795e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.523636 13861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50263f80db844239948e5e67ef3b987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.536443 14011 tablet_service.cc:1511] Processing CreateTablet for tablet 0e8d34f260fb43229460064c72d795e8 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:35.537982 14011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e8d34f260fb43229460064c72d795e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.547173 14071 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.547813 14071 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.550343 14071 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:35.560859 13771 catalog_manager.cc:5697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:35.563822 14064 tablet_bootstrap.cc:492] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:35.564446 14064 ts_tablet_manager.cc:1403] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.300s	user 0.030s	sys 0.032s
I20260709 13:03:35.571120 13939 tablet_service.cc:1511] Processing CreateTablet for tablet 34d11c517b334dd5abf16448009bdf2d (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:35.577735 13939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d11c517b334dd5abf16448009bdf2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.598160 14010 tablet_service.cc:1511] Processing CreateTablet for tablet 34d11c517b334dd5abf16448009bdf2d (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:35.599449 14010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d11c517b334dd5abf16448009bdf2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.603277 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:35.604769 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:35.608156 13989 leader_election.cc:343] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:35.609148 13989 leader_election.cc:343] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.609249 13858 tablet_service.cc:1511] Processing CreateTablet for tablet 34d11c517b334dd5abf16448009bdf2d (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:35.587697 14064 raft_consensus.cc:359] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.614964 14064 raft_consensus.cc:385] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.617218 14064 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.617941 13859 tablet_service.cc:1511] Processing CreateTablet for tablet 86de6a18f3bd4b23a8b66dea2cf5c6a9 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:35.618156 14064 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.625229 14064 ts_tablet_manager.cc:1434] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.060s	user 0.011s	sys 0.000s
I20260709 13:03:35.626988 13859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86de6a18f3bd4b23a8b66dea2cf5c6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.627826 13935 tablet_service.cc:1511] Processing CreateTablet for tablet 86de6a18f3bd4b23a8b66dea2cf5c6a9 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:35.629276 13935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86de6a18f3bd4b23a8b66dea2cf5c6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.634654 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:35.635434 13949 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:35.633914 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:35.636454 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d11c517b334dd5abf16448009bdf2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.636674 13862 tablet_service.cc:1511] Processing CreateTablet for tablet a168d82765704a87b2c60aeb7b23e8b0 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:35.638108 13987 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:35.638396 13862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a168d82765704a87b2c60aeb7b23e8b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.639269 14071 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:35.639667 14071 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.640054 14071 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.640836 14010 tablet_service.cc:1511] Processing CreateTablet for tablet 86de6a18f3bd4b23a8b66dea2cf5c6a9 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:35.642638 14010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86de6a18f3bd4b23a8b66dea2cf5c6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.650322 14066 tablet_bootstrap.cc:492] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:35.650851 14066 ts_tablet_manager.cc:1403] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.221s	user 0.011s	sys 0.026s
I20260709 13:03:35.653245 13937 tablet_service.cc:1511] Processing CreateTablet for tablet a168d82765704a87b2c60aeb7b23e8b0 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:35.654366 14066 raft_consensus.cc:359] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.654968 13937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a168d82765704a87b2c60aeb7b23e8b0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:35.649446 13987 leader_election.cc:343] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.654663 14008 tablet_service.cc:1511] Processing CreateTablet for tablet a168d82765704a87b2c60aeb7b23e8b0 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:35.655206 14066 raft_consensus.cc:385] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.661408 14066 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.661562 13940 tablet_service.cc:1511] Processing CreateTablet for tablet 92405e92560c4099974ff3e6a4564895 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:35.662949 13940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92405e92560c4099974ff3e6a4564895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.663204 14066 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.665230 13864 tablet_service.cc:1511] Processing CreateTablet for tablet 7991c821f3da4b7abc0f6bb6baf8fe99 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:35.665712 14066 ts_tablet_manager.cc:1434] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.003s
I20260709 13:03:35.657229 14089 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:35.659276 13863 tablet_service.cc:1511] Processing CreateTablet for tablet 92405e92560c4099974ff3e6a4564895 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:35.662483 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:35.672894 13865 tablet_service.cc:1511] Processing CreateTablet for tablet 10a0b27264ce49379f2aba40cdbc2b0f (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:35.677148 14013 tablet_service.cc:1511] Processing CreateTablet for tablet 92405e92560c4099974ff3e6a4564895 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:35.682660 14066 tablet_bootstrap.cc:492] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:35.686221 14012 tablet_service.cc:1511] Processing CreateTablet for tablet 7991c821f3da4b7abc0f6bb6baf8fe99 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:35.668392 13864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7991c821f3da4b7abc0f6bb6baf8fe99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.689553 14009 tablet_service.cc:1511] Processing CreateTablet for tablet 10a0b27264ce49379f2aba40cdbc2b0f (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:35.691901 13865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10a0b27264ce49379f2aba40cdbc2b0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.692435 14008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a168d82765704a87b2c60aeb7b23e8b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.692829 13860 tablet_service.cc:1511] Processing CreateTablet for tablet 47fee46f08eb4928939d6ac41891db38 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:35.696369 14065 tablet_bootstrap.cc:492] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:35.696931 14065 ts_tablet_manager.cc:1403] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.244s	user 0.012s	sys 0.014s
I20260709 13:03:35.698559 14009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10a0b27264ce49379f2aba40cdbc2b0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.700349 14065 raft_consensus.cc:359] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.701203 14065 raft_consensus.cc:385] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.709156 13936 tablet_service.cc:1511] Processing CreateTablet for tablet 7991c821f3da4b7abc0f6bb6baf8fe99 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:35.710638 13936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7991c821f3da4b7abc0f6bb6baf8fe99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.714558 13863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92405e92560c4099974ff3e6a4564895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.718266 14012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7991c821f3da4b7abc0f6bb6baf8fe99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.719010 14013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92405e92560c4099974ff3e6a4564895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.722251 14064 tablet_bootstrap.cc:492] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:35.714607 14066 tablet_bootstrap.cc:654] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.739176 13933 tablet_service.cc:1511] Processing CreateTablet for tablet 10a0b27264ce49379f2aba40cdbc2b0f (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:35.702857 14065 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.740777 13933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10a0b27264ce49379f2aba40cdbc2b0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.747967 13932 tablet_service.cc:1511] Processing CreateTablet for tablet 47fee46f08eb4928939d6ac41891db38 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:35.743139 14065 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.744803 14093 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.753120 13860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47fee46f08eb4928939d6ac41891db38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.766615 13932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47fee46f08eb4928939d6ac41891db38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.764097 14093 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.770990 14071 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.775723 14093 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:35.777148 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:35.777192 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:35.779070 13874 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:35.780524 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:03:35.780646 13989 leader_election.cc:343] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.781471 14093 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:35.781870 14093 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.782241 14093 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.789706 14093 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.793073 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:35.793758 13874 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.798683 14071 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:35.799790 14071 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:35.804284 14011 tablet_service.cc:1511] Processing CreateTablet for tablet 47fee46f08eb4928939d6ac41891db38 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:35.805779 14011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47fee46f08eb4928939d6ac41891db38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.813006 14064 tablet_bootstrap.cc:654] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.816548 14066 tablet_bootstrap.cc:492] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:35.817087 14066 ts_tablet_manager.cc:1403] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.135s	user 0.011s	sys 0.000s
I20260709 13:03:35.820488 14066 raft_consensus.cc:359] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.821261 14066 raft_consensus.cc:385] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.821547 14066 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.830355 13861 tablet_service.cc:1511] Processing CreateTablet for tablet 8dfe02d97ee6481298ba92b29baa32b7 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:35.839232 13861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dfe02d97ee6481298ba92b29baa32b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.833496 14070 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.849510 14093 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:03:35.840694 13821 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:35.832408 14085 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:35.849735 13862 tablet_service.cc:1511] Processing CreateTablet for tablet 4e20f6d8df1f4522b10fd2dc803e0e75 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:35.851531 13862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e20f6d8df1f4522b10fd2dc803e0e75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.836839 13939 tablet_service.cc:1511] Processing CreateTablet for tablet 8dfe02d97ee6481298ba92b29baa32b7 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
W20260709 13:03:35.826211 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 176561us
I20260709 13:03:35.855262 13939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dfe02d97ee6481298ba92b29baa32b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.855798 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:35.856524 13949 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.857277 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:35.823912 13874 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:35.846400 14009 tablet_service.cc:1511] Processing CreateTablet for tablet 8dfe02d97ee6481298ba92b29baa32b7 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:35.842530 14065 ts_tablet_manager.cc:1434] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.145s	user 0.025s	sys 0.009s
I20260709 13:03:35.857988 14024 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.859655 14009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dfe02d97ee6481298ba92b29baa32b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.855805 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:35.849090 14070 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.822454 14066 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.866024 14066 ts_tablet_manager.cc:1434] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.000s
I20260709 13:03:35.867750 14070 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:35.868432 14065 tablet_bootstrap.cc:492] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:35.869843 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:03:35.871632 13838 leader_election.cc:343] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.872371 14070 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:35.875376 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:03:35.882107 13840 leader_election.cc:343] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.884232 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:35.885331 14070 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:35.891188 13933 tablet_service.cc:1511] Processing CreateTablet for tablet 4e20f6d8df1f4522b10fd2dc803e0e75 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:35.892747 13933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e20f6d8df1f4522b10fd2dc803e0e75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.894068 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:35.894459 14013 tablet_service.cc:1511] Processing CreateTablet for tablet 4e20f6d8df1f4522b10fd2dc803e0e75 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:35.895069 14089 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:35.895664 14089 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:35.896011 14013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e20f6d8df1f4522b10fd2dc803e0e75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.896541 14089 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.901644 13949 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:35.906324 14066 tablet_bootstrap.cc:492] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:35.914021 13938 tablet_service.cc:1511] Processing CreateTablet for tablet f2e6672bab624b20b38e363591856fc0 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:35.915524 13938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2e6672bab624b20b38e363591856fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.916085 14008 tablet_service.cc:1511] Processing CreateTablet for tablet f2e6672bab624b20b38e363591856fc0 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:35.917484 14008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2e6672bab624b20b38e363591856fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.917714 13863 tablet_service.cc:1511] Processing CreateTablet for tablet f2e6672bab624b20b38e363591856fc0 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:35.919099 13863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2e6672bab624b20b38e363591856fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.920562 13937 tablet_service.cc:1511] Processing CreateTablet for tablet 7f4c2ba739c04f16a8ed8c28f57fe695 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:35.924142 13935 tablet_service.cc:1511] Processing CreateTablet for tablet 17bd99d4be2c4073bd867c161fa52482 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:35.925379 14010 tablet_service.cc:1511] Processing CreateTablet for tablet 7f4c2ba739c04f16a8ed8c28f57fe695 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:35.921923 13937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f4c2ba739c04f16a8ed8c28f57fe695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.926899 14010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f4c2ba739c04f16a8ed8c28f57fe695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.926838 13858 tablet_service.cc:1511] Processing CreateTablet for tablet 17bd99d4be2c4073bd867c161fa52482 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:35.928205 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17bd99d4be2c4073bd867c161fa52482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.929322 13940 tablet_service.cc:1511] Processing CreateTablet for tablet 05adb1eba4fe476d8504a36537b738db (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:35.930383 13935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17bd99d4be2c4073bd867c161fa52482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.930239 14014 tablet_service.cc:1511] Processing CreateTablet for tablet 17bd99d4be2c4073bd867c161fa52482 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:35.931025 13940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05adb1eba4fe476d8504a36537b738db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.931720 14014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17bd99d4be2c4073bd867c161fa52482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.931656 13865 tablet_service.cc:1511] Processing CreateTablet for tablet 05adb1eba4fe476d8504a36537b738db (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:35.937212 13865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05adb1eba4fe476d8504a36537b738db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.943984 13934 tablet_service.cc:1511] Processing CreateTablet for tablet 4a43a85b399e492b8391cc93e618a1fa (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:35.938225 14089 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.945416 13934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a43a85b399e492b8391cc93e618a1fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:35.944993 14089 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:35.947814 14089 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:35.949594 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:35.925841 13860 tablet_service.cc:1511] Processing CreateTablet for tablet 7f4c2ba739c04f16a8ed8c28f57fe695 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:35.958974 13860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f4c2ba739c04f16a8ed8c28f57fe695. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:35.966697 13915 leader_election.cc:343] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.969585 14024 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:35.982175 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:35.982986 13874 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:35.984371 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:35.985806 14065 tablet_bootstrap.cc:654] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.986088 14093 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:35.989461 14066 tablet_bootstrap.cc:654] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.994017 14007 tablet_service.cc:1511] Processing CreateTablet for tablet 05adb1eba4fe476d8504a36537b738db (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:35.998813 14093 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.013793 13987 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:36.019012 14100 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:36.019547 14100 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:36.021021 14064 tablet_bootstrap.cc:492] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:36.022794 14064 ts_tablet_manager.cc:1403] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.319s	user 0.024s	sys 0.048s
I20260709 13:03:36.014014 14093 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.026328 14064 raft_consensus.cc:359] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } }
I20260709 13:03:36.027115 14064 raft_consensus.cc:385] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.027470 14064 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.028376 14064 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } }
I20260709 13:03:36.032021 14010 tablet_service.cc:1511] Processing CreateTablet for tablet 4a43a85b399e492b8391cc93e618a1fa (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:36.033371 14010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a43a85b399e492b8391cc93e618a1fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.020401 14100 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.053117 14007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05adb1eba4fe476d8504a36537b738db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.058987 13859 tablet_service.cc:1511] Processing CreateTablet for tablet 4a43a85b399e492b8391cc93e618a1fa (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:36.060946 13859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a43a85b399e492b8391cc93e618a1fa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:36.066929 13915 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:36.068279 14073 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:36.040838 14064 ts_tablet_manager.cc:1434] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:03:36.078467 14064 tablet_bootstrap.cc:492] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:35.989614 13771 catalog_manager.cc:5697] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:36.089619 13858 tablet_service.cc:1511] Processing CreateTablet for tablet 481a7f1a133247ebb81e1be603cbdd57 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:36.093032 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481a7f1a133247ebb81e1be603cbdd57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.119408 14065 tablet_bootstrap.cc:492] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:36.120062 14065 ts_tablet_manager.cc:1403] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.252s	user 0.028s	sys 0.039s
I20260709 13:03:36.120546 14066 tablet_bootstrap.cc:492] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:36.121098 14066 ts_tablet_manager.cc:1403] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.254s	user 0.019s	sys 0.006s
I20260709 13:03:36.123656 14065 raft_consensus.cc:359] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.124460 14065 raft_consensus.cc:385] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.124967 14065 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.124428 14066 raft_consensus.cc:359] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.125452 14066 raft_consensus.cc:385] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.125784 14066 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.125900 14065 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.126819 14066 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.128645 14065 ts_tablet_manager.cc:1434] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.006s
I20260709 13:03:36.129724 14065 tablet_bootstrap.cc:492] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:36.131935 14066 ts_tablet_manager.cc:1434] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:03:36.140853 13938 tablet_service.cc:1511] Processing CreateTablet for tablet 481a7f1a133247ebb81e1be603cbdd57 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:36.163411 13938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481a7f1a133247ebb81e1be603cbdd57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.171996 14064 tablet_bootstrap.cc:654] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.171772 13935 tablet_service.cc:1511] Processing CreateTablet for tablet 4daef29e1ece473f9845687b3c7eae60 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:36.174532 14009 tablet_service.cc:1511] Processing CreateTablet for tablet 481a7f1a133247ebb81e1be603cbdd57 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:36.175988 14009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481a7f1a133247ebb81e1be603cbdd57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.170469 14066 tablet_bootstrap.cc:492] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:36.179371 13935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4daef29e1ece473f9845687b3c7eae60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.181628 14065 tablet_bootstrap.cc:654] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.181804 14085 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.182381 14085 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } }
I20260709 13:03:36.184723 14085 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:36.185779 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:36.186646 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:36.188580 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:36.190231 14085 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:36.192641 14093 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.200018 13865 tablet_service.cc:1511] Processing CreateTablet for tablet 4daef29e1ece473f9845687b3c7eae60 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:36.206300 13865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4daef29e1ece473f9845687b3c7eae60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.225997 14064 tablet_bootstrap.cc:492] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:36.228634 13940 tablet_service.cc:1511] Processing CreateTablet for tablet 02dcf286c6bd48e3a6e548def7d786cc (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:36.231719 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:36.232450 13874 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.234344 14064 ts_tablet_manager.cc:1403] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.156s	user 0.011s	sys 0.006s
I20260709 13:03:36.232028 13940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02dcf286c6bd48e3a6e548def7d786cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.235208 14066 tablet_bootstrap.cc:654] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.238087 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:03:36.239603 13915 leader_election.cc:343] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.241431 13934 tablet_service.cc:1511] Processing CreateTablet for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:36.241889 14009 tablet_service.cc:1511] Processing CreateTablet for tablet 4daef29e1ece473f9845687b3c7eae60 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:36.244534 14009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4daef29e1ece473f9845687b3c7eae60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.245538 13939 tablet_service.cc:1511] Processing CreateTablet for tablet 765348f2471a493d92e998132c0c746b (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:36.245913 13861 tablet_service.cc:1511] Processing CreateTablet for tablet 02dcf286c6bd48e3a6e548def7d786cc (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:36.246938 13939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 765348f2471a493d92e998132c0c746b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.247189 13861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02dcf286c6bd48e3a6e548def7d786cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.250861 14065 tablet_bootstrap.cc:492] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:36.251379 14065 ts_tablet_manager.cc:1403] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.122s	user 0.018s	sys 0.021s
I20260709 13:03:36.251400 13934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b098cc2d51e471cb2c9c1b17eebcb57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.254667 14065 raft_consensus.cc:359] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.255461 14065 raft_consensus.cc:385] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.255767 14065 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.256531 14065 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.257586 13874 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:36.259177 14065 ts_tablet_manager.cc:1434] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:36.259016 14008 tablet_service.cc:1511] Processing CreateTablet for tablet 02dcf286c6bd48e3a6e548def7d786cc (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:36.260255 14065 tablet_bootstrap.cc:492] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:36.260434 14008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02dcf286c6bd48e3a6e548def7d786cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.267030 14065 tablet_bootstrap.cc:654] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.243152 14093 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:36.270136 14064 raft_consensus.cc:359] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.270931 14064 raft_consensus.cc:385] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.271247 14064 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.285506 14064 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.290936 14064 ts_tablet_manager.cc:1434] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.000s
I20260709 13:03:36.295104 13859 tablet_service.cc:1511] Processing CreateTablet for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:36.296577 13859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b098cc2d51e471cb2c9c1b17eebcb57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.300341 13860 tablet_service.cc:1511] Processing CreateTablet for tablet 765348f2471a493d92e998132c0c746b (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:36.292152 14064 tablet_bootstrap.cc:492] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:36.303910 13860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 765348f2471a493d92e998132c0c746b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.307837 14012 tablet_service.cc:1511] Processing CreateTablet for tablet 765348f2471a493d92e998132c0c746b (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:36.309309 14012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 765348f2471a493d92e998132c0c746b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.309118 13771 catalog_manager.cc:5697] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:36.319962 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:36.321274 13863 tablet_service.cc:1511] Processing CreateTablet for tablet 99b9907790614d49bc3adf40424fc1b6 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:36.321385 14093 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:36.322701 14093 raft_consensus.cc:697] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:36.323596 14093 consensus_queue.cc:237] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.326867 13863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99b9907790614d49bc3adf40424fc1b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.327198 14007 tablet_service.cc:1511] Processing CreateTablet for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:36.328636 14007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b098cc2d51e471cb2c9c1b17eebcb57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.334714 13934 tablet_service.cc:1511] Processing CreateTablet for tablet 99b9907790614d49bc3adf40424fc1b6 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:36.342314 14008 tablet_service.cc:1511] Processing CreateTablet for tablet 99b9907790614d49bc3adf40424fc1b6 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:36.344344 14089 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:36.348500 13858 tablet_service.cc:1511] Processing CreateTablet for tablet d4033a61013848c69bbd0e79b27a8be3 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:36.349736 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4033a61013848c69bbd0e79b27a8be3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.349769 13934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99b9907790614d49bc3adf40424fc1b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.356161 13771 catalog_manager.cc:5697] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:36.358708 14013 tablet_service.cc:1511] Processing CreateTablet for tablet d4033a61013848c69bbd0e79b27a8be3 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:36.359265 13939 tablet_service.cc:1511] Processing CreateTablet for tablet d4033a61013848c69bbd0e79b27a8be3 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:36.360340 14064 tablet_bootstrap.cc:654] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.360623 13939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4033a61013848c69bbd0e79b27a8be3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.375608 14065 tablet_bootstrap.cc:492] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:36.376257 14065 ts_tablet_manager.cc:1403] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.116s	user 0.036s	sys 0.009s
I20260709 13:03:36.387413 14008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99b9907790614d49bc3adf40424fc1b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.390005 13861 tablet_service.cc:1511] Processing CreateTablet for tablet 567aed48a57a41e5820f0b937f2de9be (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:36.394121 13861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 567aed48a57a41e5820f0b937f2de9be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.395145 13940 tablet_service.cc:1511] Processing CreateTablet for tablet 567aed48a57a41e5820f0b937f2de9be (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:36.396490 13940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 567aed48a57a41e5820f0b937f2de9be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.397732 14014 tablet_service.cc:1511] Processing CreateTablet for tablet 567aed48a57a41e5820f0b937f2de9be (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:36.400785 13860 tablet_service.cc:1511] Processing CreateTablet for tablet 343e333726b74ed98b7e80869ce28ad1 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:36.402143 13860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 343e333726b74ed98b7e80869ce28ad1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.391002 14013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4033a61013848c69bbd0e79b27a8be3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.405089 14014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 567aed48a57a41e5820f0b937f2de9be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.415822 13935 tablet_service.cc:1511] Processing CreateTablet for tablet 343e333726b74ed98b7e80869ce28ad1 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:36.416263 14012 tablet_service.cc:1511] Processing CreateTablet for tablet 343e333726b74ed98b7e80869ce28ad1 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:36.422037 14012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 343e333726b74ed98b7e80869ce28ad1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.421748 13858 tablet_service.cc:1511] Processing CreateTablet for tablet 14bba4e2cef04a418c6f660869c7e6e5 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:36.423344 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14bba4e2cef04a418c6f660869c7e6e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.431530 13874 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.442833 13935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 343e333726b74ed98b7e80869ce28ad1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.451911 14100 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.452513 14100 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.454929 14100 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:36.459688 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:36.460623 13875 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:36.466900 13938 tablet_service.cc:1511] Processing CreateTablet for tablet 14bba4e2cef04a418c6f660869c7e6e5 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:36.467610 14093 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:36.468227 13938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14bba4e2cef04a418c6f660869c7e6e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.469022 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:36.476905 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:36.478117 14100 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.478369 14010 tablet_service.cc:1511] Processing CreateTablet for tablet 14bba4e2cef04a418c6f660869c7e6e5 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:36.479393 14064 tablet_bootstrap.cc:492] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:36.480806 14064 ts_tablet_manager.cc:1403] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.189s	user 0.017s	sys 0.019s
I20260709 13:03:36.491183 14010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14bba4e2cef04a418c6f660869c7e6e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.490741 14064 raft_consensus.cc:359] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.491885 14064 raft_consensus.cc:385] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.494655 14065 raft_consensus.cc:359] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.495252 14065 raft_consensus.cc:385] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.492193 14064 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.495502 14065 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.496184 14064 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.496487 14065 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.498939 14065 ts_tablet_manager.cc:1434] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.122s	user 0.006s	sys 0.003s
I20260709 13:03:36.499917 14065 tablet_bootstrap.cc:492] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:36.507522 13863 tablet_service.cc:1511] Processing CreateTablet for tablet 45103ab867574475a587979f8ea5fa4e (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:36.509018 13863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45103ab867574475a587979f8ea5fa4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.508785 13938 tablet_service.cc:1511] Processing CreateTablet for tablet 45103ab867574475a587979f8ea5fa4e (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:36.510077 13938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45103ab867574475a587979f8ea5fa4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.517230 14065 tablet_bootstrap.cc:654] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.517942 14102 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.518512 14102 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:36.517413 13989 consensus_peers.cc:597] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:36.529579 13861 tablet_service.cc:1511] Processing CreateTablet for tablet 9bfd3f44e83e4908b229bb57863e4080 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:36.530165 14100 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:36.531049 13861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bfd3f44e83e4908b229bb57863e4080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.533423 14066 tablet_bootstrap.cc:492] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:36.533967 14066 ts_tablet_manager.cc:1403] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.369s	user 0.014s	sys 0.019s
I20260709 13:03:36.534899 14102 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:36.536257 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:36.538164 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:36.538311 13987 leader_election.cc:343] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.538897 13875 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:36.540127 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:03:36.542593 14066 raft_consensus.cc:359] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.543342 14066 raft_consensus.cc:385] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.543633 14066 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.546360 14102 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:36.546799 14102 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.547212 14102 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.551375 14064 ts_tablet_manager.cc:1434] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.001s
I20260709 13:03:36.553872 14066 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.565757 14064 tablet_bootstrap.cc:492] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:36.566758 14066 ts_tablet_manager.cc:1434] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:03:36.555822 14008 tablet_service.cc:1511] Processing CreateTablet for tablet 45103ab867574475a587979f8ea5fa4e (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:36.559268 14100 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:36.560719 14007 tablet_service.cc:1511] Processing CreateTablet for tablet 9bfd3f44e83e4908b229bb57863e4080 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:36.569227 14007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bfd3f44e83e4908b229bb57863e4080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.574542 14064 tablet_bootstrap.cc:654] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.575199 14008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45103ab867574475a587979f8ea5fa4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.581280 14102 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.581874 13860 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8599b3d11641199dc0043a592857fb (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:36.583503 13860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8599b3d11641199dc0043a592857fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.584447 14093 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.586194 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:36.586354 14093 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.587123 13949 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.588554 14093 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:36.584672 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:36.584755 14102 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:36.583771 14066 tablet_bootstrap.cc:492] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:36.589659 13874 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.592374 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:36.593401 14024 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:36.560719 13938 tablet_service.cc:1511] Processing CreateTablet for tablet 9bfd3f44e83e4908b229bb57863e4080 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:36.594985 13938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bfd3f44e83e4908b229bb57863e4080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.596694 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:36.598798 14093 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:36.598871 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:36.599500 14093 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.600116 14093 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:36.600544 13915 leader_election.cc:343] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.602170 14065 tablet_bootstrap.cc:492] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:36.602744 14065 ts_tablet_manager.cc:1403] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.103s	user 0.010s	sys 0.002s
I20260709 13:03:36.604775 14066 tablet_bootstrap.cc:654] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.608134 14010 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8599b3d11641199dc0043a592857fb (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:36.609979 14010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8599b3d11641199dc0043a592857fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.614004 13874 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:36.609794 14065 raft_consensus.cc:359] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.635812 14065 raft_consensus.cc:385] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.635591 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:36.636371 14065 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.635634 13865 tablet_service.cc:1511] Processing CreateTablet for tablet f9158487769746e2a7f4f283dda5abf3 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:36.637946 14102 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:36.638449 14102 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:36.638924 13865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9158487769746e2a7f4f283dda5abf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.639295 14102 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.621637 14012 tablet_service.cc:1511] Processing CreateTablet for tablet 9a2c65bca5ce45d5af2ace962fa0a37f (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:36.623054 13935 tablet_service.cc:1511] Processing CreateTablet for tablet 617d39d9397c4724be821e5efacbde17 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:36.642988 14012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a2c65bca5ce45d5af2ace962fa0a37f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.643375 13935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 617d39d9397c4724be821e5efacbde17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.644495 14066 tablet_bootstrap.cc:492] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:36.644968 14066 ts_tablet_manager.cc:1403] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.061s	user 0.009s	sys 0.003s
I20260709 13:03:36.626874 14013 tablet_service.cc:1511] Processing CreateTablet for tablet 617d39d9397c4724be821e5efacbde17 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:36.648064 14066 raft_consensus.cc:359] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.648574 14013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 617d39d9397c4724be821e5efacbde17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.648751 14066 raft_consensus.cc:385] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.649077 14066 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.649780 14066 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.630569 13934 tablet_service.cc:1511] Processing CreateTablet for tablet 10b028ec30974d06bd9da921c268a641 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:36.652014 14066 ts_tablet_manager.cc:1434] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:03:36.652448 13934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10b028ec30974d06bd9da921c268a641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.653025 14066 tablet_bootstrap.cc:492] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:36.631529 14014 tablet_service.cc:1511] Processing CreateTablet for tablet 10b028ec30974d06bd9da921c268a641 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:36.632086 13858 tablet_service.cc:1511] Processing CreateTablet for tablet 10b028ec30974d06bd9da921c268a641 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:36.654892 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10b028ec30974d06bd9da921c268a641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.654949 14014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10b028ec30974d06bd9da921c268a641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.604907 13939 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8599b3d11641199dc0043a592857fb (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:36.633795 13937 tablet_service.cc:1511] Processing CreateTablet for tablet f9158487769746e2a7f4f283dda5abf3 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:36.663090 13937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9158487769746e2a7f4f283dda5abf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.633843 13863 tablet_service.cc:1511] Processing CreateTablet for tablet 617d39d9397c4724be821e5efacbde17 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:36.637116 14065 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.667026 13863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 617d39d9397c4724be821e5efacbde17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.667734 14065 ts_tablet_manager.cc:1434] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.001s
I20260709 13:03:36.669255 14065 tablet_bootstrap.cc:492] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:36.672349 14085 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.672863 14085 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.619134 13940 tablet_service.cc:1511] Processing CreateTablet for tablet 9a2c65bca5ce45d5af2ace962fa0a37f (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:36.689462 13940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a2c65bca5ce45d5af2ace962fa0a37f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.682652 14085 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:36.683763 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:03:36.697902 13838 leader_election.cc:343] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.684348 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:36.686527 14066 tablet_bootstrap.cc:654] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.689517 14008 tablet_service.cc:1511] Processing CreateTablet for tablet f9158487769746e2a7f4f283dda5abf3 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:36.690982 14089 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.699083 14089 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:36.699692 13840 leader_election.cc:343] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.700145 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:36.701350 14089 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:36.701627 14085 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:36.702723 14008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9158487769746e2a7f4f283dda5abf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.703682 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:36.704324 14007 tablet_service.cc:1511] Processing CreateTablet for tablet 56605211bd584b8889b54049e9f6cb4b (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:36.705686 14007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56605211bd584b8889b54049e9f6cb4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.704347 14022 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:36.693194 13938 tablet_service.cc:1511] Processing CreateTablet for tablet 56605211bd584b8889b54049e9f6cb4b (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:36.695596 13858 tablet_service.cc:1511] Processing CreateTablet for tablet 56605211bd584b8889b54049e9f6cb4b (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:36.706944 13936 tablet_service.cc:1511] Processing CreateTablet for tablet 728de4a891d847028775aa5c35ba53e1 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:36.707321 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56605211bd584b8889b54049e9f6cb4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.707523 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:36.708523 14089 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:36.709075 14089 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.709581 14089 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.712085 14009 tablet_service.cc:1511] Processing CreateTablet for tablet 728de4a891d847028775aa5c35ba53e1 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:36.713316 14009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 728de4a891d847028775aa5c35ba53e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.720359 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:36.721131 13874 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:36.721330 13936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 728de4a891d847028775aa5c35ba53e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.722361 13939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8599b3d11641199dc0043a592857fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.722978 13860 tablet_service.cc:1511] Processing CreateTablet for tablet 728de4a891d847028775aa5c35ba53e1 (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:36.724551 13860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 728de4a891d847028775aa5c35ba53e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.723883 13938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56605211bd584b8889b54049e9f6cb4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.696105 14065 tablet_bootstrap.cc:654] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.679594 13861 tablet_service.cc:1511] Processing CreateTablet for tablet 9a2c65bca5ce45d5af2ace962fa0a37f (DEFAULT_TABLE table=test-workload [id=23a417905cf742398ff129309a2ade2b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:36.727982 13861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a2c65bca5ce45d5af2ace962fa0a37f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.746529 14085 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.747337 14085 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.808064 13773 catalog_manager.cc:5697] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:36.817068 14089 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.821973 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:03:36.823733 13838 leader_election.cc:343] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.826233 14102 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.826809 14102 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.829823 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:36.830549 13874 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:36.831868 13989 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:36.832788 14100 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:36.832873 14064 tablet_bootstrap.cc:492] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:36.833209 14100 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.833595 14100 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.833642 14064 ts_tablet_manager.cc:1403] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.268s	user 0.041s	sys 0.041s
I20260709 13:03:36.834250 14085 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:36.837039 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:36.829597 14093 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:36.848078 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 114212us
I20260709 13:03:36.849187 14093 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:36.852669 14102 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:36.854279 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:36.854828 13949 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
W20260709 13:03:36.856101 13821 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:03:36.856103 13987 leader_election.cc:343] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.860240 14100 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.862509 14022 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:36.862568 14100 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:36.865279 14066 tablet_bootstrap.cc:492] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:36.865787 14066 ts_tablet_manager.cc:1403] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.213s	user 0.016s	sys 0.004s
I20260709 13:03:36.866321 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:36.866931 13874 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.867015 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:36.867601 14089 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:36.867002 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:03:36.869722 13987 leader_election.cc:343] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.871104 14064 raft_consensus.cc:359] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.871866 14064 raft_consensus.cc:385] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.872159 14064 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.872865 14064 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.880839 14066 raft_consensus.cc:359] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.881661 14066 raft_consensus.cc:385] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.884330 14065 tablet_bootstrap.cc:492] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:36.884891 14065 ts_tablet_manager.cc:1403] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.216s	user 0.009s	sys 0.009s
I20260709 13:03:36.892155 14065 raft_consensus.cc:359] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.892935 14065 raft_consensus.cc:385] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.893232 14065 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.881955 14066 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.898265 14066 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.903160 14064 ts_tablet_manager.cc:1434] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:03:36.903353 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:36.903991 13875 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.906181 14065 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.908695 14066 ts_tablet_manager.cc:1434] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:03:36.909711 14066 tablet_bootstrap.cc:492] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:36.915935 14064 tablet_bootstrap.cc:492] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:36.916432 13875 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:36.928655 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:36.929379 13873 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.931665 14073 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:36.934863 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:36.935613 14065 ts_tablet_manager.cc:1434] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 13:03:36.935461 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:36.936240 14024 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.937739 13873 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:36.938221 13874 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:36.935546 14022 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.951143 14085 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:36.952606 13989 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:03:36.944085 14024 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:36.945339 14065 tablet_bootstrap.cc:492] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:36.960795 14022 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:36.964913 14066 tablet_bootstrap.cc:654] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.965497 14065 tablet_bootstrap.cc:654] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.970531 14064 tablet_bootstrap.cc:654] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.968168 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:36.975602 14093 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:36.976194 14093 raft_consensus.cc:697] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:36.977593 14093 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.969509 14135 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:36.983150 14135 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:36.984342 14135 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:36.990392 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:36.997768 14089 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:36.998579 14089 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:36.999868 14089 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.017475 14065 tablet_bootstrap.cc:492] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:37.018026 14065 ts_tablet_manager.cc:1403] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.014s
I20260709 13:03:37.022374 14065 raft_consensus.cc:359] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.025444 14065 raft_consensus.cc:385] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.026190 14065 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.030683 14066 tablet_bootstrap.cc:492] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:37.031106 14065 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.031816 14066 ts_tablet_manager.cc:1403] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.122s	user 0.015s	sys 0.006s
I20260709 13:03:37.034790 14065 ts_tablet_manager.cc:1434] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 13:03:37.036216 14065 tablet_bootstrap.cc:492] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:37.037631 14073 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:37.047333 14066 raft_consensus.cc:359] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.048374 14066 raft_consensus.cc:385] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.048746 14066 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.050886 14066 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.060456 14065 tablet_bootstrap.cc:654] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.068034 13771 catalog_manager.cc:5697] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:37.072440 13773 catalog_manager.cc:5697] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:37.073065 14066 ts_tablet_manager.cc:1434] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.001s
I20260709 13:03:37.076251 14066 tablet_bootstrap.cc:492] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:37.077986 13771 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:37.087558 14064 tablet_bootstrap.cc:492] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:37.089834 14064 ts_tablet_manager.cc:1403] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.174s	user 0.010s	sys 0.015s
I20260709 13:03:37.092166 14066 tablet_bootstrap.cc:654] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.105938 14064 raft_consensus.cc:359] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.107445 14064 raft_consensus.cc:385] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.107872 14064 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.108819 14064 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.115697 14064 ts_tablet_manager.cc:1434] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.002s
I20260709 13:03:37.118710 14064 tablet_bootstrap.cc:492] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:37.120223 14073 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.120756 14073 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.123448 14135 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:37.124666 14073 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:37.126804 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:37.127442 14022 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:37.129220 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:37.131732 13915 leader_election.cc:343] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:37.132970 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:37.134430 14089 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:37.134826 14089 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.135190 14089 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.141804 13872 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.145020 14065 tablet_bootstrap.cc:492] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:37.145713 14065 ts_tablet_manager.cc:1403] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.110s	user 0.023s	sys 0.031s
I20260709 13:03:37.149730 14065 raft_consensus.cc:359] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.150719 14065 raft_consensus.cc:385] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.151706 14065 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.152549 14065 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.160266 14065 ts_tablet_manager.cc:1434] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 13:03:37.164275 14135 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:37.164755 14089 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.180111 14089 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:37.182415 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:37.182801 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:37.183696 14022 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:37.183986 13915 leader_election.cc:343] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:37.186585 14064 tablet_bootstrap.cc:654] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.192325 14065 tablet_bootstrap.cc:492] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:37.210251 14066 tablet_bootstrap.cc:492] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:37.210960 14066 ts_tablet_manager.cc:1403] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.135s	user 0.037s	sys 0.030s
I20260709 13:03:37.212721 14065 tablet_bootstrap.cc:654] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.215003 14066 raft_consensus.cc:359] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.215873 14066 raft_consensus.cc:385] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.216181 14066 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.217075 14066 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.219352 14066 ts_tablet_manager.cc:1434] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:03:37.220459 14066 tablet_bootstrap.cc:492] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:37.226317 14064 tablet_bootstrap.cc:492] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:37.226794 14073 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.226981 14064 ts_tablet_manager.cc:1403] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.109s	user 0.009s	sys 0.005s
I20260709 13:03:37.227367 14073 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.233664 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:37.234645 14024 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:37.234930 14022 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:37.239450 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:37.240074 13872 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:37.241384 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:37.243080 14089 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:37.243605 14089 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.244078 14089 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.249615 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:37.255591 14066 tablet_bootstrap.cc:654] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.241395 14064 raft_consensus.cc:359] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.256248 14064 raft_consensus.cc:385] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.256569 14064 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.237506 14073 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:37.258265 14144 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:37.258826 14144 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:37.259699 14144 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.271948 14065 tablet_bootstrap.cc:492] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:37.272536 14065 ts_tablet_manager.cc:1403] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.084s	user 0.020s	sys 0.026s
I20260709 13:03:37.275666 14065 raft_consensus.cc:359] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.276476 14065 raft_consensus.cc:385] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.276754 14064 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.280241 14089 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.276753 14065 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.283329 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:37.283903 13872 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.284353 14064 ts_tablet_manager.cc:1434] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.057s	user 0.004s	sys 0.003s
I20260709 13:03:37.285486 14064 tablet_bootstrap.cc:492] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:37.290410 14065 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.292346 13872 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:37.299218 14065 ts_tablet_manager.cc:1434] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 13:03:37.300271 14065 tablet_bootstrap.cc:492] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:37.303220 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:37.304366 14073 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:37.305141 14073 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:37.306120 14073 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.308002 14064 tablet_bootstrap.cc:654] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.311798 14065 tablet_bootstrap.cc:654] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.314877 14089 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:37.318374 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:37.319348 14065 tablet_bootstrap.cc:492] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:37.319216 14022 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.324774 13773 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:37.331364 14085 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.331977 14085 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.334254 14085 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:37.336700 14066 tablet_bootstrap.cc:492] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:37.337317 14066 ts_tablet_manager.cc:1403] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.117s	user 0.038s	sys 0.014s
I20260709 13:03:37.338444 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:37.339382 14024 raft_consensus.cc:2393] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:37.336529 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:37.341473 14066 raft_consensus.cc:359] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.342355 14066 raft_consensus.cc:385] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.342167 13838 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:37.342758 14066 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.343673 14066 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.346231 14066 ts_tablet_manager.cc:1434] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:03:37.346701 14065 ts_tablet_manager.cc:1403] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.003s
I20260709 13:03:37.347522 14066 tablet_bootstrap.cc:492] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:37.348660 14085 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.349833 14065 raft_consensus.cc:359] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.350574 14065 raft_consensus.cc:385] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.350848 14065 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.351472 14022 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:37.355080 14066 tablet_bootstrap.cc:654] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.358829 14064 tablet_bootstrap.cc:492] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:37.359913 14064 ts_tablet_manager.cc:1403] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.075s	user 0.007s	sys 0.012s
I20260709 13:03:37.359767 13773 catalog_manager.cc:5697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:37.356006 14065 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.364127 14065 ts_tablet_manager.cc:1434] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260709 13:03:37.365368 14064 raft_consensus.cc:359] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.366192 14064 raft_consensus.cc:385] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.366293 14085 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:37.366500 14064 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.369905 14065 tablet_bootstrap.cc:492] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:37.376367 14064 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.378834 14064 ts_tablet_manager.cc:1434] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:03:37.379796 14064 tablet_bootstrap.cc:492] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:37.394519 14135 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.395145 14135 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.398864 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:37.399631 13950 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:37.401808 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:37.402021 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:37.402940 14102 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:37.403309 14102 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.403663 14102 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:37.404975 13989 leader_election.cc:343] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:37.410240 14135 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:37.411554 14065 tablet_bootstrap.cc:654] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.412847 14102 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.416857 14064 tablet_bootstrap.cc:654] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.416994 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:03:37.418445 13989 leader_election.cc:343] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:37.421048 14066 tablet_bootstrap.cc:492] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:37.421545 14066 ts_tablet_manager.cc:1403] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.074s	user 0.043s	sys 0.020s
I20260709 13:03:37.422031 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:37.422704 13950 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.424774 14066 raft_consensus.cc:359] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.425446 14066 raft_consensus.cc:385] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.425796 14066 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.426649 14066 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.432207 13950 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:37.422207 14102 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:37.434247 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:37.435495 14102 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:37.436890 14102 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:37.437773 14102 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.439826 14066 ts_tablet_manager.cc:1434] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.004s
I20260709 13:03:37.441144 14066 tablet_bootstrap.cc:492] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:37.447963 14066 tablet_bootstrap.cc:654] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.453941 14066 tablet_bootstrap.cc:492] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:37.453670 13773 catalog_manager.cc:5697] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:37.454689 14066 ts_tablet_manager.cc:1403] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.003s
I20260709 13:03:37.458184 14066 raft_consensus.cc:359] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.458972 14066 raft_consensus.cc:385] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.459311 14066 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.459882 14065 tablet_bootstrap.cc:492] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:37.460041 14066 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.460387 14065 ts_tablet_manager.cc:1403] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.091s	user 0.012s	sys 0.002s
I20260709 13:03:37.466607 14066 ts_tablet_manager.cc:1434] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:03:37.466544 14065 raft_consensus.cc:359] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.467506 14065 raft_consensus.cc:385] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.467505 14102 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.467803 14064 tablet_bootstrap.cc:492] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:37.468125 14065 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.468505 14064 ts_tablet_manager.cc:1403] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.089s	user 0.014s	sys 0.005s
I20260709 13:03:37.468271 14102 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.469421 14065 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.471702 14065 ts_tablet_manager.cc:1434] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.007s
I20260709 13:03:37.473188 14065 tablet_bootstrap.cc:492] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:37.476889 14064 raft_consensus.cc:359] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.477638 14064 raft_consensus.cc:385] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.477917 14064 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.478868 14066 tablet_bootstrap.cc:492] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:37.478662 14064 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.478981 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:37.479673 13873 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:37.481096 14065 tablet_bootstrap.cc:654] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.481132 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:37.482749 14135 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:37.483139 14135 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.483515 14135 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.486323 14089 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.486572 14066 tablet_bootstrap.cc:654] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.486858 14064 ts_tablet_manager.cc:1434] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:03:37.486833 14089 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.488008 14064 tablet_bootstrap.cc:492] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:37.489198 14089 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:37.492743 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:37.493513 14022 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:37.494704 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:37.495411 13872 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:37.497272 14064 tablet_bootstrap.cc:654] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.497087 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:37.498728 14142 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:37.499249 14142 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.499711 14142 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.505403 14135 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.508683 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:37.509413 13873 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.513033 14144 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:37.518291 14144 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:37.520233 14102 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:37.522666 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:37.523644 13950 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:37.534128 14144 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:37.536048 14065 tablet_bootstrap.cc:492] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:37.536832 14065 ts_tablet_manager.cc:1403] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.003s
I20260709 13:03:37.561054 14100 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.562584 14135 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:37.562803 14100 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.564055 13873 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:37.565210 14100 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:37.569895 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:37.570742 13949 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.570771 13950 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.584529 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:37.586424 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:37.586820 14100 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:37.587373 14100 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:37.588239 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:37.588326 14100 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.587393 14144 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:03:37.592594 13987 leader_election.cc:343] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:37.597211 14064 tablet_bootstrap.cc:492] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:37.597330 14066 tablet_bootstrap.cc:492] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:37.597967 14064 ts_tablet_manager.cc:1403] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.110s	user 0.023s	sys 0.032s
I20260709 13:03:37.598112 14066 ts_tablet_manager.cc:1403] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.119s	user 0.050s	sys 0.047s
W20260709 13:03:37.603843 13989 leader_election.cc:343] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:37.604156 13949 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:37.607144 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:37.617480 14134 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:37.607314 14102 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:03:37.584285 14142 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.623657 14142 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:37.628693 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:37.640084 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:37.641023 14023 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.645571 14064 raft_consensus.cc:359] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.646395 14064 raft_consensus.cc:385] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.646678 14064 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.647221 14065 raft_consensus.cc:359] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.647373 14064 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.647998 14065 raft_consensus.cc:385] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.648330 14065 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.649035 14065 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.649632 14064 ts_tablet_manager.cc:1434] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.051s	user 0.004s	sys 0.000s
I20260709 13:03:37.651240 14023 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:37.652999 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:37.629396 13873 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.654910 14064 tablet_bootstrap.cc:492] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:37.675045 14170 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:37.654074 14142 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:37.691040 13873 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:37.693094 14065 ts_tablet_manager.cc:1434] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.135s	user 0.004s	sys 0.016s
I20260709 13:03:37.693058 14066 raft_consensus.cc:359] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.693789 14066 raft_consensus.cc:385] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.694015 14066 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.694759 14066 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.696120 13770 catalog_manager.cc:5697] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:37.697127 14066 ts_tablet_manager.cc:1434] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.098s	user 0.011s	sys 0.010s
I20260709 13:03:37.697340 14065 tablet_bootstrap.cc:492] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:37.714942 14066 tablet_bootstrap.cc:492] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:37.719599 14064 tablet_bootstrap.cc:654] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.683369 14142 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:37.732285 14066 tablet_bootstrap.cc:654] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.734030 14144 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:37.724380 14142 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.749277 14065 tablet_bootstrap.cc:654] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.756191 14073 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:37.765879 14066 tablet_bootstrap.cc:492] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:37.766476 14066 ts_tablet_manager.cc:1403] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.052s	user 0.009s	sys 0.013s
I20260709 13:03:37.767932 14064 tablet_bootstrap.cc:492] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:37.768535 14064 ts_tablet_manager.cc:1403] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.114s	user 0.016s	sys 0.006s
I20260709 13:03:37.769392 14066 raft_consensus.cc:359] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.770177 14066 raft_consensus.cc:385] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.770535 14066 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.771297 14066 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.774773 14064 raft_consensus.cc:359] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.776611 14064 raft_consensus.cc:385] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.776960 14064 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.777725 14064 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.779939 14064 ts_tablet_manager.cc:1434] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:03:37.780817 14066 ts_tablet_manager.cc:1434] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.005s
I20260709 13:03:37.781199 14064 tablet_bootstrap.cc:492] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:37.781803 14066 tablet_bootstrap.cc:492] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:37.787393 13773 catalog_manager.cc:5697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:37.783129 14065 tablet_bootstrap.cc:492] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:37.789230 14065 ts_tablet_manager.cc:1403] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.095s	user 0.016s	sys 0.017s
I20260709 13:03:37.792440 14073 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:37.793519 14065 raft_consensus.cc:359] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.794286 14065 raft_consensus.cc:385] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.794502 14073 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:37.800009 14065 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.800995 14065 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.803525 14065 ts_tablet_manager.cc:1434] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:03:37.809367 14065 tablet_bootstrap.cc:492] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:37.810585 14066 tablet_bootstrap.cc:654] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.799611 14064 tablet_bootstrap.cc:654] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.819039 14065 tablet_bootstrap.cc:654] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.835062 14065 tablet_bootstrap.cc:492] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:37.835970 14065 ts_tablet_manager.cc:1403] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.000s
I20260709 13:03:37.838922 14065 raft_consensus.cc:359] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.839645 14065 raft_consensus.cc:385] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.839924 14065 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.840750 14065 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.842401 14066 tablet_bootstrap.cc:492] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:37.842936 14066 ts_tablet_manager.cc:1403] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.001s
I20260709 13:03:37.843120 14065 ts_tablet_manager.cc:1434] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:03:37.844144 14065 tablet_bootstrap.cc:492] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:37.845815 14066 raft_consensus.cc:359] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.846592 14066 raft_consensus.cc:385] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.846946 14066 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.848274 14066 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.850426 14065 tablet_bootstrap.cc:654] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.850435 14066 ts_tablet_manager.cc:1434] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:37.851626 14066 tablet_bootstrap.cc:492] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:37.859246 14073 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.859799 14073 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.861972 14073 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:37.862267 14064 tablet_bootstrap.cc:492] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:37.864475 14064 ts_tablet_manager.cc:1403] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.084s	user 0.009s	sys 0.010s
I20260709 13:03:37.865190 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:37.865180 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:37.866012 13874 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:37.866021 14022 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:37.867537 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:37.867852 14064 raft_consensus.cc:359] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.868695 14064 raft_consensus.cc:385] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.868979 14064 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.869498 14073 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:37.869647 14064 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.869951 14073 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.870512 14073 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.871996 14064 ts_tablet_manager.cc:1434] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:37.873239 14064 tablet_bootstrap.cc:492] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742: Bootstrap starting.
W20260709 13:03:37.874612 13821 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:37.876873 14066 tablet_bootstrap.cc:654] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.877933 14073 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.879451 14064 tablet_bootstrap.cc:654] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.881284 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:37.882104 13874 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.887569 14066 tablet_bootstrap.cc:492] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:37.888168 14066 ts_tablet_manager.cc:1403] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.007s
I20260709 13:03:37.891156 14066 raft_consensus.cc:359] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.892019 14066 raft_consensus.cc:385] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.891944 13874 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:37.892462 14066 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.893603 14065 tablet_bootstrap.cc:492] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:37.893319 14066 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.893548 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:37.894260 14065 ts_tablet_manager.cc:1403] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.001s
I20260709 13:03:37.894675 14089 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:37.895171 14089 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:37.895304 14064 tablet_bootstrap.cc:492] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:37.896205 14073 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:37.896308 14064 ts_tablet_manager.cc:1403] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.010s
I20260709 13:03:37.896304 14066 ts_tablet_manager.cc:1434] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:03:37.897924 14089 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.898208 14065 raft_consensus.cc:359] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.898867 14065 raft_consensus.cc:385] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.899158 14065 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.899130 14064 raft_consensus.cc:359] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.899912 14064 raft_consensus.cc:385] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.900017 14066 tablet_bootstrap.cc:492] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:37.900228 14064 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.900151 14065 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.902897 14065 ts_tablet_manager.cc:1434] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:03:37.904367 14065 tablet_bootstrap.cc:492] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:37.906868 14066 tablet_bootstrap.cc:654] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.910790 14065 tablet_bootstrap.cc:654] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.901158 14064 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.915457 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:37.916235 14022 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.916469 14064 ts_tablet_manager.cc:1434] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.002s
I20260709 13:03:37.918646 14064 tablet_bootstrap.cc:492] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:37.924134 14065 tablet_bootstrap.cc:492] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:37.924654 14065 ts_tablet_manager.cc:1403] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 13:03:37.927044 14022 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:37.928719 14066 tablet_bootstrap.cc:492] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:37.929208 14066 ts_tablet_manager.cc:1403] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.004s
I20260709 13:03:37.930645 14065 raft_consensus.cc:359] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.931710 14065 raft_consensus.cc:385] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.932061 14065 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.932982 14065 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.936767 14064 tablet_bootstrap.cc:654] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.937163 14066 raft_consensus.cc:359] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.937881 14066 raft_consensus.cc:385] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.938234 14066 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.945820 14065 ts_tablet_manager.cc:1434] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 13:03:37.947883 13771 catalog_manager.cc:5697] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:37.938880 14066 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.951946 14066 ts_tablet_manager.cc:1434] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.004s
I20260709 13:03:37.953446 14066 tablet_bootstrap.cc:492] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:37.955389 14065 tablet_bootstrap.cc:492] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:37.961862 14066 tablet_bootstrap.cc:654] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.961957 14065 tablet_bootstrap.cc:654] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.968713 14064 tablet_bootstrap.cc:492] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:37.969515 14064 ts_tablet_manager.cc:1403] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.051s	user 0.019s	sys 0.010s
I20260709 13:03:37.969784 14066 tablet_bootstrap.cc:492] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:37.970306 14066 ts_tablet_manager.cc:1403] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 13:03:37.972842 14066 raft_consensus.cc:359] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.973460 14066 raft_consensus.cc:385] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.973773 14066 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.973619 14064 raft_consensus.cc:359] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.974473 14064 raft_consensus.cc:385] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.974807 14064 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.974625 14066 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.975503 14064 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.976168 14065 tablet_bootstrap.cc:492] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:37.976645 14065 ts_tablet_manager.cc:1403] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.006s
I20260709 13:03:37.976749 14066 ts_tablet_manager.cc:1434] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:03:37.978394 14064 ts_tablet_manager.cc:1434] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:03:37.979322 14066 tablet_bootstrap.cc:492] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:37.979444 14065 raft_consensus.cc:359] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.980136 14065 raft_consensus.cc:385] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.980422 14065 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.981503 14064 tablet_bootstrap.cc:492] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:37.981106 14065 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:37.986924 14066 tablet_bootstrap.cc:654] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.992012 14064 tablet_bootstrap.cc:654] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.992471 14065 ts_tablet_manager.cc:1434] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:03:37.993523 14065 tablet_bootstrap.cc:492] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:38.002336 14066 tablet_bootstrap.cc:492] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:38.003142 14065 tablet_bootstrap.cc:654] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.004357 14066 ts_tablet_manager.cc:1403] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.004s
I20260709 13:03:38.006309 14064 tablet_bootstrap.cc:492] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:38.006793 14064 ts_tablet_manager.cc:1403] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.007s
I20260709 13:03:38.008266 14066 raft_consensus.cc:359] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.009266 14066 raft_consensus.cc:385] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.009634 14066 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.010004 14064 raft_consensus.cc:359] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.010785 14064 raft_consensus.cc:385] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.011299 14064 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.010740 14066 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.011991 14064 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.016059 14064 ts_tablet_manager.cc:1434] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 13:03:38.017170 14064 tablet_bootstrap.cc:492] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:38.021345 14142 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.022112 14142 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.024544 14142 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.025385 14064 tablet_bootstrap.cc:654] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.025354 14170 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:38.027932 14066 ts_tablet_manager.cc:1434] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:03:38.039350 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:38.040485 14022 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:38.043829 14066 tablet_bootstrap.cc:492] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:38.049640 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:38.050983 13874 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:38.053109 14066 tablet_bootstrap.cc:654] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.053131 14170 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.055209 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:38.057426 14073 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:38.058046 14073 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.058564 14073 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.059016 14065 tablet_bootstrap.cc:492] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:38.059803 14065 ts_tablet_manager.cc:1403] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.066s	user 0.021s	sys 0.032s
I20260709 13:03:38.067001 14065 raft_consensus.cc:359] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.068077 14065 raft_consensus.cc:385] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.068425 14065 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.069574 13872 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.069432 14065 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.072595 14065 ts_tablet_manager.cc:1434] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:03:38.073649 14065 tablet_bootstrap.cc:492] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:38.074229 14170 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.079568 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:38.080745 14085 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.081260 14085 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:38.081697 13989 leader_election.cc:343] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.083348 14085 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.084903 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:03:38.086401 13838 leader_election.cc:343] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.092089 14170 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:38.093824 14066 tablet_bootstrap.cc:492] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:38.094363 14066 ts_tablet_manager.cc:1403] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.015s
I20260709 13:03:38.096928 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:38.097808 13949 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:38.098014 14102 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:38.099480 14102 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.099884 14066 raft_consensus.cc:359] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.101198 14102 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.101508 14066 raft_consensus.cc:385] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.101843 14065 tablet_bootstrap.cc:654] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.101955 14066 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.102731 14066 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.103600 14102 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:38.105332 14066 ts_tablet_manager.cc:1434] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:03:38.106359 14066 tablet_bootstrap.cc:492] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:38.108106 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:38.110942 13987 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:38.112879 14100 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:38.113481 14100 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.113905 14100 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.116883 14066 tablet_bootstrap.cc:654] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.117849 14134 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.118343 14134 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:38.119659 13989 leader_election.cc:343] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.120395 14134 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:38.099537 14073 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.124598 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:38.125599 13873 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:38.099450 14085 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.143129 14065 tablet_bootstrap.cc:492] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:38.142897 14085 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.143872 14065 ts_tablet_manager.cc:1403] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.070s	user 0.011s	sys 0.000s
I20260709 13:03:38.144816 14064 tablet_bootstrap.cc:492] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:38.145329 14064 ts_tablet_manager.cc:1403] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.128s	user 0.055s	sys 0.054s
I20260709 13:03:38.146160 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:38.146775 13950 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:38.148005 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:38.148082 14064 raft_consensus.cc:359] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.126847 14073 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.129891 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:38.136888 14134 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:03:38.149369 13872 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.137941 14142 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.150961 14142 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.151659 14085 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.127228 14202 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.154693 14085 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:38.154400 14202 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.158092 13872 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:38.148828 14064 raft_consensus.cc:385] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.158990 14064 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.159411 14202 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.159850 14064 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.160176 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:38.161353 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:38.161438 14089 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:38.162143 14022 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:38.162791 14064 ts_tablet_manager.cc:1434] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.001s
I20260709 13:03:38.162210 14089 raft_consensus.cc:697] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:38.160447 14085 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.163767 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:38.164849 14085 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.166906 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:38.167351 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:38.168010 14024 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:38.173148 14170 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:38.173668 14170 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.169135 14065 raft_consensus.cc:359] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.174366 14065 raft_consensus.cc:385] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.174119 14170 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.165292 14064 tablet_bootstrap.cc:492] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:38.164849 13872 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:38.153810 14142 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.174707 14065 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.165343 14089 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.176636 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:38.176784 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:38.177783 14142 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:38.178239 14142 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.178593 14142 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.179554 14065 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.180449 14100 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.184003 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:38.187569 14142 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.197755 14065 ts_tablet_manager.cc:1434] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.053s	user 0.009s	sys 0.000s
I20260709 13:03:38.198201 13950 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:38.199668 14142 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.188160 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:38.189653 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:38.200793 13948 raft_consensus.cc:2393] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:38.193706 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:03:38.204739 14100 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:38.205472 14206 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:38.210651 14065 tablet_bootstrap.cc:492] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:38.197011 14205 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.211223 14205 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.206506 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:38.212577 13872 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.214865 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:38.215528 13949 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:38.216938 14065 tablet_bootstrap.cc:654] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.216979 13838 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:38.217900 14202 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:38.218276 14202 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.218683 14202 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.222805 14064 tablet_bootstrap.cc:654] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.224184 14066 tablet_bootstrap.cc:492] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:38.224188 14089 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:38.225018 14066 ts_tablet_manager.cc:1403] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.119s	user 0.017s	sys 0.034s
I20260709 13:03:38.224313 13772 catalog_manager.cc:5697] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:38.206401 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:38.210765 14206 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.208844 14085 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.229198 14206 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.219580 14205 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.236846 14085 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.237828 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:38.238499 13949 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.239547 14066 raft_consensus.cc:359] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.240541 14066 raft_consensus.cc:385] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.240890 14066 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.226192 14100 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:03:38.231313 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:38.242240 13872 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:38.242098 14066 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.259965 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:38.260385 13949 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:38.260807 14066 ts_tablet_manager.cc:1434] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.003s
I20260709 13:03:38.261476 14089 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:38.261999 14089 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:38.261746 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:38.264325 14066 tablet_bootstrap.cc:492] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:38.262910 14085 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:38.235256 14205 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:38.262902 14089 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.269551 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:38.269678 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:38.271979 14024 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.266391 14102 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:38.272675 14102 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.273200 14102 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:38.276753 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 70434.3us
I20260709 13:03:38.268736 14085 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:03:38.282917 14085 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.294893 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:38.302956 14202 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.325559 14024 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:38.327441 14066 tablet_bootstrap.cc:654] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.337802 14065 tablet_bootstrap.cc:492] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:38.338536 14065 ts_tablet_manager.cc:1403] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.128s	user 0.008s	sys 0.025s
I20260709 13:03:38.328644 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:38.339949 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:38.347306 13949 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.361775 14065 raft_consensus.cc:359] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.367199 14065 raft_consensus.cc:385] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.367627 14065 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.349211 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:38.369359 14065 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.356998 13948 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.360411 14202 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.379493 14073 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:38.380942 14206 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.368993 14024 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:38.370678 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:38.383998 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:38.404520 14206 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.404071 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:38.399097 14021 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.408845 14023 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.406455 14102 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.436604 14102 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:38.413726 14170 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:38.447965 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 79874.4us
I20260709 13:03:38.449308 14022 raft_consensus.cc:2393] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:38.444916 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:38.451771 14024 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.453415 14102 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.460824 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:38.476230 13772 catalog_manager.cc:5697] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:38.475839 13946 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.468894 14102 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.504941 13771 catalog_manager.cc:5697] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:38.506908 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:38.507704 13945 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:38.508116 13989 leader_election.cc:343] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.511106 14065 ts_tablet_manager.cc:1434] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.172s	user 0.017s	sys 0.009s
I20260709 13:03:38.512825 14065 tablet_bootstrap.cc:492] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:38.518754 14222 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.521386 14066 tablet_bootstrap.cc:492] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:38.521831 14066 ts_tablet_manager.cc:1403] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.258s	user 0.011s	sys 0.013s
I20260709 13:03:38.524997 14064 tablet_bootstrap.cc:492] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:38.525388 14064 ts_tablet_manager.cc:1403] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.360s	user 0.037s	sys 0.017s
I20260709 13:03:38.526178 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:38.528483 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:38.533286 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:38.534288 13873 raft_consensus.cc:2393] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:38.524050 14222 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.535130 14064 raft_consensus.cc:359] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.539964 14064 raft_consensus.cc:385] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.541203 14064 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.535858 14024 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:38.520517 14023 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:38.536284 14222 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.537374 14170 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:38.539204 14066 raft_consensus.cc:359] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.544495 14066 raft_consensus.cc:385] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.544806 14066 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.545444 14066 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.547098 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:38.547981 14066 ts_tablet_manager.cc:1434] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:03:38.547936 14206 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:38.548581 14206 raft_consensus.cc:697] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:03:38.549079 14066 tablet_bootstrap.cc:492] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:38.549831 14021 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:38.549479 14206 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.548079 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:38.552448 14202 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:38.552778 14102 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:38.554212 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:38.556721 13950 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:38.553040 14202 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:03:38.558820 14064 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.562744 14202 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.552699 13949 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
W20260709 13:03:38.553093 13989 leader_election.cc:343] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.573575 13987 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:38.555624 13948 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:38.586938 13946 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:38.587236 13945 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:38.588096 14227 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:38.588191 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:38.590266 14134 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:38.590809 14134 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:38.590377 14064 ts_tablet_manager.cc:1434] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.003s
I20260709 13:03:38.592969 14064 tablet_bootstrap.cc:492] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:38.593860 14134 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.599721 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:38.603232 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:38.588698 14227 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.606374 14227 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.611908 13987 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:38.613617 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:38.615918 14100 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:38.627761 14066 tablet_bootstrap.cc:654] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.628996 14221 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:03:38.631773 14228 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:38.624352 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:38.622388 14100 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:38.646662 14100 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.648118 13772 catalog_manager.cc:5697] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:38.649392 13772 catalog_manager.cc:5697] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:38.634620 13915 leader_election.cc:343] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.675361 14065 tablet_bootstrap.cc:654] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:38.663081 13989 leader_election.cc:343] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.683792 14064 tablet_bootstrap.cc:654] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.689043 13770 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:38.692257 14224 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:38.691259 13770 catalog_manager.cc:5697] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:38.700420 14227 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.702096 14021 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:38.719285 14227 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:38.719905 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:38.723904 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:38.724851 13948 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.728442 14064 tablet_bootstrap.cc:492] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:38.730119 14064 ts_tablet_manager.cc:1403] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.137s	user 0.011s	sys 0.006s
W20260709 13:03:38.733353 13989 leader_election.cc:343] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.739146 13948 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:38.740861 13987 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:38.741792 14227 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:38.742360 14227 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:38.743055 14227 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.743212 14064 raft_consensus.cc:359] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.743974 14064 raft_consensus.cc:385] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.744256 14064 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.744957 14064 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.747546 14065 tablet_bootstrap.cc:492] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:38.748121 14065 ts_tablet_manager.cc:1403] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.236s	user 0.012s	sys 0.017s
I20260709 13:03:38.750806 14066 tablet_bootstrap.cc:492] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:38.750968 13915 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:38.751394 14066 ts_tablet_manager.cc:1403] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.203s	user 0.012s	sys 0.017s
I20260709 13:03:38.751905 14222 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:38.751969 14065 raft_consensus.cc:359] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.752980 14065 raft_consensus.cc:385] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.753286 14065 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.754027 14065 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.754479 14222 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:38.756908 14065 ts_tablet_manager.cc:1434] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:03:38.756793 14066 raft_consensus.cc:359] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.757625 14066 raft_consensus.cc:385] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.757944 14066 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.758082 14065 tablet_bootstrap.cc:492] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:38.760462 14222 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.761389 14222 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.763898 14222 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.760089 14064 ts_tablet_manager.cc:1434] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:03:38.766738 14064 tablet_bootstrap.cc:492] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:38.766533 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:38.767086 14065 tablet_bootstrap.cc:654] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.767268 14020 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:38.767736 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:38.770861 13771 catalog_manager.cc:5697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:38.778566 14064 tablet_bootstrap.cc:654] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.779701 14066 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.781994 14221 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:38.783531 14066 ts_tablet_manager.cc:1434] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
W20260709 13:03:38.784621 13915 leader_election.cc:343] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.787340 14066 tablet_bootstrap.cc:492] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:38.794701 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:38.795683 14221 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:38.796067 14221 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.796440 14221 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.800060 14066 tablet_bootstrap.cc:654] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.804251 14221 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.811136 14224 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:38.815073 14221 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:38.828320 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:38.829208 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:38.829952 14022 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.845877 14022 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:38.846853 14065 tablet_bootstrap.cc:492] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:38.847365 14065 ts_tablet_manager.cc:1403] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.090s	user 0.021s	sys 0.024s
W20260709 13:03:38.847508 13915 leader_election.cc:343] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.850353 14065 raft_consensus.cc:359] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.851076 14065 raft_consensus.cc:385] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.853091 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:38.851361 14065 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.855640 14065 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.856562 14221 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:38.857241 14221 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:38.857774 14065 ts_tablet_manager.cc:1434] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:03:38.858145 14221 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.858734 14065 tablet_bootstrap.cc:492] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:38.862277 14222 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:38.863183 14064 tablet_bootstrap.cc:492] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:38.863763 14064 ts_tablet_manager.cc:1403] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.097s	user 0.016s	sys 0.000s
I20260709 13:03:38.866572 14065 tablet_bootstrap.cc:654] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.868912 14064 raft_consensus.cc:359] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.869764 14064 raft_consensus.cc:385] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.875954 14064 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.877027 14064 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.883433 14066 tablet_bootstrap.cc:492] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:38.884001 14066 ts_tablet_manager.cc:1403] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.097s	user 0.012s	sys 0.022s
I20260709 13:03:38.887055 14066 raft_consensus.cc:359] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.887751 14066 raft_consensus.cc:385] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.888018 14066 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.888666 14066 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.891124 14066 ts_tablet_manager.cc:1434] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:03:38.891320 14206 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:38.892078 14066 tablet_bootstrap.cc:492] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:38.897063 14064 ts_tablet_manager.cc:1434] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.002s
I20260709 13:03:38.909281 14215 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:38.910499 14064 tablet_bootstrap.cc:492] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:38.911396 14227 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.912024 14227 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.914436 14227 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:38.916515 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:38.918838 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:38.917999 13987 leader_election.cc:343] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:38.920689 13989 leader_election.cc:343] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.921180 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:38.922339 14227 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:38.929039 13770 catalog_manager.cc:5697] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:38.934801 14066 tablet_bootstrap.cc:654] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.949535 14065 tablet_bootstrap.cc:492] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:38.950105 14065 ts_tablet_manager.cc:1403] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.092s	user 0.023s	sys 0.011s
I20260709 13:03:38.953264 14064 tablet_bootstrap.cc:654] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.961618 14065 raft_consensus.cc:359] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:38.964821 13821 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:38.964869 14065 raft_consensus.cc:385] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.965481 14065 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.965930 14066 tablet_bootstrap.cc:492] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:38.966431 14066 ts_tablet_manager.cc:1403] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.075s	user 0.032s	sys 0.005s
I20260709 13:03:38.966289 14065 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.969058 14065 ts_tablet_manager.cc:1434] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:03:38.970109 14064 tablet_bootstrap.cc:492] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:38.970654 14064 ts_tablet_manager.cc:1403] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.006s
I20260709 13:03:38.976369 14202 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:38.976958 14066 raft_consensus.cc:359] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.977692 14066 raft_consensus.cc:385] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.977957 14066 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.979656 14064 raft_consensus.cc:359] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.980473 14064 raft_consensus.cc:385] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.980792 14064 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.981508 14064 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.983491 14065 tablet_bootstrap.cc:492] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:38.984176 14064 ts_tablet_manager.cc:1434] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:03:38.985248 14064 tablet_bootstrap.cc:492] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:38.985435 14202 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.985327 14066 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.985971 14202 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:38.989369 14066 ts_tablet_manager.cc:1434] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:03:38.990229 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:38.990541 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:38.991600 13949 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:38.990424 14066 tablet_bootstrap.cc:492] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:38.993937 13838 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:38.994856 14215 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.995002 14064 tablet_bootstrap.cc:654] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.007247 14064 tablet_bootstrap.cc:492] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:39.006915 14085 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:39.009650 14202 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.012800 14085 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:39.013790 14066 tablet_bootstrap.cc:654] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.016656 14064 ts_tablet_manager.cc:1403] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.008s
I20260709 13:03:39.019169 14215 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.019805 14064 raft_consensus.cc:359] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.020603 14064 raft_consensus.cc:385] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.020977 14064 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.021747 14064 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.023906 14064 ts_tablet_manager.cc:1434] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:39.027240 14064 tablet_bootstrap.cc:492] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:39.029155 14066 tablet_bootstrap.cc:492] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:39.029726 14066 ts_tablet_manager.cc:1403] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.003s
I20260709 13:03:39.032543 14066 raft_consensus.cc:359] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.033501 14066 raft_consensus.cc:385] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.033552 14064 tablet_bootstrap.cc:654] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.033874 14066 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.035595 14066 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.038200 14066 ts_tablet_manager.cc:1434] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:03:39.039180 14066 tablet_bootstrap.cc:492] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:39.040376 14085 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:39.045957 14066 tablet_bootstrap.cc:654] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.059437 14066 tablet_bootstrap.cc:492] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:39.060030 14066 ts_tablet_manager.cc:1403] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.007s
I20260709 13:03:39.066201 14065 tablet_bootstrap.cc:654] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.066238 14066 raft_consensus.cc:359] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.067129 14066 raft_consensus.cc:385] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.067438 14066 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.070007 14066 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.071928 14066 ts_tablet_manager.cc:1434] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:03:39.073006 14066 tablet_bootstrap.cc:492] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:39.079751 14064 tablet_bootstrap.cc:492] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:39.080323 14064 ts_tablet_manager.cc:1403] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.056s	user 0.021s	sys 0.027s
I20260709 13:03:39.080726 14066 tablet_bootstrap.cc:654] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.083654 14073 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.083534 14064 raft_consensus.cc:359] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.084363 14064 raft_consensus.cc:385] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.084196 14073 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.084744 14064 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.085680 14227 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:39.085633 14064 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.087656 14073 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.090214 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:39.090865 14022 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:39.093984 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:39.094388 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:39.095160 14064 ts_tablet_manager.cc:1434] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:03:39.095547 14221 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:39.095929 14221 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.096141 14064 tablet_bootstrap.cc:492] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:39.096306 14221 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:39.099378 13915 leader_election.cc:343] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.109117 14215 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.109701 14215 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.110239 14073 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.110710 14073 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.112084 14064 tablet_bootstrap.cc:654] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.112807 14073 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.112972 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:39.113629 13949 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:39.114416 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:39.115056 13838 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:39.115934 14085 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:39.116291 14085 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.116631 14085 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.116694 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:39.117357 14024 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:39.118729 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:03:39.119623 13915 leader_election.cc:343] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.120491 14222 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:39.120908 14222 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.121238 14222 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.124609 14215 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.126523 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:03:39.132194 13840 leader_election.cc:343] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.127647 13872 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.138212 14219 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.138826 14219 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.141602 14219 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.144212 14085 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.147425 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:39.148360 13874 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:39.149981 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:39.143227 14102 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:03:39.160121 13874 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.121206 14221 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.169281 14221 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.171200 14102 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:39.177960 14134 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:39.179606 14134 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:39.150559 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:39.177901 14222 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.185302 14222 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.187851 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:39.188643 13946 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.191303 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:39.191818 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:39.188550 14219 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:39.209452 14219 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.209190 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:39.215050 14024 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.216033 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:39.216511 14022 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.214699 14219 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.223012 14064 tablet_bootstrap.cc:492] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:39.223496 14064 ts_tablet_manager.cc:1403] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.128s	user 0.021s	sys 0.034s
W20260709 13:03:39.275573 13915 leader_election.cc:343] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.275033 14219 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.284044 14065 tablet_bootstrap.cc:492] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:39.285416 14065 ts_tablet_manager.cc:1403] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.302s	user 0.029s	sys 0.043s
I20260709 13:03:39.290311 14066 tablet_bootstrap.cc:492] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:39.291468 14066 ts_tablet_manager.cc:1403] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.219s	user 0.037s	sys 0.065s
I20260709 13:03:39.297838 14219 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.299171 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:39.309669 14134 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:39.312872 13915 leader_election.cc:343] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:39.314172 13915 leader_election.cc:343] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.314929 14215 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.315809 14215 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.317324 14022 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:39.313853 14085 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.323022 14066 raft_consensus.cc:359] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.324030 14066 raft_consensus.cc:385] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.324389 14066 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.324558 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:39.325057 14064 raft_consensus.cc:359] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.326010 14064 raft_consensus.cc:385] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.326371 14064 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.329797 13946 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:39.333397 14215 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.333391 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:39.336115 13870 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.325385 14066 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.328583 14064 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.341796 14219 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.347440 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:39.348098 13838 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:39.349195 14065 raft_consensus.cc:359] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.352082 14065 raft_consensus.cc:385] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.352460 14065 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.353133 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:39.355603 14023 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:39.352421 14219 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.359045 14065 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.361297 14064 ts_tablet_manager.cc:1434] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.138s	user 0.010s	sys 0.000s
I20260709 13:03:39.364949 14085 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:39.365583 14085 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:03:39.366855 14085 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.369367 14065 ts_tablet_manager.cc:1434] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.006s
I20260709 13:03:39.371130 14065 tablet_bootstrap.cc:492] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
W20260709 13:03:39.379442 13840 leader_election.cc:343] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.381199 13840 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:39.390714 14219 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.393293 14206 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.396744 13870 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:39.398437 14064 tablet_bootstrap.cc:492] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:39.409849 14066 ts_tablet_manager.cc:1434] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.118s	user 0.009s	sys 0.006s
I20260709 13:03:39.409849 14206 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.411650 14066 tablet_bootstrap.cc:492] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:39.416781 14064 tablet_bootstrap.cc:654] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.410105 14024 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:39.422905 14065 tablet_bootstrap.cc:654] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.442143 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:39.445647 14219 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:39.452487 14219 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:39.462400 14066 tablet_bootstrap.cc:654] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.469197 14219 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.474174 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:39.475021 13875 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:39.483022 14065 tablet_bootstrap.cc:492] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:39.483657 14065 ts_tablet_manager.cc:1403] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.113s	user 0.016s	sys 0.017s
I20260709 13:03:39.486555 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:39.487686 14024 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:39.494259 14065 raft_consensus.cc:359] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.495193 14065 raft_consensus.cc:385] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.495541 14065 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.496366 14065 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.498914 14222 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.499816 14222 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.511126 14222 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.512318 14065 ts_tablet_manager.cc:1434] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.002s
I20260709 13:03:39.513999 14065 tablet_bootstrap.cc:492] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:39.521112 14066 tablet_bootstrap.cc:492] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:39.527134 14066 ts_tablet_manager.cc:1403] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.116s	user 0.020s	sys 0.006s
I20260709 13:03:39.524344 14064 tablet_bootstrap.cc:492] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:39.525102 13770 catalog_manager.cc:5697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:39.528409 14064 ts_tablet_manager.cc:1403] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.144s	user 0.029s	sys 0.015s
I20260709 13:03:39.514318 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:03:39.530272 13915 leader_election.cc:343] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.531555 14222 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:39.531839 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:39.532104 14222 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:39.533038 14221 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:39.531172 14066 raft_consensus.cc:359] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.534096 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:39.534480 14066 raft_consensus.cc:385] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.534781 14066 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.536962 14221 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:39.538187 14073 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:03:39.538533 14066 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.538998 14221 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.533565 14222 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.540647 14066 ts_tablet_manager.cc:1434] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:03:39.545647 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:39.541046 14219 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:39.551961 14219 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.554714 14219 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.595417 14066 tablet_bootstrap.cc:492] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:39.567654 14064 raft_consensus.cc:359] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.602319 14064 raft_consensus.cc:385] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.602699 14064 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.573321 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:39.596148 14102 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.607535 14102 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:39.609624 13915 leader_election.cc:343] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.610580 14102 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:39.595464 14073 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:03:39.612394 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:39.613229 13946 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
W20260709 13:03:39.615684 13915 leader_election.cc:343] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.616093 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:39.617841 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:39.622471 14224 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.621555 14064 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.623739 13870 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:39.624655 14064 ts_tablet_manager.cc:1434] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.020s
I20260709 13:03:39.625957 14064 tablet_bootstrap.cc:492] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:39.626955 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:39.628345 14134 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:39.628692 14134 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.629061 14134 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.640244 14065 tablet_bootstrap.cc:654] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.646693 14219 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.654879 14219 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.645535 14134 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.657960 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:39.658241 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:39.663463 14064 tablet_bootstrap.cc:654] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.664283 13872 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.666882 14023 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.674881 14066 tablet_bootstrap.cc:654] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.676468 14134 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:39.680400 13872 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:39.681025 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:39.681756 13946 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.682181 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:39.683908 14073 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:39.684978 14073 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:39.686226 14073 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.687767 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:39.690922 13875 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.689883 13946 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:39.707808 14065 tablet_bootstrap.cc:492] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:39.709123 14065 ts_tablet_manager.cc:1403] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.196s	user 0.023s	sys 0.004s
I20260709 13:03:39.711272 14023 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:39.712392 14065 raft_consensus.cc:359] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.713197 14065 raft_consensus.cc:385] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.713477 14065 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.714251 14065 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.716615 14065 ts_tablet_manager.cc:1434] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:39.717800 14065 tablet_bootstrap.cc:492] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:39.730427 13987 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:39.731854 14134 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:39.733067 14134 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:39.734444 14134 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.740731 13875 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:39.754701 14066 tablet_bootstrap.cc:492] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:39.755247 14066 ts_tablet_manager.cc:1403] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.188s	user 0.014s	sys 0.005s
I20260709 13:03:39.760481 14066 raft_consensus.cc:359] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.762077 14066 raft_consensus.cc:385] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.765036 14065 tablet_bootstrap.cc:654] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.762424 14066 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.770979 14066 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.783793 14134 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.784423 14134 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.787827 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:39.788996 14066 ts_tablet_manager.cc:1434] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.009s
I20260709 13:03:39.790302 14066 tablet_bootstrap.cc:492] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:39.791294 14134 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:39.792920 14064 tablet_bootstrap.cc:492] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:39.793493 14064 ts_tablet_manager.cc:1403] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.168s	user 0.020s	sys 0.016s
I20260709 13:03:39.794909 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:39.795786 13946 raft_consensus.cc:2393] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:03:39.798938 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:39.801671 14065 tablet_bootstrap.cc:492] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:39.802270 14065 ts_tablet_manager.cc:1403] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.002s
I20260709 13:03:39.802577 14102 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.802685 14064 raft_consensus.cc:359] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.799190 13773 catalog_manager.cc:5697] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:39.803472 14064 raft_consensus.cc:385] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.803815 14064 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.805143 14065 raft_consensus.cc:359] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.805943 14065 raft_consensus.cc:385] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.806277 14065 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.807051 14065 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.805162 14064 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.809180 14065 ts_tablet_manager.cc:1434] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:03:39.809718 14064 ts_tablet_manager.cc:1434] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:03:39.810338 14065 tablet_bootstrap.cc:492] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:39.812135 14102 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.817361 14065 tablet_bootstrap.cc:654] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.817361 14066 tablet_bootstrap.cc:654] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.818533 14064 tablet_bootstrap.cc:492] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:39.829309 14064 tablet_bootstrap.cc:654] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.849680 14066 tablet_bootstrap.cc:492] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:39.850769 14066 ts_tablet_manager.cc:1403] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.061s	user 0.006s	sys 0.013s
I20260709 13:03:39.854750 14134 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.855581 14134 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.845736 13772 catalog_manager.cc:5697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:39.857684 14134 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:39.858232 13772 catalog_manager.cc:5697] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:39.860347 14066 raft_consensus.cc:359] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.860033 13772 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:39.861001 14066 raft_consensus.cc:385] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.861236 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:39.861636 13772 catalog_manager.cc:5697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:39.862704 14065 tablet_bootstrap.cc:492] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:39.863220 14065 ts_tablet_manager.cc:1403] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.012s
I20260709 13:03:39.864722 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:39.866171 14065 raft_consensus.cc:359] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.866535 14085 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.866837 14065 raft_consensus.cc:385] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.867165 14065 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.865842 13875 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:39.872641 14064 tablet_bootstrap.cc:492] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:39.873153 14064 ts_tablet_manager.cc:1403] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.055s	user 0.024s	sys 0.023s
I20260709 13:03:39.861285 14066 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.867214 14085 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.867893 14065 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.876466 14065 ts_tablet_manager.cc:1434] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:03:39.874161 14066 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.877557 14065 tablet_bootstrap.cc:492] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:39.878340 14066 ts_tablet_manager.cc:1434] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:03:39.879410 14066 tablet_bootstrap.cc:492] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:39.882202 14064 raft_consensus.cc:359] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.882990 14064 raft_consensus.cc:385] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.883301 14064 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.883972 14064 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.885236 14065 tablet_bootstrap.cc:654] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.872927 13989 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:39.887164 14100 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.890692 14085 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.891047 14066 tablet_bootstrap.cc:654] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.892403 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:39.893306 14021 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:39.884500 14215 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:03:39.885051 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:39.902081 13838 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:39.902904 14064 ts_tablet_manager.cc:1434] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:03:39.903405 14206 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.904009 14065 tablet_bootstrap.cc:492] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:39.904165 14064 tablet_bootstrap.cc:492] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:39.904551 14065 ts_tablet_manager.cc:1403] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.008s
I20260709 13:03:39.907887 14065 raft_consensus.cc:359] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.908592 14065 raft_consensus.cc:385] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.908885 14065 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.909636 14065 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.912014 14065 ts_tablet_manager.cc:1434] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:39.913017 14064 tablet_bootstrap.cc:654] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.913149 14065 tablet_bootstrap.cc:492] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:39.919772 14073 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:39.921627 14065 tablet_bootstrap.cc:654] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.922199 14066 tablet_bootstrap.cc:492] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:39.922715 14066 ts_tablet_manager.cc:1403] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.008s
I20260709 13:03:39.925655 14066 raft_consensus.cc:359] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.926431 14066 raft_consensus.cc:385] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.926723 14066 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.927142 14102 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.927636 14066 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.927783 14102 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.931766 14102 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:39.936738 14064 tablet_bootstrap.cc:492] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:39.937273 14064 ts_tablet_manager.cc:1403] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.033s	user 0.006s	sys 0.007s
I20260709 13:03:39.941308 14066 ts_tablet_manager.cc:1434] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 13:03:39.941398 14064 raft_consensus.cc:359] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.942301 14064 raft_consensus.cc:385] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.942571 14064 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.944164 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:39.944839 13946 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:39.946928 13987 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:39.948664 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:39.949370 13872 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:39.952276 14066 tablet_bootstrap.cc:492] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:39.952487 14064 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.961570 14065 tablet_bootstrap.cc:492] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:39.962157 14065 ts_tablet_manager.cc:1403] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.026s
I20260709 13:03:39.958391 14102 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:39.971074 14065 raft_consensus.cc:359] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.971974 14065 raft_consensus.cc:385] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.972301 14065 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.972699 14085 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.963299 14102 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.973078 14065 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.974334 14066 tablet_bootstrap.cc:654] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.973431 14085 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.975733 14102 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.980417 14219 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.981204 14219 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:39.982848 14085 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.983407 14219 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:39.985483 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:39.986460 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:39.987010 14065 ts_tablet_manager.cc:1434] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.001s
W20260709 13:03:39.987182 13915 leader_election.cc:343] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.987744 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:39.988395 14020 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:39.985915 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:39.989391 13946 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
W20260709 13:03:39.989676 13915 leader_election.cc:343] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.989377 14065 tablet_bootstrap.cc:492] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:39.990142 13840 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:39.993777 14300 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:39.994213 14300 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.994306 14100 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.994617 14300 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.991190 14064 ts_tablet_manager.cc:1434] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.054s	user 0.017s	sys 0.003s
I20260709 13:03:39.990170 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:39.998643 14219 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.999696 14064 tablet_bootstrap.cc:492] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:40.007290 14206 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:40.011821 14206 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:40.009121 14102 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.015934 14219 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:40.016942 14300 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.017596 14102 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:40.020663 14073 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.024178 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:40.024894 13870 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.027375 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:40.032565 14300 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:40.036964 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:40.037940 14023 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.041994 14073 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.042414 14221 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:40.042747 14073 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.038854 13946 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.062423 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:40.066823 14064 tablet_bootstrap.cc:654] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.056197 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:40.123893 13946 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:40.108282 13870 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:40.124918 14065 tablet_bootstrap.cc:654] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.126091 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:03:40.075415 13915 leader_election.cc:343] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.133123 14102 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.133694 14102 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:03:40.109408 13821 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:40.109598 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:40.134660 14102 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:40.128542 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 53173.7us
I20260709 13:03:40.103883 14023 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:40.142110 13840 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:40.154090 14300 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.154851 14300 raft_consensus.cc:697] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:03:40.071861 14073 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:40.128414 13948 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.158793 14066 tablet_bootstrap.cc:492] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:40.160706 14066 ts_tablet_manager.cc:1403] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.209s	user 0.015s	sys 0.009s
W20260709 13:03:40.162398 13915 leader_election.cc:343] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.162884 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:40.163861 14073 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:40.163793 14066 raft_consensus.cc:359] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.167941 14066 raft_consensus.cc:385] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.169266 14066 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.155794 14300 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.174590 14066 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.176560 14066 ts_tablet_manager.cc:1434] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:03:40.177824 14066 tablet_bootstrap.cc:492] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:40.186892 14065 tablet_bootstrap.cc:492] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:40.187476 14065 ts_tablet_manager.cc:1403] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.199s	user 0.019s	sys 0.022s
W20260709 13:03:40.190970 13915 consensus_peers.cc:597] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:40.199995 13948 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:40.202040 14065 raft_consensus.cc:359] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.206606 14065 raft_consensus.cc:385] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.206913 14065 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.207816 14065 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.210821 14065 ts_tablet_manager.cc:1434] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:03:40.202332 14102 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.213706 14073 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:40.214445 14102 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.217360 14065 tablet_bootstrap.cc:492] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:40.229183 14066 tablet_bootstrap.cc:654] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:40.189291 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 63328.5us
I20260709 13:03:40.237614 14102 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:40.239913 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:40.242102 13948 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:40.243549 13772 catalog_manager.cc:5697] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:40.245651 13987 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:40.247083 14218 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.248325 14102 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.248767 14102 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.249200 14102 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.259989 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:40.261941 14064 tablet_bootstrap.cc:492] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:40.280612 14064 ts_tablet_manager.cc:1403] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.281s	user 0.027s	sys 0.022s
I20260709 13:03:40.285049 14102 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.269852 14065 tablet_bootstrap.cc:654] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.268582 14134 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:03:40.287313 14134 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.287986 14134 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.290510 14134 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:40.292799 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:40.273283 14066 tablet_bootstrap.cc:492] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:40.292774 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:40.294045 14066 ts_tablet_manager.cc:1403] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.116s	user 0.027s	sys 0.003s
I20260709 13:03:40.294305 13948 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:40.273556 13989 leader_election.cc:343] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.297065 14278 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:40.300828 13773 catalog_manager.cc:5697] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:40.303236 13948 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
W20260709 13:03:40.308123 13987 leader_election.cc:343] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.309128 14066 raft_consensus.cc:359] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.311046 13987 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:40.312314 14065 tablet_bootstrap.cc:492] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:40.287334 14102 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:40.314639 14102 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.315224 14102 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:40.316107 14102 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.320168 14065 ts_tablet_manager.cc:1403] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.103s	user 0.013s	sys 0.002s
I20260709 13:03:40.309883 14066 raft_consensus.cc:385] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.322186 14066 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.314625 14064 raft_consensus.cc:359] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.327399 14064 raft_consensus.cc:385] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.327725 14064 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.322906 14066 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.340283 14064 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.343468 14065 raft_consensus.cc:359] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.344539 14065 raft_consensus.cc:385] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.344854 14065 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.349574 14066 ts_tablet_manager.cc:1434] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.000s
W20260709 13:03:40.347815 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45449.2us
I20260709 13:03:40.364639 14064 ts_tablet_manager.cc:1434] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.000s
I20260709 13:03:40.370313 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:40.364730 14065 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.376930 14066 tablet_bootstrap.cc:492] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:40.380586 14065 ts_tablet_manager.cc:1434] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.002s
I20260709 13:03:40.380106 13772 catalog_manager.cc:5697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:40.379210 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:40.382361 14065 tablet_bootstrap.cc:492] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:40.383816 14064 tablet_bootstrap.cc:492] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742: Bootstrap starting.
W20260709 13:03:40.383039 13989 leader_election.cc:343] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.391445 13989 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:40.392238 14100 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:40.393914 13989 leader_election.cc:343] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.394212 14100 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.394718 14100 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.399679 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:03:40.401386 13987 leader_election.cc:343] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.396999 14100 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:40.401031 14319 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:03:40.404851 14065 tablet_bootstrap.cc:654] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.413859 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:40.417769 13989 leader_election.cc:343] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.421099 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:40.421833 14100 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:40.428221 14066 tablet_bootstrap.cc:654] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.448534 14064 tablet_bootstrap.cc:654] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.453015 14065 tablet_bootstrap.cc:492] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:40.453671 14065 ts_tablet_manager.cc:1403] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.072s	user 0.011s	sys 0.010s
I20260709 13:03:40.456681 14065 raft_consensus.cc:359] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.457592 14065 raft_consensus.cc:385] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.457922 14065 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.458774 14065 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.460830 14065 ts_tablet_manager.cc:1434] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:03:40.461694 14064 tablet_bootstrap.cc:492] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:40.462286 14064 ts_tablet_manager.cc:1403] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.083s	user 0.008s	sys 0.010s
I20260709 13:03:40.466770 14065 tablet_bootstrap.cc:492] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:40.471285 14066 tablet_bootstrap.cc:492] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:40.471881 14066 ts_tablet_manager.cc:1403] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.100s	user 0.017s	sys 0.004s
I20260709 13:03:40.473711 14065 tablet_bootstrap.cc:654] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.473815 14064 raft_consensus.cc:359] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.474886 14064 raft_consensus.cc:385] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.475297 14064 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.476111 14064 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.486757 14064 ts_tablet_manager.cc:1434] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.004s
I20260709 13:03:40.487669 14065 tablet_bootstrap.cc:492] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:40.487946 14064 tablet_bootstrap.cc:492] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:40.488238 14065 ts_tablet_manager.cc:1403] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.004s
I20260709 13:03:40.509754 14065 raft_consensus.cc:359] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.510032 14066 raft_consensus.cc:359] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.510587 14065 raft_consensus.cc:385] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.510715 14066 raft_consensus.cc:385] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.510910 14065 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.511000 14066 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.511718 14065 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.513756 14066 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.518195 14064 tablet_bootstrap.cc:654] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.520211 14066 ts_tablet_manager.cc:1434] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 13:03:40.523471 14066 tablet_bootstrap.cc:492] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:40.526087 14065 ts_tablet_manager.cc:1434] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.001s
I20260709 13:03:40.530342 14066 tablet_bootstrap.cc:654] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.544342 14066 tablet_bootstrap.cc:492] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:40.544898 14066 ts_tablet_manager.cc:1403] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.005s
I20260709 13:03:40.547840 14066 raft_consensus.cc:359] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.548404 14065 tablet_bootstrap.cc:492] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:40.548578 14066 raft_consensus.cc:385] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.548892 14066 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.549571 14066 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.551735 14066 ts_tablet_manager.cc:1434] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:40.552865 14066 tablet_bootstrap.cc:492] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:40.559969 13870 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.577356 14066 tablet_bootstrap.cc:654] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.585701 14065 tablet_bootstrap.cc:654] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.591763 14066 tablet_bootstrap.cc:492] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:40.592131 14064 tablet_bootstrap.cc:492] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:40.592572 14066 ts_tablet_manager.cc:1403] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.007s
I20260709 13:03:40.592576 14064 ts_tablet_manager.cc:1403] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.105s	user 0.025s	sys 0.033s
I20260709 13:03:40.596110 14066 raft_consensus.cc:359] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.596896 14066 raft_consensus.cc:385] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.597179 14066 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.597898 14066 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.600217 14066 ts_tablet_manager.cc:1434] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:40.600266 14064 raft_consensus.cc:359] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.601028 14064 raft_consensus.cc:385] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.601306 14064 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.601387 14066 tablet_bootstrap.cc:492] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:40.602276 14064 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.604451 14064 ts_tablet_manager.cc:1434] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:03:40.604606 14085 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.605741 14064 tablet_bootstrap.cc:492] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:40.606786 14319 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.608788 14065 tablet_bootstrap.cc:492] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:40.608790 14066 tablet_bootstrap.cc:654] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.609472 14065 ts_tablet_manager.cc:1403] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.061s	user 0.009s	sys 0.005s
I20260709 13:03:40.626379 14065 raft_consensus.cc:359] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.627291 14065 raft_consensus.cc:385] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.627693 14065 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.630460 14064 tablet_bootstrap.cc:654] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.634155 14319 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.634718 14319 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.639348 14300 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:40.640894 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:40.641426 14066 tablet_bootstrap.cc:492] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:40.642192 14066 ts_tablet_manager.cc:1403] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.008s
W20260709 13:03:40.642581 13989 leader_election.cc:343] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.643790 14319 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:40.646556 14066 raft_consensus.cc:359] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.647583 14066 raft_consensus.cc:385] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.647915 14066 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.648708 14066 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.649494 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:40.650300 13949 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:40.651641 14066 ts_tablet_manager.cc:1434] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:03:40.651415 14065 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.653019 14066 tablet_bootstrap.cc:492] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:40.655714 14065 ts_tablet_manager.cc:1434] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.046s	user 0.003s	sys 0.006s
I20260709 13:03:40.657708 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:40.658561 14102 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.658928 14102 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.659276 14102 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.660205 14065 tablet_bootstrap.cc:492] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:40.662912 14100 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.667295 14066 tablet_bootstrap.cc:654] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.668411 14065 tablet_bootstrap.cc:654] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.679644 14066 tablet_bootstrap.cc:492] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:40.680322 14066 ts_tablet_manager.cc:1403] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.000s
I20260709 13:03:40.681712 14102 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.683351 14066 raft_consensus.cc:359] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.688926 14066 raft_consensus.cc:385] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.679644 14064 tablet_bootstrap.cc:492] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:40.689277 14066 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.689734 14064 ts_tablet_manager.cc:1403] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.007s
I20260709 13:03:40.690097 14066 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.685304 14065 tablet_bootstrap.cc:492] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:40.691602 14065 ts_tablet_manager.cc:1403] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.007s
I20260709 13:03:40.692479 14066 ts_tablet_manager.cc:1434] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:03:40.693648 14066 tablet_bootstrap.cc:492] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:40.700942 14066 tablet_bootstrap.cc:654] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.708263 14064 raft_consensus.cc:359] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.709173 14064 raft_consensus.cc:385] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.709473 14064 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.710260 14064 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.712750 14064 ts_tablet_manager.cc:1434] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.001s
I20260709 13:03:40.713768 14064 tablet_bootstrap.cc:492] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:40.715495 14065 raft_consensus.cc:359] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.716279 14065 raft_consensus.cc:385] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.716568 14065 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.717252 14065 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.718946 14065 ts_tablet_manager.cc:1434] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.001s
I20260709 13:03:40.718636 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:40.719563 13873 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.720034 14065 tablet_bootstrap.cc:492] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:40.727810 14102 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:40.730250 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:40.731315 13949 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.739475 14065 tablet_bootstrap.cc:654] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.742835 14064 tablet_bootstrap.cc:654] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.742959 14073 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.743721 14073 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.745983 14073 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:40.749539 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:40.751744 13915 leader_election.cc:343] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.753108 14065 tablet_bootstrap.cc:492] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:40.753705 14065 ts_tablet_manager.cc:1403] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.000s
I20260709 13:03:40.754209 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:40.755198 14021 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:40.765758 14278 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.766660 14278 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.769483 14278 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:40.769558 14065 raft_consensus.cc:359] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.772572 14065 raft_consensus.cc:385] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.772898 14065 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.775283 14065 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.777385 14065 ts_tablet_manager.cc:1434] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 13:03:40.777667 14134 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.778527 14065 tablet_bootstrap.cc:492] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:40.778244 14134 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.780380 14134 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:40.785835 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:40.788786 13989 leader_election.cc:343] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.792589 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:40.793637 14224 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.794162 14224 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.794621 14224 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.801537 14224 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.807015 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:40.808030 14021 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:40.811983 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:40.812976 13946 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:40.772291 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:40.824909 14218 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.825845 14064 tablet_bootstrap.cc:492] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:40.825685 14218 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.826592 14064 ts_tablet_manager.cc:1403] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.113s	user 0.018s	sys 0.002s
I20260709 13:03:40.827492 14224 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:40.806085 14278 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.828280 14278 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.828573 14218 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:40.830323 14278 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:40.802139 14102 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.830996 14102 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.832065 14064 raft_consensus.cc:359] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.838645 14064 raft_consensus.cc:385] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.837610 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:40.834228 14102 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:40.839151 14064 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.839345 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:40.840266 14064 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.840930 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:40.841583 13989 leader_election.cc:343] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.842598 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:40.839355 13870 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:40.850996 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:40.851812 14278 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.852208 14278 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.852551 14278 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.847471 14064 ts_tablet_manager.cc:1434] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.002s
I20260709 13:03:40.854306 14064 tablet_bootstrap.cc:492] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:40.821692 13949 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:40.845963 14065 tablet_bootstrap.cc:654] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.861586 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:40.862728 13948 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:40.870189 14064 tablet_bootstrap.cc:654] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:40.871618 13915 leader_election.cc:343] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:40.872689 13915 leader_election.cc:343] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:40.873292 13915 leader_election.cc:343] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.874598 14065 tablet_bootstrap.cc:492] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:40.875118 14065 ts_tablet_manager.cc:1403] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.097s	user 0.013s	sys 0.000s
W20260709 13:03:40.876971 13989 consensus_peers.cc:597] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:40.880767 14066 tablet_bootstrap.cc:492] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:40.881287 14066 ts_tablet_manager.cc:1403] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.188s	user 0.063s	sys 0.105s
I20260709 13:03:40.881992 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:40.882627 14021 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.878258 14065 raft_consensus.cc:359] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.883319 14064 tablet_bootstrap.cc:492] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:40.883431 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:40.883677 14065 raft_consensus.cc:385] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.884047 14065 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.882725 14278 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.884735 14064 ts_tablet_manager.cc:1403] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.006s
I20260709 13:03:40.885955 14066 raft_consensus.cc:359] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.886369 14278 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:40.886672 14066 raft_consensus.cc:385] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.886981 14066 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.887646 14066 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.887949 14224 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.888411 14224 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.888805 14224 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.888890 14065 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.891327 14065 ts_tablet_manager.cc:1434] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:03:40.892298 14064 raft_consensus.cc:359] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.895587 14064 raft_consensus.cc:385] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.895929 14064 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.897887 14066 ts_tablet_manager.cc:1434] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 13:03:40.898258 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:40.898789 14064 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.900960 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:40.901638 14102 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.901708 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:40.902333 14319 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.895617 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:40.897698 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:40.902878 14319 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.902518 14065 tablet_bootstrap.cc:492] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:40.902210 14102 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:40.904297 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:40.904563 14102 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.906019 14134 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.906337 13872 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.906546 14134 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.908632 14134 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:40.909380 14064 ts_tablet_manager.cc:1434] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:03:40.904963 14024 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:40.903292 14020 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:40.903434 14319 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.913285 14134 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.913738 14134 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.914424 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:40.918264 14073 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.918637 14073 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.918933 14073 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.915990 13873 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:40.916934 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:40.920835 14066 tablet_bootstrap.cc:492] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:40.930470 14064 tablet_bootstrap.cc:492] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:40.921559 14134 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.938038 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:40.921615 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:40.940904 13987 leader_election.cc:343] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.941740 14334 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.946812 14334 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.939648 14319 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:40.976161 14019 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:40.979738 14319 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:40.983496 14219 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.000533 14073 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.003932 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:41.004585 14073 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:41.004846 13948 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:41.008368 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:03:41.012895 13989 leader_election.cc:343] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.013321 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:40.984169 14219 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:41.021934 13915 leader_election.cc:343] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.013525 13989 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:41.025300 14319 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:41.027179 14334 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:41.028568 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:41.029253 14020 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:41.029678 14319 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.030135 14319 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.031267 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:41.031844 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:41.032409 14319 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:41.033066 14334 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:41.033129 14319 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.033600 14334 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.034005 14334 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.036104 13872 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:41.036597 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:41.037276 13946 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:41.039384 13987 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:41.040294 14219 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:41.055795 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:41.056939 14019 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:41.057734 14100 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:41.019685 14224 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.073407 14100 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.075882 14100 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.023420 14134 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.079233 14023 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:41.059507 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:41.080161 13773 catalog_manager.cc:5697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:41.062106 14102 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.081645 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:41.081991 14102 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.082518 13946 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:41.084018 14102 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:41.085088 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:41.085753 13946 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:41.086949 13987 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:41.087687 14134 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:41.087877 14073 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.088327 14134 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:41.088454 14073 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:41.088727 14134 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.089119 14134 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.087935 14224 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:41.089500 14073 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.057906 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:41.072279 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:41.074002 14334 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.095058 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:41.063055 14064 tablet_bootstrap.cc:654] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.101639 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:41.110013 14346 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:41.110625 14346 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.111091 14346 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.128960 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:41.129963 14023 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.130476 14334 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:41.104518 13949 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:41.133285 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:41.134236 13873 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.134451 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:41.105799 14278 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.117635 14021 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:41.214212 14065 tablet_bootstrap.cc:654] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.179873 14224 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.134488 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:41.204439 14349 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.141463 14300 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.222561 13949 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.241200 14066 tablet_bootstrap.cc:654] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.220902 14224 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:41.243948 13915 leader_election.cc:343] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.224269 14349 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.223647 14300 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.251703 14073 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.254302 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:41.258437 14219 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.258937 14219 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:41.259797 14219 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.263726 14134 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.253845 14073 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.261425 14100 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.265830 14134 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:41.279836 14073 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:41.292285 14100 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:41.241273 13989 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:41.300899 14019 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:41.304062 14349 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:41.305250 13873 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:41.305956 14300 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:41.307849 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:41.308583 13944 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:41.310348 13838 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:41.311331 14300 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:41.311739 14300 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.289527 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:41.316691 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:41.326231 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:41.326661 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:41.327337 14017 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:41.327747 13873 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:03:41.312093 14300 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.314651 14319 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.319778 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:41.318926 14346 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.361248 14023 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:41.362500 13948 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:41.363670 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:41.369138 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:41.361778 14346 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:41.370327 13947 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:41.373366 14224 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:03:41.352339 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 118908us
I20260709 13:03:41.353991 14218 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.381322 14218 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:41.382680 14218 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.354154 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:41.392560 14065 tablet_bootstrap.cc:492] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:41.393193 14065 ts_tablet_manager.cc:1403] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.495s	user 0.012s	sys 0.016s
I20260709 13:03:41.351594 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:41.406137 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 51577.2us
I20260709 13:03:41.414471 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:41.417025 13915 leader_election.cc:343] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.418800 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:41.429499 14064 tablet_bootstrap.cc:492] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:41.430197 14064 ts_tablet_manager.cc:1403] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.500s	user 0.028s	sys 0.024s
I20260709 13:03:41.432307 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:41.433247 14018 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:41.427277 14347 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:03:41.434405 13915 leader_election.cc:343] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.434741 14347 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.434898 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:41.435184 14347 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.436939 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:41.436977 14066 tablet_bootstrap.cc:492] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:41.437803 14066 ts_tablet_manager.cc:1403] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.517s	user 0.034s	sys 0.021s
I20260709 13:03:41.440254 14224 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:41.450805 13989 leader_election.cc:343] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.451334 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:41.453506 14319 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.454762 14065 raft_consensus.cc:359] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.455469 14065 raft_consensus.cc:385] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.455710 14065 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.442853 13949 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:41.463447 13838 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:41.468173 14334 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.468768 14334 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:03:41.469776 14334 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.471359 14300 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.479692 14300 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:41.481546 14064 raft_consensus.cc:359] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.482470 14064 raft_consensus.cc:385] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.482816 14064 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.481411 14065 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.483886 14064 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.485675 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:41.487334 14065 ts_tablet_manager.cc:1434] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.094s	user 0.008s	sys 0.002s
I20260709 13:03:41.488291 13873 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:03:41.489002 14065 tablet_bootstrap.cc:492] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:41.491086 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:41.491701 13873 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.490578 14064 ts_tablet_manager.cc:1434] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.000s
I20260709 13:03:41.494212 14064 tablet_bootstrap.cc:492] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:41.499877 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:41.503016 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:41.503261 13949 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.511102 14351 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:41.514529 14064 tablet_bootstrap.cc:654] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.526309 13949 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:41.537149 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:41.485235 14066 raft_consensus.cc:359] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.540278 14066 raft_consensus.cc:385] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.540665 14066 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.534029 14347 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.527797 13838 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:41.541455 14066 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.543411 14064 tablet_bootstrap.cc:492] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:41.543989 14064 ts_tablet_manager.cc:1403] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.050s	user 0.018s	sys 0.002s
I20260709 13:03:41.548573 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:41.549572 13870 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:03:41.549527 13771 catalog_manager.cc:5697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.551355 14073 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:41.552608 13821 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:41.553934 13772 catalog_manager.cc:5697] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:41.555646 13989 leader_election.cc:343] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.556064 13772 catalog_manager.cc:5697] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.557849 13772 catalog_manager.cc:5697] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.559688 14019 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:41.562659 14347 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:41.564251 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:41.566576 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:41.566869 13989 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:41.568540 14319 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.569154 14019 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:41.569809 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:41.570130 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:41.570788 13946 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:03:41.570641 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:41.571946 14065 tablet_bootstrap.cc:654] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.572216 14351 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.573091 14066 ts_tablet_manager.cc:1434] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.135s	user 0.013s	sys 0.001s
I20260709 13:03:41.572459 14073 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.556160 14300 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.574821 14300 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:03:41.575858 14300 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.547246 13873 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:41.577003 14319 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:41.578760 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:41.579747 14073 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.581724 14073 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:41.581915 14066 tablet_bootstrap.cc:492] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:41.583612 14073 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.594626 14064 raft_consensus.cc:359] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.595512 14064 raft_consensus.cc:385] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.596592 14064 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.597520 14064 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.600178 14064 ts_tablet_manager.cc:1434] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.000s
I20260709 13:03:41.601629 14064 tablet_bootstrap.cc:492] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:41.665833 14064 tablet_bootstrap.cc:654] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.667786 14065 tablet_bootstrap.cc:492] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:41.668865 14065 ts_tablet_manager.cc:1403] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.180s	user 0.013s	sys 0.039s
I20260709 13:03:41.673877 14066 tablet_bootstrap.cc:654] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.761098 14065 raft_consensus.cc:359] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.792394 14065 raft_consensus.cc:385] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.792744 14065 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.793660 14065 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.798115 14334 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.798705 14334 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.799531 14065 ts_tablet_manager.cc:1434] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.130s	user 0.016s	sys 0.000s
I20260709 13:03:41.800670 14065 tablet_bootstrap.cc:492] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:41.801000 14334 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:41.801365 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:41.804349 14100 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.806876 14066 tablet_bootstrap.cc:492] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:41.807557 14066 ts_tablet_manager.cc:1403] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.226s	user 0.020s	sys 0.006s
I20260709 13:03:41.814373 14066 raft_consensus.cc:359] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.815104 14066 raft_consensus.cc:385] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.815382 14066 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.816058 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:41.816968 13950 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:41.818675 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:41.819501 14019 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:41.820475 14345 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:41.819531 14066 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.827742 14064 tablet_bootstrap.cc:492] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:41.828277 14064 ts_tablet_manager.cc:1403] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.227s	user 0.015s	sys 0.005s
I20260709 13:03:41.829360 13771 catalog_manager.cc:5697] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.832751 14064 raft_consensus.cc:359] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.833521 14064 raft_consensus.cc:385] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.833784 14064 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.835523 14066 ts_tablet_manager.cc:1434] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.001s
I20260709 13:03:41.836577 14066 tablet_bootstrap.cc:492] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:41.836516 14064 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.839092 14064 ts_tablet_manager.cc:1434] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:03:41.840130 14064 tablet_bootstrap.cc:492] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:41.842442 14357 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:41.844139 13915 consensus_peers.cc:597] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:41.847183 13838 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:41.851693 14065 tablet_bootstrap.cc:654] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.853777 14064 tablet_bootstrap.cc:654] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.856129 14066 tablet_bootstrap.cc:654] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.860872 14334 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.874075 14065 tablet_bootstrap.cc:492] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:41.874672 14065 ts_tablet_manager.cc:1403] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.074s	user 0.017s	sys 0.000s
I20260709 13:03:41.875780 13772 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:41.881235 13976 tablet.cc:2406] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:03:41.883315 14334 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.877609 14065 raft_consensus.cc:359] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.907457 14065 raft_consensus.cc:385] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.907846 14065 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.908561 14065 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.910434 14065 ts_tablet_manager.cc:1434] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:03:41.911448 14065 tablet_bootstrap.cc:492] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:41.913606 13868 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.917771 14065 tablet_bootstrap.cc:654] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.900481 14357 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:41.922277 14347 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.922854 14347 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.925247 14347 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:41.927526 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:41.928470 13946 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:41.933730 14345 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.933837 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:41.934521 14065 tablet_bootstrap.cc:492] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:41.934609 14345 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:41.935556 13989 leader_election.cc:343] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.937392 14347 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.939033 14347 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.939383 14345 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:41.941113 14347 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:41.942674 14065 ts_tablet_manager.cc:1403] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.000s
I20260709 13:03:41.994737 14064 tablet_bootstrap.cc:492] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:41.995446 14064 ts_tablet_manager.cc:1403] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.156s	user 0.057s	sys 0.054s
I20260709 13:03:41.998334 14065 raft_consensus.cc:359] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:41.999141 14065 raft_consensus.cc:385] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.999469 14065 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.999781 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:42.001545 13989 leader_election.cc:343] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.000176 14065 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.004371 14065 ts_tablet_manager.cc:1434] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.061s	user 0.011s	sys 0.007s
I20260709 13:03:42.005587 14065 tablet_bootstrap.cc:492] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:42.013681 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:42.014460 14020 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:42.014720 14065 tablet_bootstrap.cc:654] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.019434 14066 tablet_bootstrap.cc:492] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:42.020004 14066 ts_tablet_manager.cc:1403] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.184s	user 0.016s	sys 0.040s
I20260709 13:03:42.021814 14355 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:42.023499 14066 raft_consensus.cc:359] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.024384 14066 raft_consensus.cc:385] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.024662 14066 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.025372 14066 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.025635 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:42.026376 13946 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:42.030534 14064 raft_consensus.cc:359] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.031298 14064 raft_consensus.cc:385] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.031577 14064 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.032258 14064 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.033878 14064 ts_tablet_manager.cc:1434] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.005s
I20260709 13:03:42.033908 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:42.034345 14219 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:42.035115 14064 tablet_bootstrap.cc:492] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:42.038116 14066 ts_tablet_manager.cc:1434] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:42.039268 14066 tablet_bootstrap.cc:492] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:42.042184 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:42.046351 14319 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.046770 14319 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.047113 14319 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.047515 13915 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:42.048188 14066 tablet_bootstrap.cc:654] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.049140 14362 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.049547 14362 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.049888 14362 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:42.056529 13915 leader_election.cc:343] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.061787 14319 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.064770 14362 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.057713 13987 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:42.067590 14362 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:42.072110 14064 tablet_bootstrap.cc:654] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.073302 14347 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.073822 14347 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.074206 14347 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.075510 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:03:42.076965 13989 leader_election.cc:343] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.079181 14319 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:42.081749 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:42.085800 14357 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:42.089422 13915 leader_election.cc:343] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.135133 14319 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.135780 14319 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.137974 14319 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:42.139629 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:42.141147 13989 leader_election.cc:343] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.085721 14347 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.158766 14065 tablet_bootstrap.cc:492] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:42.159314 14065 ts_tablet_manager.cc:1403] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.154s	user 0.014s	sys 0.039s
I20260709 13:03:42.165464 14355 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:42.166909 14065 raft_consensus.cc:359] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.169713 14065 raft_consensus.cc:385] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.170028 14065 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.170786 14065 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.175984 14065 ts_tablet_manager.cc:1434] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.006s
I20260709 13:03:42.178292 14065 tablet_bootstrap.cc:492] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:42.181931 14357 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.182689 14357 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.185575 14357 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:42.188259 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:42.190315 13915 leader_election.cc:343] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.193041 14065 tablet_bootstrap.cc:654] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.201858 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:03:42.203941 13989 leader_election.cc:343] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.204862 14319 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.205408 14319 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.207615 14319 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:42.209393 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:42.211145 13989 leader_election.cc:343] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.214326 14066 tablet_bootstrap.cc:492] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:42.214881 14066 ts_tablet_manager.cc:1403] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.176s	user 0.030s	sys 0.030s
I20260709 13:03:42.220643 14064 tablet_bootstrap.cc:492] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:42.221405 14064 ts_tablet_manager.cc:1403] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.187s	user 0.045s	sys 0.074s
I20260709 13:03:42.222733 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:42.223786 13948 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.224753 14066 raft_consensus.cc:359] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.225485 14066 raft_consensus.cc:385] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.225759 14066 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.226253 14064 raft_consensus.cc:359] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.227182 14064 raft_consensus.cc:385] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.227526 14064 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.228335 14064 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.228592 14066 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.232074 14064 ts_tablet_manager.cc:1434] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:03:42.233210 14064 tablet_bootstrap.cc:492] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:42.234259 14347 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:42.234480 14355 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:42.238389 13948 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:42.244125 14066 ts_tablet_manager.cc:1434] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:03:42.246551 14066 tablet_bootstrap.cc:492] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:42.247586 14355 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:42.258983 14066 tablet_bootstrap.cc:654] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.265368 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:42.270077 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:42.277290 14023 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:42.277868 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:42.280092 14015 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:42.281567 14064 tablet_bootstrap.cc:654] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.285081 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:42.285852 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:42.286707 13945 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:42.290185 14065 tablet_bootstrap.cc:492] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:42.294250 14065 ts_tablet_manager.cc:1403] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.116s	user 0.023s	sys 0.004s
I20260709 13:03:42.295351 14219 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:42.299043 13915 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:42.299904 14357 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:42.306136 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:42.298612 14065 raft_consensus.cc:359] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.308651 14065 raft_consensus.cc:385] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.309124 14065 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.309932 14065 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.311030 14347 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.311609 14347 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:42.312563 14347 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.313032 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:42.314255 14357 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.314667 14357 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.315014 14357 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:42.319278 13987 leader_election.cc:343] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.319813 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:42.321254 14100 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:42.323665 13987 leader_election.cc:343] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.324098 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:42.325662 14065 ts_tablet_manager.cc:1434] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:03:42.326897 14065 tablet_bootstrap.cc:492] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:42.330938 14100 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:42.332376 13987 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:42.338778 14319 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:42.380559 14102 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.381286 14102 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.390664 14102 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:42.393240 14334 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.393858 14334 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.396143 14334 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:42.400238 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:42.400993 13944 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:42.421576 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:42.425292 13987 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:42.427206 14102 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.427737 14102 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.428140 14102 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:42.437337 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 94090.5us
I20260709 13:03:42.440718 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:42.441679 14395 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:03:42.450564 13989 leader_election.cc:343] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.452822 14102 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.465152 14102 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:42.472342 14064 tablet_bootstrap.cc:492] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:42.473002 14064 ts_tablet_manager.cc:1403] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.240s	user 0.035s	sys 0.081s
W20260709 13:03:42.475711 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 50531.5us
I20260709 13:03:42.478160 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:42.478850 13949 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.481990 14066 tablet_bootstrap.cc:492] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:42.483666 14066 ts_tablet_manager.cc:1403] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.237s	user 0.017s	sys 0.009s
I20260709 13:03:42.490530 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:42.482703 13871 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.488684 13772 catalog_manager.cc:5697] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:42.499562 13949 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:42.500721 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:42.501155 13987 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:42.503491 14102 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.504222 14102 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:42.505050 14102 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.494714 14066 raft_consensus.cc:359] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.509465 14066 raft_consensus.cc:385] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.509914 14066 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.513484 14066 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:42.513828 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 62016.7us
I20260709 13:03:42.518440 14319 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.521521 14319 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.524859 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:42.536029 14065 tablet_bootstrap.cc:654] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:42.534348 13989 leader_election.cc:343] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.542608 14066 ts_tablet_manager.cc:1434] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.059s	user 0.009s	sys 0.001s
I20260709 13:03:42.544081 14064 raft_consensus.cc:359] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.544875 14066 tablet_bootstrap.cc:492] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:42.544819 14064 raft_consensus.cc:385] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.545574 14064 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.546325 14064 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.549516 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:42.547974 14319 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:42.550994 14064 ts_tablet_manager.cc:1434] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.078s	user 0.011s	sys 0.003s
I20260709 13:03:42.552412 14064 tablet_bootstrap.cc:492] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Bootstrap starting.
W20260709 13:03:42.554450 13989 leader_election.cc:343] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.561885 14357 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.563493 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:42.564989 14357 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:03:42.565094 13987 leader_election.cc:343] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.565629 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:42.567833 14347 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:42.577461 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:42.577760 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:42.578408 14017 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.578344 13873 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.590770 14017 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:42.595829 14064 tablet_bootstrap.cc:654] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.601140 14066 tablet_bootstrap.cc:654] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.606352 14401 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:42.613797 14065 tablet_bootstrap.cc:492] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:42.614643 14065 ts_tablet_manager.cc:1403] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.288s	user 0.015s	sys 0.015s
I20260709 13:03:42.619102 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:42.619956 14357 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.620586 14357 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:42.621963 14357 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:42.628546 13821 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:42.630487 14065 raft_consensus.cc:359] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.631290 14065 raft_consensus.cc:385] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.631595 14065 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.632349 14065 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.635205 14065 ts_tablet_manager.cc:1434] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.002s
I20260709 13:03:42.641140 14065 tablet_bootstrap.cc:492] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:42.660763 13873 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:42.671017 13769 catalog_manager.cc:5697] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:42.674680 14401 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.694684 14401 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.695778 14065 tablet_bootstrap.cc:654] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.698028 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:42.683563 14064 tablet_bootstrap.cc:492] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:42.699715 14064 ts_tablet_manager.cc:1403] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.148s	user 0.026s	sys 0.023s
W20260709 13:03:42.700306 13838 leader_election.cc:343] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.681170 14066 tablet_bootstrap.cc:492] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:42.702598 14066 ts_tablet_manager.cc:1403] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.159s	user 0.013s	sys 0.035s
I20260709 13:03:42.706956 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:42.708019 14022 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:42.708374 13771 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:42.709919 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:03:42.710788 14402 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.711165 14402 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.711541 14402 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.715059 14064 raft_consensus.cc:359] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.716109 14064 raft_consensus.cc:385] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.716444 14064 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.717342 14064 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.720778 14402 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.722568 14066 raft_consensus.cc:359] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.723235 14066 raft_consensus.cc:385] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.724952 14066 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.725662 14066 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.728274 14066 ts_tablet_manager.cc:1434] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:03:42.728847 14401 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:42.729630 14066 tablet_bootstrap.cc:492] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:42.730930 14402 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:42.733382 14064 ts_tablet_manager.cc:1434] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:03:42.735868 14065 tablet_bootstrap.cc:492] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:42.736245 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:42.736562 14065 ts_tablet_manager.cc:1403] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.096s	user 0.015s	sys 0.028s
W20260709 13:03:42.738369 13838 leader_election.cc:343] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.740509 14066 tablet_bootstrap.cc:654] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.741356 14065 raft_consensus.cc:359] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.742225 14065 raft_consensus.cc:385] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.742499 14065 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.742928 14064 tablet_bootstrap.cc:492] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:42.743325 14065 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.751466 14064 tablet_bootstrap.cc:654] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.755915 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:42.756695 14022 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.762794 14065 ts_tablet_manager.cc:1434] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.026s	user 0.013s	sys 0.004s
I20260709 13:03:42.764766 14065 tablet_bootstrap.cc:492] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:42.766462 14022 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:42.770260 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:03:42.771490 14402 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.772017 14402 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:03:42.773005 14065 tablet_bootstrap.cc:654] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.772931 14402 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.784636 14066 tablet_bootstrap.cc:492] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:42.785216 14066 ts_tablet_manager.cc:1403] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.004s
I20260709 13:03:42.792800 13873 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.808475 14066 raft_consensus.cc:359] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.812182 14066 raft_consensus.cc:385] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.812508 14066 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.813206 14066 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.815447 14066 ts_tablet_manager.cc:1434] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:03:42.817494 13773 catalog_manager.cc:5697] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:42.818390 14065 tablet_bootstrap.cc:492] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:42.818480 14319 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:42.819543 14066 tablet_bootstrap.cc:492] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:42.821251 14065 ts_tablet_manager.cc:1403] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.057s	user 0.021s	sys 0.031s
I20260709 13:03:42.824275 14065 raft_consensus.cc:359] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.824958 14065 raft_consensus.cc:385] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.825265 14065 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.826009 14065 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.820436 14064 tablet_bootstrap.cc:492] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:42.827078 14064 ts_tablet_manager.cc:1403] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.084s	user 0.014s	sys 0.021s
I20260709 13:03:42.829509 14066 tablet_bootstrap.cc:654] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.830142 14064 raft_consensus.cc:359] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.832675 14065 ts_tablet_manager.cc:1434] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:03:42.833617 14064 raft_consensus.cc:385] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.833992 14064 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.834875 14064 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.837042 14065 tablet_bootstrap.cc:492] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:42.844056 14065 tablet_bootstrap.cc:654] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.847959 14218 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.848610 14064 ts_tablet_manager.cc:1434] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:03:42.848829 14218 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.849627 14064 tablet_bootstrap.cc:492] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:42.851723 14218 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:42.854748 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:42.854970 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:42.855656 14017 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:42.856185 14064 tablet_bootstrap.cc:654] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.857050 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:03:42.860265 13915 leader_election.cc:343] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.860787 14065 tablet_bootstrap.cc:492] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:42.861294 14065 ts_tablet_manager.cc:1403] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.003s
I20260709 13:03:42.862112 14218 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.862527 14218 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.862860 14218 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.864207 14065 raft_consensus.cc:359] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.864954 14065 raft_consensus.cc:385] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.865242 14065 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.865911 14065 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.866310 14351 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:42.876626 14065 ts_tablet_manager.cc:1434] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.001s
I20260709 13:03:42.881172 14100 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.881803 14100 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.881412 14066 tablet_bootstrap.cc:492] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:42.886133 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:42.888648 14065 tablet_bootstrap.cc:492] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:42.883366 14066 ts_tablet_manager.cc:1403] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.064s	user 0.023s	sys 0.013s
I20260709 13:03:42.890801 13873 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:42.886988 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:42.884259 14100 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:42.891888 13944 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:42.892220 13989 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:42.894151 14351 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.894577 14351 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.894896 14351 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.911850 14347 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:42.915743 14218 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.918171 14066 raft_consensus.cc:359] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.919056 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:42.919157 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:42.925176 14017 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:42.926119 13915 leader_election.cc:343] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.946568 14065 tablet_bootstrap.cc:654] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.918219 14218 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:42.923696 14351 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.954783 14218 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.955013 14351 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:42.955403 14218 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.957783 14218 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:42.958554 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:42.958582 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:42.960443 13868 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.967859 14064 tablet_bootstrap.cc:492] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:42.968501 14064 ts_tablet_manager.cc:1403] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.119s	user 0.016s	sys 0.035s
I20260709 13:03:42.971494 14064 raft_consensus.cc:359] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.972287 14064 raft_consensus.cc:385] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.934865 14017 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:42.972613 14064 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
W20260709 13:03:42.967901 13915 leader_election.cc:343] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.974113 14064 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.976236 14064 ts_tablet_manager.cc:1434] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:03:42.976701 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:42.977286 13949 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.978219 14064 tablet_bootstrap.cc:492] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:42.981860 13868 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:42.987572 13989 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:42.925153 14066 raft_consensus.cc:385] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.990525 14066 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.991649 14066 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:42.993808 14066 ts_tablet_manager.cc:1434] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.103s	user 0.007s	sys 0.000s
I20260709 13:03:43.000924 14319 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.001571 14319 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.001763 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:43.004163 14319 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.004683 14064 tablet_bootstrap.cc:654] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.004406 14022 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:43.008340 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:43.009920 14357 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:43.010516 14357 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:43.000559 14066 tablet_bootstrap.cc:492] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:43.010926 13949 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:43.011489 14357 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.014847 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:43.015091 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:43.015700 13946 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:43.017666 14319 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:03:43.019004 13989 leader_election.cc:343] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.019310 14351 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.022136 13987 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:43.021862 14319 raft_consensus.cc:697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:43.025259 14102 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.025629 14102 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.025949 14102 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.026824 14319 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.027868 14100 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.028399 14100 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.024573 14351 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.030864 14100 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.032363 14351 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.037891 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:43.038990 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:43.039826 13870 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:43.041149 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:43.043658 14219 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.044122 14219 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.041008 14102 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:43.047278 13989 consensus_peers.cc:597] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:43.041807 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:43.050371 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:43.051026 14102 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.051875 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:03:43.052240 13989 leader_election.cc:343] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.044523 14219 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.052045 13944 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:43.054868 14065 tablet_bootstrap.cc:492] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:43.055322 14065 ts_tablet_manager.cc:1403] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.167s	user 0.012s	sys 0.004s
W20260709 13:03:43.056589 13989 leader_election.cc:343] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.051931 14404 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:03:43.058578 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:43.058766 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:43.059373 13945 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:43.048352 14351 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:43.065284 13771 catalog_manager.cc:5697] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.073828 14065 raft_consensus.cc:359] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.075348 14065 raft_consensus.cc:385] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.075717 14065 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.076527 14065 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.063634 14351 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.079360 14351 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:43.079025 14065 ts_tablet_manager.cc:1434] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:03:43.083753 14065 tablet_bootstrap.cc:492] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:43.085595 14066 tablet_bootstrap.cc:654] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.088164 14351 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.092584 14064 tablet_bootstrap.cc:492] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:43.093282 14064 ts_tablet_manager.cc:1403] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.115s	user 0.006s	sys 0.006s
I20260709 13:03:43.093401 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:43.094126 13868 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:43.096737 14064 raft_consensus.cc:359] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.097617 14064 raft_consensus.cc:385] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.096876 13769 catalog_manager.cc:5697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:43.097951 14064 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.099534 14219 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.100613 14351 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.101094 13950 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:43.100796 14064 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.098120 14319 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.106213 14319 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.108737 14319 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.098349 14065 tablet_bootstrap.cc:654] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.109431 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:43.111040 13989 leader_election.cc:343] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.105545 14064 ts_tablet_manager.cc:1434] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 13:03:43.113519 14064 tablet_bootstrap.cc:492] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:43.112516 13868 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:03:43.115423 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:43.117074 14319 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:43.118804 14319 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:43.119755 14319 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.126852 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:43.128314 14102 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.134977 14219 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:43.130131 14357 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:03:43.139439 14102 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.133955 14066 tablet_bootstrap.cc:492] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:43.139884 14102 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.140213 14066 ts_tablet_manager.cc:1403] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.140s	user 0.012s	sys 0.002s
I20260709 13:03:43.137820 14064 tablet_bootstrap.cc:654] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.138597 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:43.142644 14066 raft_consensus.cc:359] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.143399 14066 raft_consensus.cc:385] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.143435 13870 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.128221 14065 tablet_bootstrap.cc:492] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:43.144459 14065 ts_tablet_manager.cc:1403] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.008s
I20260709 13:03:43.138381 13987 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:43.145951 14319 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:43.147949 14065 raft_consensus.cc:359] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.148785 14065 raft_consensus.cc:385] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.149083 14065 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.149803 14065 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.152944 14065 ts_tablet_manager.cc:1434] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:03:43.156348 14064 tablet_bootstrap.cc:492] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:43.156932 14064 ts_tablet_manager.cc:1403] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.044s	user 0.015s	sys 0.000s
I20260709 13:03:43.143756 14066 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.159441 14066 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.161727 14066 ts_tablet_manager.cc:1434] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.000s
I20260709 13:03:43.162086 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:43.162842 14066 tablet_bootstrap.cc:492] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:43.162774 13950 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:43.164122 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:43.167115 14065 tablet_bootstrap.cc:492] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:43.171203 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:43.172036 14022 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:03:43.174687 13987 leader_election.cc:343] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.175174 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:43.175915 14319 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:43.176591 14065 tablet_bootstrap.cc:654] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.184716 14102 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.194681 14102 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.202657 13773 catalog_manager.cc:5697] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 2, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:43.212913 14403 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.213822 14403 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.216967 14064 raft_consensus.cc:359] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.217693 14064 raft_consensus.cc:385] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.217983 14064 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.223336 14064 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.226724 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:43.227582 14017 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:03:43.224073 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:43.228286 13944 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.212935 13950 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:03:43.218883 14403 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:43.245613 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:43.246272 14066 tablet_bootstrap.cc:654] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.250856 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:43.252177 13870 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:43.252466 14404 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.261513 14064 ts_tablet_manager.cc:1434] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.104s	user 0.018s	sys 0.010s
I20260709 13:03:43.249233 13944 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:43.263952 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:43.270147 14219 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:43.270782 14219 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:43.271644 14219 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.279306 14404 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:43.283483 14065 tablet_bootstrap.cc:492] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:43.284039 14065 ts_tablet_manager.cc:1403] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.117s	user 0.013s	sys 0.008s
I20260709 13:03:43.288897 14402 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.292029 14065 raft_consensus.cc:359] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.292723 14065 raft_consensus.cc:385] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.293133 14065 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.292644 14402 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.293870 14065 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.295596 14402 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:43.288405 14319 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.296473 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:43.297290 13946 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:43.296296 14319 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.300171 14319 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.301214 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:43.302091 14064 tablet_bootstrap.cc:492] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:43.302363 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:03:43.302973 13989 leader_election.cc:343] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.303119 14022 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:43.301911 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:43.303705 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:03:43.305106 13989 leader_election.cc:343] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.310653 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:43.312091 14065 ts_tablet_manager.cc:1434] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:03:43.312585 14402 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.313405 14065 tablet_bootstrap.cc:492] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:43.321764 14065 tablet_bootstrap.cc:654] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.325606 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:43.327180 14319 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:03:43.327472 13987 leader_election.cc:343] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.327700 14319 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:43.328157 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:43.328644 14319 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.329344 14351 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:43.335449 14400 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:43.345207 14065 tablet_bootstrap.cc:492] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:43.345749 14065 ts_tablet_manager.cc:1403] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.000s
I20260709 13:03:43.347889 14402 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:43.358155 14351 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.358829 14351 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.361474 14351 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.377290 13769 catalog_manager.cc:5697] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.389315 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:43.396680 13989 leader_election.cc:343] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.422685 14065 raft_consensus.cc:359] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.425407 14064 tablet_bootstrap.cc:700] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742: Time spent opening tablet: real 0.112s	user 0.017s	sys 0.017s
I20260709 13:03:43.432536 14066 tablet_bootstrap.cc:492] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:43.444895 14064 tablet_bootstrap.cc:654] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.444962 14066 ts_tablet_manager.cc:1403] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.282s	user 0.006s	sys 0.035s
I20260709 13:03:43.424533 14065 raft_consensus.cc:385] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.448381 14065 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.443061 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:43.448963 14066 raft_consensus.cc:359] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.449232 14065 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.438647 13771 catalog_manager.cc:5697] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:43.449827 14066 raft_consensus.cc:385] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.450361 14064 tablet_bootstrap.cc:492] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:43.450989 14064 ts_tablet_manager.cc:1403] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.149s	user 0.025s	sys 0.018s
I20260709 13:03:43.450636 14066 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.452220 14065 ts_tablet_manager.cc:1434] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.106s	user 0.003s	sys 0.004s
I20260709 13:03:43.451985 14066 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.452981 13946 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:43.453323 14065 tablet_bootstrap.cc:492] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:43.454027 14064 raft_consensus.cc:359] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.455654 14064 raft_consensus.cc:385] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.455916 14064 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.457770 14064 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.460034 14064 ts_tablet_manager.cc:1434] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:03:43.461122 14064 tablet_bootstrap.cc:492] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:43.466396 13874 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.468580 14064 tablet_bootstrap.cc:654] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.475099 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:43.476199 14100 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.476621 14100 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.476967 14100 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.480496 14064 tablet_bootstrap.cc:492] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:43.481093 14064 ts_tablet_manager.cc:1403] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.009s
I20260709 13:03:43.482897 14065 tablet_bootstrap.cc:654] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.484143 14064 raft_consensus.cc:359] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.484917 14064 raft_consensus.cc:385] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.485213 14064 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.486158 14064 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.486613 14100 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.488374 14064 ts_tablet_manager.cc:1434] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:43.489073 14401 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.491618 14066 ts_tablet_manager.cc:1434] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.001s
I20260709 13:03:43.492384 14402 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:43.499066 14404 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.498903 14401 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.499567 14404 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.500022 14064 tablet_bootstrap.cc:492] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:43.501317 14401 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:43.501675 14404 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:43.503279 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:43.503950 13946 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:43.505650 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:43.506297 13871 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:43.508899 14066 tablet_bootstrap.cc:492] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:43.509611 13838 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:43.513803 14401 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.514513 14064 tablet_bootstrap.cc:654] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.514374 14401 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.515017 14401 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.518589 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:43.520108 13915 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:03:43.520339 13989 leader_election.cc:343] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.497114 14065 tablet_bootstrap.cc:492] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:43.520905 14403 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.521091 14065 ts_tablet_manager.cc:1403] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.002s
I20260709 13:03:43.521307 14403 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.521694 14403 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.524873 14100 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.527532 14066 tablet_bootstrap.cc:654] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.528571 14401 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.529258 14403 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.532732 14065 raft_consensus.cc:359] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.533812 14065 raft_consensus.cc:385] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.533684 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:43.534138 14065 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.534390 14021 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
I20260709 13:03:43.534847 14065 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.535681 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:43.536284 13946 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.537125 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:43.537720 14015 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:43.540447 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:43.546545 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:43.547219 14022 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.549013 14064 tablet_bootstrap.cc:492] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:43.549549 14064 ts_tablet_manager.cc:1403] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.000s
I20260709 13:03:43.551141 14402 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:03:43.551779 14402 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.540488 14065 ts_tablet_manager.cc:1434] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.004s
I20260709 13:03:43.552745 14064 raft_consensus.cc:359] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.553776 14064 raft_consensus.cc:385] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.554070 14064 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.554800 14064 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.556399 14064 ts_tablet_manager.cc:1434] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:03:43.557893 14064 tablet_bootstrap.cc:492] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:43.560576 14402 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:43.562592 14065 tablet_bootstrap.cc:492] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:43.565083 13946 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:43.533105 14403 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:43.574368 13868 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:43.566749 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:43.576083 14022 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:43.577549 13870 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:03:43.579010 14406 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:43.570182 14064 tablet_bootstrap.cc:654] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.571782 13872 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.580904 14065 tablet_bootstrap.cc:654] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.582571 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:43.584116 13915 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:43.585284 14403 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:43.585777 14403 raft_consensus.cc:697] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:43.586642 14403 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.594161 14218 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:43.595052 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:43.616911 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:43.620488 14065 tablet_bootstrap.cc:492] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:43.621206 14065 ts_tablet_manager.cc:1403] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.059s	user 0.017s	sys 0.001s
I20260709 13:03:43.621732 14449 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:43.625873 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:43.630961 14401 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:43.636951 14102 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:43.637490 14102 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:43.638528 14102 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.639434 14066 tablet_bootstrap.cc:492] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:43.639973 14066 ts_tablet_manager.cc:1403] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.138s	user 0.014s	sys 0.001s
I20260709 13:03:43.643754 14064 tablet_bootstrap.cc:492] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:43.644336 14064 ts_tablet_manager.cc:1403] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.087s	user 0.007s	sys 0.004s
I20260709 13:03:43.644726 13950 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:43.651121 14066 raft_consensus.cc:359] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.651966 14066 raft_consensus.cc:385] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.652287 14066 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.652993 14066 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.653685 14406 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:43.655062 14065 raft_consensus.cc:359] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.655769 14065 raft_consensus.cc:385] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.656032 14065 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.656697 14065 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.660072 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:43.661969 14022 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:43.671455 14064 raft_consensus.cc:359] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.672425 14064 raft_consensus.cc:385] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.672484 13840 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:43.672873 14064 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.673326 14402 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:43.673749 14064 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.677192 14064 ts_tablet_manager.cc:1434] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:03:43.682977 14064 tablet_bootstrap.cc:492] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742: Bootstrap starting.
W20260709 13:03:43.689011 13821 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:43.675156 14066 ts_tablet_manager.cc:1434] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.004s
I20260709 13:03:43.691046 14066 tablet_bootstrap.cc:492] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:43.677757 14065 ts_tablet_manager.cc:1434] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.000s
I20260709 13:03:43.683485 14219 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:43.695021 14065 tablet_bootstrap.cc:492] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Bootstrap starting.
I20260709 13:03:43.697427 14351 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:43.697606 13769 catalog_manager.cc:5697] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:43.699816 14351 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:43.714334 14065 tablet_bootstrap.cc:654] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.719455 14064 tablet_bootstrap.cc:654] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.721119 14066 tablet_bootstrap.cc:654] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.733330 14351 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.733659 14066 tablet_bootstrap.cc:492] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:43.734180 14066 ts_tablet_manager.cc:1403] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.006s
I20260709 13:03:43.733906 14351 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.736119 14351 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.736766 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:43.738114 13989 leader_election.cc:343] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.737257 14066 raft_consensus.cc:359] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.739228 14066 raft_consensus.cc:385] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.739557 14066 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.740374 14064 tablet_bootstrap.cc:492] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:43.740259 14066 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.740831 14064 ts_tablet_manager.cc:1403] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.058s	user 0.007s	sys 0.008s
I20260709 13:03:43.747268 14066 ts_tablet_manager.cc:1434] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 13:03:43.748340 14066 tablet_bootstrap.cc:492] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:43.748309 14064 raft_consensus.cc:359] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.749325 14064 raft_consensus.cc:385] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.749677 14064 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.750501 14064 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.752171 14065 tablet_bootstrap.cc:492] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: No bootstrap required, opened a new log
I20260709 13:03:43.752753 14064 ts_tablet_manager.cc:1434] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.007s
I20260709 13:03:43.752784 14065 ts_tablet_manager.cc:1403] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent bootstrapping tablet: real 0.058s	user 0.007s	sys 0.005s
I20260709 13:03:43.754443 14064 tablet_bootstrap.cc:492] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:43.755568 14066 tablet_bootstrap.cc:654] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.758771 13771 catalog_manager.cc:5697] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.781082 14064 tablet_bootstrap.cc:654] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.755975 14065 raft_consensus.cc:359] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.789455 14065 raft_consensus.cc:385] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.789860 14065 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.786850 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:43.797938 13944 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:43.799562 14065 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.811515 14065 ts_tablet_manager.cc:1434] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.000s
W20260709 13:03:43.811905 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 67448.6us
I20260709 13:03:43.814121 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:43.814936 14406 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.815325 14406 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.815689 14406 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.816108 14066 tablet_bootstrap.cc:492] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:43.816880 14066 ts_tablet_manager.cc:1403] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.069s	user 0.036s	sys 0.024s
I20260709 13:03:43.821565 14066 raft_consensus.cc:359] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.822678 14066 raft_consensus.cc:385] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.823289 14066 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.824245 14066 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.826707 14404 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:43.827824 14066 ts_tablet_manager.cc:1434] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:03:43.832688 14066 tablet_bootstrap.cc:492] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:43.835760 14406 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.838111 14406 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.839437 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:03:43.841221 13989 leader_election.cc:343] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.842250 14404 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:43.846164 14066 tablet_bootstrap.cc:654] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.861100 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:43.861893 13944 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.865891 14064 tablet_bootstrap.cc:492] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:43.866488 14064 ts_tablet_manager.cc:1403] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.112s	user 0.025s	sys 0.017s
I20260709 13:03:43.869536 14064 raft_consensus.cc:359] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.870396 14064 raft_consensus.cc:385] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.870697 14064 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.871501 14064 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.873782 14064 ts_tablet_manager.cc:1434] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:43.877452 14064 tablet_bootstrap.cc:492] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:43.877345 13944 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:43.887841 14102 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:43.889220 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:43.892117 14064 tablet_bootstrap.cc:654] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.894526 14347 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.896387 14401 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.907899 14347 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.908782 14401 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.910279 14347 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.910921 14401 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:43.911653 14347 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:43.912123 14347 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:43.913218 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:43.912988 14347 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.913848 13874 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:43.917752 13989 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:43.919539 14102 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.920157 14102 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.928812 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:43.929668 14102 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.934355 14401 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.934988 14401 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.937472 14319 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.938019 14319 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:43.937496 14401 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:43.940218 14319 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:43.941009 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:43.941952 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:43.942236 13950 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:03:43.943502 13989 leader_election.cc:343] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.950733 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:43.958096 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:43.975167 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:43.982429 14219 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.983197 14219 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.015872 14066 tablet_bootstrap.cc:492] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:44.017072 14066 ts_tablet_manager.cc:1403] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.185s	user 0.021s	sys 0.039s
I20260709 13:03:44.020640 14219 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:44.020517 14066 raft_consensus.cc:359] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.021315 14066 raft_consensus.cc:385] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.021642 14066 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.022367 14066 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.024492 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:44.024606 13840 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:03:44.024861 13987 leader_election.cc:343] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.024106 14066 ts_tablet_manager.cc:1434] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:03:44.032601 14066 tablet_bootstrap.cc:492] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:44.025523 14441 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:44.035899 13868 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.054090 14064 tablet_bootstrap.cc:492] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:44.053876 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:44.055811 14064 ts_tablet_manager.cc:1403] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.179s	user 0.041s	sys 0.055s
I20260709 13:03:44.057945 14347 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.058493 14347 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.060644 14347 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:44.056016 14102 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.062269 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:44.063189 13773 catalog_manager.cc:5697] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.064461 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:44.065079 13871 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:44.069322 13987 leader_election.cc:343] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.069799 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:44.062559 14102 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:44.074467 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:44.076848 13871 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:44.077085 14347 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:44.076862 13915 leader_election.cc:343] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:44.074033 13989 leader_election.cc:343] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.082013 13989 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:44.083853 14441 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:44.084053 14064 raft_consensus.cc:359] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.084848 14064 raft_consensus.cc:385] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.085158 14064 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.085551 13870 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.086845 14347 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.087986 14066 tablet_bootstrap.cc:654] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.090886 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:44.093315 13949 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:44.094920 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:44.097359 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:44.097441 14319 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.098196 14100 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.096138 14347 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:44.098565 14100 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.098958 14100 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.099404 14347 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.085826 14064 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.103081 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:44.103893 14021 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
W20260709 13:03:44.098568 13987 leader_election.cc:343] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.107654 14064 ts_tablet_manager.cc:1434] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.049s	user 0.003s	sys 0.003s
I20260709 13:03:44.108812 14064 tablet_bootstrap.cc:492] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:44.110993 14403 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.114216 14066 tablet_bootstrap.cc:492] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:44.114766 14066 ts_tablet_manager.cc:1403] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.082s	user 0.010s	sys 0.007s
I20260709 13:03:44.117686 14102 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:44.121515 14100 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.127084 14218 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:44.128106 14100 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:44.127662 14066 raft_consensus.cc:359] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.128587 14066 raft_consensus.cc:385] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.128867 14066 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.129544 14066 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.131197 14066 ts_tablet_manager.cc:1434] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:03:44.132265 14066 tablet_bootstrap.cc:492] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:44.145339 14100 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:44.147419 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:44.150629 14064 tablet_bootstrap.cc:654] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.154701 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:44.155089 14219 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.155490 13949 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.155512 14219 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.156046 14219 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.178576 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:44.184122 14219 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.186369 14219 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:44.189522 13949 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
W20260709 13:03:44.193006 13989 leader_election.cc:343] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.201857 14066 tablet_bootstrap.cc:654] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.204005 13773 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.217691 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:44.218874 14102 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.220288 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:44.224681 14102 raft_consensus.cc:697] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:44.231194 14102 consensus_queue.cc:237] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.236063 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:44.239770 14015 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.259470 14319 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.260313 14319 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:44.262768 13915 leader_election.cc:343] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.265185 14219 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.265548 14319 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:44.265690 14219 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.266258 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:44.267164 13872 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:44.268901 13989 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:44.270036 14319 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:44.270419 14319 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.270744 14319 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:44.271780 14064 tablet_bootstrap.cc:492] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:44.272305 14064 ts_tablet_manager.cc:1403] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.164s	user 0.012s	sys 0.005s
I20260709 13:03:44.274571 14219 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:03:44.275372 14479 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:03:44.278474 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:44.279511 13868 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:44.281891 14064 raft_consensus.cc:359] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.282996 14400 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.283468 14400 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.285717 14400 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:44.282708 14064 raft_consensus.cc:385] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.288102 14064 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.289292 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:44.292346 13945 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:44.292825 13915 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:44.293576 14472 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:44.293903 14472 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.294260 14472 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:44.294539 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:44.295101 13838 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:44.296821 14441 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.297202 14441 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.297675 14441 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.297781 14319 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.307215 14020 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 2: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:44.292114 14064 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.299353 14102 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.310472 14102 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.311902 14064 ts_tablet_manager.cc:1434] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.000s
I20260709 13:03:44.312623 14102 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:44.291139 14015 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:44.313016 14064 tablet_bootstrap.cc:492] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:44.306737 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:44.301474 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:44.313720 14017 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 0.
I20260709 13:03:44.318897 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:44.319111 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:44.320104 13871 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 0.
I20260709 13:03:44.321586 14319 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:44.322026 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:44.323125 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:44.324038 13868 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:44.325623 14102 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.326076 14102 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.326611 14102 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.329152 14472 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.330539 13946 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 2: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:44.331187 14064 tablet_bootstrap.cc:654] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.331527 14472 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:44.334808 14400 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.337333 14400 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.340102 13773 catalog_manager.cc:5697] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.340826 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:44.341267 14400 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:44.342025 13946 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:44.351197 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:44.352298 14403 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.352847 14403 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:44.353755 14441 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.354038 14403 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:44.352880 13987 leader_election.cc:343] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.357467 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:44.359570 13946 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.336189 14102 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.361748 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:44.335976 14218 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.362625 13944 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 2: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:44.362812 14218 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.363545 14102 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:44.364178 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:44.364750 13872 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.364985 14218 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:44.370333 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:44.372727 14066 tablet_bootstrap.cc:492] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:44.373276 14066 ts_tablet_manager.cc:1403] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.241s	user 0.016s	sys 0.013s
I20260709 13:03:44.373381 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:44.373176 13872 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:44.375015 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:44.376358 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:44.376760 14400 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.378573 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:44.379267 14319 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.379772 14319 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:44.380592 14319 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.349700 13868 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:03:44.383843 13989 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:03:44.384245 14066 raft_consensus.cc:359] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.384585 14100 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:03:44.384608 13987 leader_election.cc:343] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.385022 14066 raft_consensus.cc:385] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.385205 14100 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:03:44.385434 14066 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.386016 14100 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.387763 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:44.390594 14400 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:44.393268 13946 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:44.397634 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:44.398458 14015 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 0.
W20260709 13:03:44.401312 13915 leader_election.cc:343] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.403816 14441 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:44.403170 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:44.403163 14066 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.407398 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:44.413286 14015 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.407421 13838 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:03:44.420876 14441 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.424297 14015 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:03:44.425020 14441 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:03:44.431449 14064 tablet_bootstrap.cc:492] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:44.430811 14441 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.431986 14064 ts_tablet_manager.cc:1403] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.119s	user 0.011s	sys 0.031s
I20260709 13:03:44.433117 14100 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:03:44.437973 14064 raft_consensus.cc:359] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.438760 14064 raft_consensus.cc:385] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.439049 14064 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.441067 14066 ts_tablet_manager.cc:1434] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 13:03:44.427304 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:44.443343 14066 tablet_bootstrap.cc:492] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Bootstrap starting.
I20260709 13:03:44.451124 14472 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.451623 14472 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.452023 14472 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.452100 14064 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.459573 14064 ts_tablet_manager.cc:1434] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.027s	user 0.001s	sys 0.008s
I20260709 13:03:44.459273 13773 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.460682 14064 tablet_bootstrap.cc:492] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:44.461061 13773 catalog_manager.cc:5697] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 0 to 2, leader changed from <none> to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.461283 13772 catalog_manager.cc:5697] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:44.467192 13915 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:44.470549 14403 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:44.472059 14319 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.481837 14064 tablet_bootstrap.cc:654] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.483891 14400 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.484520 14400 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.492508 14472 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.494779 14472 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:44.496603 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:44.499222 14400 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:44.501767 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:03:44.508430 14066 tablet_bootstrap.cc:654] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.511200 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:44.512130 14020 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:44.518795 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:44.519563 14021 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.526633 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:44.529433 14400 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:44.538983 13915 leader_election.cc:343] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.544450 14064 tablet_bootstrap.cc:492] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:44.545203 14064 ts_tablet_manager.cc:1403] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.085s	user 0.029s	sys 0.025s
I20260709 13:03:44.545838 14021 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:03:44.538022 13773 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:44.543345 14400 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:44.559456 13987 consensus_peers.cc:597] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:44.562721 14066 tablet_bootstrap.cc:492] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: No bootstrap required, opened a new log
I20260709 13:03:44.563305 14066 ts_tablet_manager.cc:1403] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Time spent bootstrapping tablet: real 0.120s	user 0.011s	sys 0.001s
I20260709 13:03:44.567512 14066 raft_consensus.cc:359] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.568341 14066 raft_consensus.cc:385] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.568619 14066 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.569247 14066 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.569600 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:03:44.570443 14218 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.570793 14064 raft_consensus.cc:359] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.571363 14218 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:44.571599 14064 raft_consensus.cc:385] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.571970 14064 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.572186 14066 ts_tablet_manager.cc:1434] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 13:03:44.572734 14064 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.572350 14218 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.575304 14064 ts_tablet_manager.cc:1434] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:03:44.578130 14064 tablet_bootstrap.cc:492] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Bootstrap starting.
I20260709 13:03:44.585971 14064 tablet_bootstrap.cc:654] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.607527 14064 tablet_bootstrap.cc:492] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: No bootstrap required, opened a new log
I20260709 13:03:44.608949 14064 ts_tablet_manager.cc:1403] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Time spent bootstrapping tablet: real 0.031s	user 0.022s	sys 0.005s
I20260709 13:03:44.615856 14064 raft_consensus.cc:359] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.616755 14064 raft_consensus.cc:385] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.616186 13772 catalog_manager.cc:5697] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:44.617118 14064 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.617830 14064 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.620501 14064 ts_tablet_manager.cc:1434] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:03:44.621137 14218 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.621649 14218 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.624023 14218 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:44.625265 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:44.625941 13872 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 1: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:44.626649 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:44.629112 13915 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:44.630179 14218 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.638460 14218 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:44.649001 14100 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.729542 13871 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.737668 14100 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.750761 14319 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.794793 13945 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:44.796738 14100 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.816291 13872 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:44.817694 14102 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.832244 14404 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:44.845130 14218 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.905200 14102 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.917500 14441 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.917695 14319 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.928169 14472 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.928812 14472 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:44.931072 14472 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:44.935400 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:44.940634 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:44.947109 14441 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.945741 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:03:44.954223 14472 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:45.004179 14449 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.004727 14449 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:45.006742 14449 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:45.007757 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:45.007754 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:45.009052 14015 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 1: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:03:45.010711 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:45.011466 14449 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.018355 14449 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:45.045038 13946 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.064262 14218 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:45.089880 14218 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:46.615177 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:46.644333 14514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:46.644351 14513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:46.645182 12725 server_base.cc:1061] running on GCE node
W20260709 13:03:46.646225 14516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:46.647701 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:46.648010 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:46.648298 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602226648276 us; error 0 us; skew 500 ppm
I20260709 13:03:46.649194 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:46.652218 12725 webserver.cc:533] Webserver started at http://127.12.109.68:37423/ using document root <none> and password file <none>
I20260709 13:03:46.652858 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:46.653088 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:46.653365 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:46.654912 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-3-root/instance:
uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
format_stamp: "Formatted at 2026-07-09 13:03:46 on dist-test-slave-zhdc"
I20260709 13:03:46.660882 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:03:46.665645 14521 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:46.666658 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:03:46.667114 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-3-root
uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
format_stamp: "Formatted at 2026-07-09 13:03:46 on dist-test-slave-zhdc"
I20260709 13:03:46.667394 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:46.697080 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:46.699280 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:46.702320 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:46.706240 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:46.706605 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:46.706945 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:46.707193 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:46.785842 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.68:42851
I20260709 13:03:46.785984 14585 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.68:42851 every 8 connection(s)
I20260709 13:03:46.814079 12725 auto_leader_rebalancer-test.cc:384] add a tserver: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:03:46.828930 14586 heartbeater.cc:344] Connected to a master server at 127.12.109.126:43315
I20260709 13:03:46.829562 14586 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:46.830542 14586 heartbeater.cc:507] Master 127.12.109.126:43315 requested a full tablet report, sending...
I20260709 13:03:46.832938 13771 ts_manager.cc:194] Registered new tserver with Master: 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:03:46.834993 13771 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54072
I20260709 13:03:47.837613 14586 heartbeater.cc:499] Master 127.12.109.126:43315 was elected leader, sending a full tablet report...
I20260709 13:03:47.963290 13871 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:47.975750 14015 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:47.976102 13946 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:47.977759 14590 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:47.978751 14593 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:47.989676 14593 raft_consensus.cc:2955] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
W20260709 13:03:47.992554 13839 consensus_peers.cc:597] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 47fee46f08eb4928939d6ac41891db38. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:47.996414 13946 raft_consensus.cc:2955] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:47.996004 14015 raft_consensus.cc:2955] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:48.018909 13772 catalog_manager.cc:5697] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:48.045930 13948 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:48.063642 13871 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.065722 14015 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.066219 14404 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.073565 14592 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:03:48.081251 13914 consensus_peers.cc:597] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e8d34f260fb43229460064c72d795e8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:48.092038 14592 raft_consensus.cc:2955] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:48.094626 14015 raft_consensus.cc:2955] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:48.104285 13871 raft_consensus.cc:2955] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:48.109580 13771 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.109138 13946 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:48.128090 13871 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.129932 14592 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:48.131613 13914 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f5c93fb008bd4a4f8db94cdc59c65d93. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:48.132895 14015 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.134581 14592 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:48.142259 14592 raft_consensus.cc:2955] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:48.144052 13871 raft_consensus.cc:2955] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:48.154620 14015 raft_consensus.cc:2955] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:48.175352 13946 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:48.175364 13772 catalog_manager.cc:5697] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
W20260709 13:03:48.190457 13914 consensus_peers.cc:597] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eeee4c6df564edcb3c96e2887208cc4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:48.191962 14015 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.193552 13871 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.195036 14614 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.196116 14404 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.216223 14614 raft_consensus.cc:2955] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:48.217730 14015 raft_consensus.cc:2955] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:48.236187 13773 catalog_manager.cc:5697] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:48.237224 13871 raft_consensus.cc:2955] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:48.426512 14618 ts_tablet_manager.cc:933] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer e1649107c781428692458f10597c5742 (127.12.109.65:43607)
I20260709 13:03:48.432165 14618 tablet_copy_client.cc:323] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.65:43607
I20260709 13:03:48.448602 13885 tablet_copy_service.cc:140] P e1649107c781428692458f10597c5742: Received BeginTabletCopySession request for tablet 47fee46f08eb4928939d6ac41891db38 from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:42526)
I20260709 13:03:48.449230 13885 tablet_copy_service.cc:161] P e1649107c781428692458f10597c5742: Beginning new tablet copy session on tablet 47fee46f08eb4928939d6ac41891db38 from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:42526: session id = 4dd265d61fc141a48ca9d3d5c8f52921-47fee46f08eb4928939d6ac41891db38
I20260709 13:03:48.455732 13885 tablet_copy_source_session.cc:215] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:48.460120 14618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47fee46f08eb4928939d6ac41891db38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:48.472559 14618 tablet_copy_client.cc:806] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:48.473328 14618 tablet_copy_client.cc:670] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:48.478458 14618 tablet_copy_client.cc:538] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:48.487515 14618 tablet_bootstrap.cc:492] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:03:48.507858 14618 tablet_bootstrap.cc:492] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:48.508817 14618 tablet_bootstrap.cc:492] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:03:48.509454 14618 ts_tablet_manager.cc:1403] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 13:03:48.512691 14618 raft_consensus.cc:359] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:48.513440 14618 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:03:48.514328 14618 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:48.515559 14622 ts_tablet_manager.cc:933] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:48.529915 14622 tablet_copy_client.cc:323] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.66:43953
I20260709 13:03:48.549492 14618 ts_tablet_manager.cc:1434] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.040s	user 0.011s	sys 0.031s
I20260709 13:03:48.555927 13885 tablet_copy_service.cc:342] P e1649107c781428692458f10597c5742: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-47fee46f08eb4928939d6ac41891db38 received from {username='slave'} at 127.0.0.1:42526
I20260709 13:03:48.556509 13885 tablet_copy_service.cc:434] P e1649107c781428692458f10597c5742: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-47fee46f08eb4928939d6ac41891db38 on tablet 47fee46f08eb4928939d6ac41891db38 with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:03:48.565214 14618 ts_tablet_manager.cc:933] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:48.588495 13960 tablet_copy_service.cc:140] P 3779649036364744823fc035dac892c7: Received BeginTabletCopySession request for tablet 0e8d34f260fb43229460064c72d795e8 from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:58096)
I20260709 13:03:48.589257 13960 tablet_copy_service.cc:161] P 3779649036364744823fc035dac892c7: Beginning new tablet copy session on tablet 0e8d34f260fb43229460064c72d795e8 from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:58096: session id = 4dd265d61fc141a48ca9d3d5c8f52921-0e8d34f260fb43229460064c72d795e8
I20260709 13:03:48.607736 13960 tablet_copy_source_session.cc:215] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:48.610446 14618 tablet_copy_client.cc:323] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.66:43953
I20260709 13:03:48.611346 14622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e8d34f260fb43229460064c72d795e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:48.613405 13960 tablet_copy_service.cc:140] P 3779649036364744823fc035dac892c7: Received BeginTabletCopySession request for tablet f5c93fb008bd4a4f8db94cdc59c65d93 from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:58096)
I20260709 13:03:48.618276 13960 tablet_copy_service.cc:161] P 3779649036364744823fc035dac892c7: Beginning new tablet copy session on tablet f5c93fb008bd4a4f8db94cdc59c65d93 from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:58096: session id = 4dd265d61fc141a48ca9d3d5c8f52921-f5c93fb008bd4a4f8db94cdc59c65d93
I20260709 13:03:48.636631 14622 tablet_copy_client.cc:806] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:48.637243 13960 tablet_copy_source_session.cc:215] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:48.637498 14622 tablet_copy_client.cc:670] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:48.641036 14618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c93fb008bd4a4f8db94cdc59c65d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:48.643784 14622 tablet_copy_client.cc:538] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:48.653772 14622 tablet_bootstrap.cc:492] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:03:48.658903 14618 tablet_copy_client.cc:806] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:48.659840 14618 tablet_copy_client.cc:670] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:48.667182 14618 tablet_copy_client.cc:538] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:48.676996 14618 tablet_bootstrap.cc:492] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:03:48.683498 14622 tablet_bootstrap.cc:492] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:48.684653 14622 tablet_bootstrap.cc:492] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:03:48.685501 14622 ts_tablet_manager.cc:1403] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.032s	user 0.024s	sys 0.005s
I20260709 13:03:48.689185 14622 raft_consensus.cc:359] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:48.690207 14622 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:03:48.690861 14622 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:48.693540 14622 ts_tablet_manager.cc:1434] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:03:48.695680 13960 tablet_copy_service.cc:342] P 3779649036364744823fc035dac892c7: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-0e8d34f260fb43229460064c72d795e8 received from {username='slave'} at 127.0.0.1:58096
I20260709 13:03:48.696169 13960 tablet_copy_service.cc:434] P 3779649036364744823fc035dac892c7: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-0e8d34f260fb43229460064c72d795e8 on tablet 0e8d34f260fb43229460064c72d795e8 with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:03:48.722898 14618 tablet_bootstrap.cc:492] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:48.723858 14618 tablet_bootstrap.cc:492] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:03:48.724655 14618 ts_tablet_manager.cc:1403] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.048s	user 0.019s	sys 0.009s
I20260709 13:03:48.727139 14618 raft_consensus.cc:359] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:48.727844 14618 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:03:48.728380 14618 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:48.730912 14618 ts_tablet_manager.cc:1434] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:03:48.733013 13960 tablet_copy_service.cc:342] P 3779649036364744823fc035dac892c7: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-f5c93fb008bd4a4f8db94cdc59c65d93 received from {username='slave'} at 127.0.0.1:58096
I20260709 13:03:48.733441 13960 tablet_copy_service.cc:434] P 3779649036364744823fc035dac892c7: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-f5c93fb008bd4a4f8db94cdc59c65d93 on tablet f5c93fb008bd4a4f8db94cdc59c65d93 with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:03:48.779002 14618 ts_tablet_manager.cc:933] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:48.780872 14618 tablet_copy_client.cc:323] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.66:43953
I20260709 13:03:48.784839 13960 tablet_copy_service.cc:140] P 3779649036364744823fc035dac892c7: Received BeginTabletCopySession request for tablet 4eeee4c6df564edcb3c96e2887208cc4 from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:58096)
I20260709 13:03:48.785552 13960 tablet_copy_service.cc:161] P 3779649036364744823fc035dac892c7: Beginning new tablet copy session on tablet 4eeee4c6df564edcb3c96e2887208cc4 from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:58096: session id = 4dd265d61fc141a48ca9d3d5c8f52921-4eeee4c6df564edcb3c96e2887208cc4
I20260709 13:03:48.792649 13960 tablet_copy_source_session.cc:215] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:48.796442 14618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eeee4c6df564edcb3c96e2887208cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:48.814679 14618 tablet_copy_client.cc:806] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:48.815352 14618 tablet_copy_client.cc:670] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:48.819536 14618 tablet_copy_client.cc:538] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:48.830188 14618 tablet_bootstrap.cc:492] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:03:48.853396 14618 tablet_bootstrap.cc:492] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:48.854329 14618 tablet_bootstrap.cc:492] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:03:48.854941 14618 ts_tablet_manager.cc:1403] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.011s
I20260709 13:03:48.857928 14618 raft_consensus.cc:359] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:48.858896 14618 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:03:48.859506 14618 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:48.861898 14618 ts_tablet_manager.cc:1434] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:48.869083 13960 tablet_copy_service.cc:342] P 3779649036364744823fc035dac892c7: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-4eeee4c6df564edcb3c96e2887208cc4 received from {username='slave'} at 127.0.0.1:58096
I20260709 13:03:48.869656 13960 tablet_copy_service.cc:434] P 3779649036364744823fc035dac892c7: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-4eeee4c6df564edcb3c96e2887208cc4 on tablet 4eeee4c6df564edcb3c96e2887208cc4 with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:03:48.959950 14561 raft_consensus.cc:1217] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:49.010768 14561 raft_consensus.cc:1217] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:49.114933 14561 raft_consensus.cc:1217] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:49.262801 14561 raft_consensus.cc:1217] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:49.428661 14633 raft_consensus.cc:1064] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:03:49.436270 14633 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:49.521101 14561 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.533133 14590 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:49.534590 13950 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.536878 14593 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:49.543213 14557 raft_consensus.cc:1064] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
W20260709 13:03:49.561192 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44035.6us
W20260709 13:03:49.562611 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 54103.6us
I20260709 13:03:49.628664 14557 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:49.662700 14017 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.683907 14592 raft_consensus.cc:1064] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:03:49.687989 14592 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:49.702796 14590 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:49.719939 14561 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.722157 14639 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:49.765793 14590 raft_consensus.cc:2955] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:49.775597 13950 raft_consensus.cc:2955] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:49.787041 13873 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.804560 14634 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:49.801398 14561 raft_consensus.cc:2955] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:49.820158 14560 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.824252 13874 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.826970 14557 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:49.833967 14015 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.842555 14015 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.843922 14592 raft_consensus.cc:1064] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:03:49.845671 14638 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:49.856087 14592 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:49.858007 13771 catalog_manager.cc:5697] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:49.867892 14021 raft_consensus.cc:2955] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:49.874287 14638 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:03:49.905030 14640 raft_consensus.cc:2955] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:49.894086 14638 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:03:49.920756 14404 raft_consensus.cc:2955] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:49.947506 14560 raft_consensus.cc:2955] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:49.984561 14017 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.983449 14561 raft_consensus.cc:2955] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:49.991456 14559 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.995500 14614 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:49.968631 14020 raft_consensus.cc:2955] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:49.996773 14638 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:50.088629 13873 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.084091 13874 raft_consensus.cc:2955] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.090325 13875 raft_consensus.cc:2955] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.110438 14592 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:50.072638 14017 raft_consensus.cc:2955] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.129522 14592 raft_consensus.cc:2955] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.137903 14559 raft_consensus.cc:2955] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.176296 14017 raft_consensus.cc:2955] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.205569 13873 raft_consensus.cc:2955] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.225870 13771 catalog_manager.cc:5697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.228124 13771 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.230525 13771 catalog_manager.cc:5697] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.261647 13946 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:50.261672 13945 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:50.264147 13950 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:50.271577 14559 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:50.275960 14559 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:50.276140 14640 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:50.276391 13875 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:50.275950 13874 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:50.280489 14017 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:50.282697 14659 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:50.284068 14559 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:50.282387 14660 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:03:50.284667 14659 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:50.287820 14020 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:50.290192 14592 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:50.290417 14614 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:50.292181 14404 raft_consensus.cc:2955] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.296449 14015 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:50.298125 14614 raft_consensus.cc:2955] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.299018 14633 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:50.306550 14638 raft_consensus.cc:2955] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.311261 14559 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:50.310891 14590 raft_consensus.cc:2955] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.312702 14633 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:50.312884 14561 raft_consensus.cc:2955] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.319233 14560 raft_consensus.cc:2955] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.325873 13868 raft_consensus.cc:2955] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.328137 14558 raft_consensus.cc:2955] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.338954 14015 raft_consensus.cc:2955] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.339674 14020 raft_consensus.cc:2955] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.352058 13773 catalog_manager.cc:5697] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.366675 13757 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0e8d34f260fb43229460064c72d795e8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:50.337502 14559 raft_consensus.cc:2955] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.375711 13772 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:50.380330 13860 tablet_service.cc:1558] Processing DeleteTablet for tablet 4eeee4c6df564edcb3c96e2887208cc4 with delete_type TABLET_DATA_TOMBSTONED (TS e1649107c781428692458f10597c5742 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41672
I20260709 13:03:50.382524 13757 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f5c93fb008bd4a4f8db94cdc59c65d93 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:50.405012 14668 tablet_replica.cc:333] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:03:50.406418 13865 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e8d34f260fb43229460064c72d795e8 with delete_type TABLET_DATA_TOMBSTONED (TS e1649107c781428692458f10597c5742 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41672
I20260709 13:03:50.418962 13773 catalog_manager.cc:5697] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.420627 14020 raft_consensus.cc:2955] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:50.422554 13759 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 47fee46f08eb4928939d6ac41891db38 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:50.444057 13937 tablet_service.cc:1558] Processing DeleteTablet for tablet 47fee46f08eb4928939d6ac41891db38 with delete_type TABLET_DATA_TOMBSTONED (TS 3779649036364744823fc035dac892c7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35880
I20260709 13:03:50.456267 14668 raft_consensus.cc:2243] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:50.457222 14668 raft_consensus.cc:2272] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:50.454319 13772 catalog_manager.cc:5697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:50.492812 13757 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4eeee4c6df564edcb3c96e2887208cc4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:50.533037 14668 ts_tablet_manager.cc:1916] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:50.533792 14669 tablet_replica.cc:333] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:03:50.535665 14669 raft_consensus.cc:2243] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:50.542742 14012 tablet_service.cc:1558] Processing DeleteTablet for tablet f5c93fb008bd4a4f8db94cdc59c65d93 with delete_type TABLET_DATA_TOMBSTONED (TS d1b4fd04200247d99ba9ad1fbeb42b42 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49624
I20260709 13:03:50.543286 14669 raft_consensus.cc:2272] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:50.554581 14669 ts_tablet_manager.cc:1916] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:50.556234 14671 tablet_replica.cc:333] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
W20260709 13:03:50.555998 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 57017.1us
I20260709 13:03:50.565150 14671 raft_consensus.cc:2243] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:50.566119 14671 raft_consensus.cc:2272] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:50.574863 14671 ts_tablet_manager.cc:1916] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:50.605115 14669 ts_tablet_manager.cc:1929] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:50.605865 14669 log.cc:1199] T 47fee46f08eb4928939d6ac41891db38 P 3779649036364744823fc035dac892c7: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-1-root/wals/47fee46f08eb4928939d6ac41891db38
I20260709 13:03:50.608023 13757 catalog_manager.cc:5028] TS 3779649036364744823fc035dac892c7 (127.12.109.66:43953): tablet 47fee46f08eb4928939d6ac41891db38 (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:03:50.610526 14671 ts_tablet_manager.cc:1929] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:50.611047 14671 log.cc:1199] T f5c93fb008bd4a4f8db94cdc59c65d93 P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root/wals/f5c93fb008bd4a4f8db94cdc59c65d93
I20260709 13:03:50.612856 13759 catalog_manager.cc:5028] TS d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): tablet f5c93fb008bd4a4f8db94cdc59c65d93 (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:03:50.614279 14668 ts_tablet_manager.cc:1929] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:50.614779 14668 log.cc:1199] T 4eeee4c6df564edcb3c96e2887208cc4 P e1649107c781428692458f10597c5742: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/wals/4eeee4c6df564edcb3c96e2887208cc4
I20260709 13:03:50.616552 13759 catalog_manager.cc:5028] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): tablet 4eeee4c6df564edcb3c96e2887208cc4 (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:03:50.616698 14668 tablet_replica.cc:333] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:03:50.617756 14668 raft_consensus.cc:2243] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:50.618554 14668 raft_consensus.cc:2272] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:50.621349 14668 ts_tablet_manager.cc:1916] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:50.647128 14668 ts_tablet_manager.cc:1929] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:50.647585 14668 log.cc:1199] T 0e8d34f260fb43229460064c72d795e8 P e1649107c781428692458f10597c5742: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/wals/0e8d34f260fb43229460064c72d795e8
I20260709 13:03:50.649160 13759 catalog_manager.cc:5028] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): tablet 0e8d34f260fb43229460064c72d795e8 (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:03:51.682521 14015 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:51.692451 13945 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:51.695201 13875 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:51.701126 14627 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:51.702548 14672 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:51.710000 13988 consensus_peers.cc:597] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3f6783295084924801a104ff3faf10e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:51.715410 14672 raft_consensus.cc:2955] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.717168 13950 raft_consensus.cc:2955] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.722254 13875 raft_consensus.cc:2955] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.737972 13771 catalog_manager.cc:5697] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.748376 13950 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:03:51.759181 13914 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ef6c4cc997504124a8bb7fa9c7f10e34. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:51.759980 14015 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:51.762356 13873 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:51.763870 14673 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:51.766702 14614 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:51.775576 14614 raft_consensus.cc:2955] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.780344 14015 raft_consensus.cc:2955] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.784716 13873 raft_consensus.cc:2955] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.797766 13945 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:51.803982 13773 catalog_manager.cc:5697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:51.807960 13914 consensus_peers.cc:597] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 17bd99d4be2c4073bd867c161fa52482. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:51.811564 13875 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:51.811722 14015 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:51.814296 14592 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:51.817080 14592 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:51.845568 14673 raft_consensus.cc:2955] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.848492 13875 raft_consensus.cc:2955] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.863530 14017 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:51.871716 14015 raft_consensus.cc:2955] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.874343 13773 catalog_manager.cc:5697] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:51.878320 13988 consensus_peers.cc:597] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 208a6eaac64647a59ee173ef6726ae8b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:51.883034 13873 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:51.884792 14636 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:51.881340 13945 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:51.886713 14682 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:51.900183 14682 raft_consensus.cc:2955] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.902148 13875 raft_consensus.cc:2955] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.915490 13945 raft_consensus.cc:2955] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:51.931321 13771 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:52.191370 14696 ts_tablet_manager.cc:933] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:52.193384 14696 tablet_copy_client.cc:323] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.67:46599
I20260709 13:03:52.215732 14700 ts_tablet_manager.cc:933] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:52.223569 14700 tablet_copy_client.cc:323] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.66:43953
I20260709 13:03:52.224993 14034 tablet_copy_service.cc:140] P d1b4fd04200247d99ba9ad1fbeb42b42: Received BeginTabletCopySession request for tablet c3f6783295084924801a104ff3faf10e from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:55076)
I20260709 13:03:52.225692 14034 tablet_copy_service.cc:161] P d1b4fd04200247d99ba9ad1fbeb42b42: Beginning new tablet copy session on tablet c3f6783295084924801a104ff3faf10e from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:55076: session id = 4dd265d61fc141a48ca9d3d5c8f52921-c3f6783295084924801a104ff3faf10e
I20260709 13:03:52.225817 13960 tablet_copy_service.cc:140] P 3779649036364744823fc035dac892c7: Received BeginTabletCopySession request for tablet ef6c4cc997504124a8bb7fa9c7f10e34 from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:58096)
I20260709 13:03:52.226514 13960 tablet_copy_service.cc:161] P 3779649036364744823fc035dac892c7: Beginning new tablet copy session on tablet ef6c4cc997504124a8bb7fa9c7f10e34 from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:58096: session id = 4dd265d61fc141a48ca9d3d5c8f52921-ef6c4cc997504124a8bb7fa9c7f10e34
I20260709 13:03:52.240676 13960 tablet_copy_source_session.cc:215] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:52.247438 14034 tablet_copy_source_session.cc:215] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:52.247610 14700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef6c4cc997504124a8bb7fa9c7f10e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:52.250878 14696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3f6783295084924801a104ff3faf10e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:52.299193 14702 ts_tablet_manager.cc:933] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:52.311658 14702 tablet_copy_client.cc:323] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.66:43953
I20260709 13:03:52.312506 14700 tablet_copy_client.cc:806] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:52.313267 14700 tablet_copy_client.cc:670] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:52.313421 13960 tablet_copy_service.cc:140] P 3779649036364744823fc035dac892c7: Received BeginTabletCopySession request for tablet 17bd99d4be2c4073bd867c161fa52482 from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:58096)
I20260709 13:03:52.314088 13960 tablet_copy_service.cc:161] P 3779649036364744823fc035dac892c7: Beginning new tablet copy session on tablet 17bd99d4be2c4073bd867c161fa52482 from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:58096: session id = 4dd265d61fc141a48ca9d3d5c8f52921-17bd99d4be2c4073bd867c161fa52482
I20260709 13:03:52.315912 14696 tablet_copy_client.cc:806] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:52.316812 14696 tablet_copy_client.cc:670] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:52.317896 14700 tablet_copy_client.cc:538] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:52.327533 13960 tablet_copy_source_session.cc:215] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:52.328110 14696 tablet_copy_client.cc:538] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:52.332396 14702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17bd99d4be2c4073bd867c161fa52482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:52.339252 14700 tablet_bootstrap.cc:492] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:03:52.351956 14696 tablet_bootstrap.cc:492] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:03:52.355871 14702 tablet_copy_client.cc:806] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:52.356541 14702 tablet_copy_client.cc:670] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:52.381063 14702 tablet_copy_client.cc:538] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:52.382241 14700 tablet_bootstrap.cc:492] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:52.383327 14700 tablet_bootstrap.cc:492] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:03:52.385069 14700 ts_tablet_manager.cc:1403] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.046s	user 0.027s	sys 0.003s
I20260709 13:03:52.388556 14700 raft_consensus.cc:359] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:52.389644 14700 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:03:52.390393 14700 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:52.393311 14702 tablet_bootstrap.cc:492] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:03:52.425477 14696 tablet_bootstrap.cc:492] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:52.428660 14696 tablet_bootstrap.cc:492] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:03:52.433858 14696 ts_tablet_manager.cc:1403] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.082s	user 0.019s	sys 0.008s
I20260709 13:03:52.439016 14696 raft_consensus.cc:359] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:52.442690 14696 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:03:52.443490 14696 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:52.458737 14702 tablet_bootstrap.cc:492] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:52.461154 14702 tablet_bootstrap.cc:492] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:03:52.475884 14710 ts_tablet_manager.cc:933] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:52.472007 14696 ts_tablet_manager.cc:1434] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.003s
I20260709 13:03:52.484561 14702 ts_tablet_manager.cc:1403] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.083s	user 0.029s	sys 0.000s
I20260709 13:03:52.488216 14702 raft_consensus.cc:359] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:52.489342 14702 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:03:52.490238 14702 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:52.493185 14034 tablet_copy_service.cc:342] P d1b4fd04200247d99ba9ad1fbeb42b42: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-c3f6783295084924801a104ff3faf10e received from {username='slave'} at 127.0.0.1:55076
I20260709 13:03:52.494027 14034 tablet_copy_service.cc:434] P d1b4fd04200247d99ba9ad1fbeb42b42: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-c3f6783295084924801a104ff3faf10e on tablet c3f6783295084924801a104ff3faf10e with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:03:52.500191 14702 ts_tablet_manager.cc:1434] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:03:52.508464 13960 tablet_copy_service.cc:342] P 3779649036364744823fc035dac892c7: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-17bd99d4be2c4073bd867c161fa52482 received from {username='slave'} at 127.0.0.1:58096
I20260709 13:03:52.509085 13960 tablet_copy_service.cc:434] P 3779649036364744823fc035dac892c7: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-17bd99d4be2c4073bd867c161fa52482 on tablet 17bd99d4be2c4073bd867c161fa52482 with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:03:52.574957 14700 ts_tablet_manager.cc:1434] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.190s	user 0.048s	sys 0.078s
I20260709 13:03:52.581514 13960 tablet_copy_service.cc:342] P 3779649036364744823fc035dac892c7: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-ef6c4cc997504124a8bb7fa9c7f10e34 received from {username='slave'} at 127.0.0.1:58096
I20260709 13:03:52.582250 13960 tablet_copy_service.cc:434] P 3779649036364744823fc035dac892c7: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-ef6c4cc997504124a8bb7fa9c7f10e34 on tablet ef6c4cc997504124a8bb7fa9c7f10e34 with peer 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:03:52.580503 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 80786.7us
I20260709 13:03:52.590775 14559 raft_consensus.cc:1217] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:52.606892 14710 tablet_copy_client.cc:323] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.67:46599
I20260709 13:03:52.615118 14034 tablet_copy_service.cc:140] P d1b4fd04200247d99ba9ad1fbeb42b42: Received BeginTabletCopySession request for tablet 208a6eaac64647a59ee173ef6726ae8b from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:55076)
I20260709 13:03:52.615765 14034 tablet_copy_service.cc:161] P d1b4fd04200247d99ba9ad1fbeb42b42: Beginning new tablet copy session on tablet 208a6eaac64647a59ee173ef6726ae8b from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:55076: session id = 4dd265d61fc141a48ca9d3d5c8f52921-208a6eaac64647a59ee173ef6726ae8b
I20260709 13:03:52.624056 14034 tablet_copy_source_session.cc:215] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:52.629807 14710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208a6eaac64647a59ee173ef6726ae8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:52.648559 14710 tablet_copy_client.cc:806] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:52.649363 14710 tablet_copy_client.cc:670] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:52.653888 14710 tablet_copy_client.cc:538] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:52.663096 14710 tablet_bootstrap.cc:492] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:03:52.683645 14710 tablet_bootstrap.cc:492] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:52.684640 14710 tablet_bootstrap.cc:492] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:03:52.685391 14710 ts_tablet_manager.cc:1403] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.010s
I20260709 13:03:52.689213 14710 raft_consensus.cc:359] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:52.690184 14710 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:03:52.690878 14710 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:52.693801 14710 ts_tablet_manager.cc:1434] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.003s
I20260709 13:03:52.695740 14034 tablet_copy_service.cc:342] P d1b4fd04200247d99ba9ad1fbeb42b42: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-208a6eaac64647a59ee173ef6726ae8b received from {username='slave'} at 127.0.0.1:55076
I20260709 13:03:52.696200 14034 tablet_copy_service.cc:434] P d1b4fd04200247d99ba9ad1fbeb42b42: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-208a6eaac64647a59ee173ef6726ae8b on tablet 208a6eaac64647a59ee173ef6726ae8b with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:03:52.699908 14559 raft_consensus.cc:1217] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:52.790024 14559 raft_consensus.cc:1217] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:53.912595 14719 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:03:53.915478 14719 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:53.918321 14719 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:53.918951 14719 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:03:53.919509 14719 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:53.921103 14719 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:03:53.923555 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 193022us
I20260709 13:03:53.930757 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:53.939874 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:53.950775 14728 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:03:53.951589 14728 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:53.956394 14558 raft_consensus.cc:1217] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:53.956346 14728 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:03:53.978858 14728 raft_consensus.cc:1064] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:03:53.981819 14719 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:03:53.982703 14719 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:53.984782 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.985072 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:03:53.985777 14559 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:03:53.987534 13988 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:03:53.988924 14636 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:53.989382 14636 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:03:53.989727 14636 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:53.989902 14719 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:53.982404 14728 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:54.033667 14731 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:03:54.034446 14731 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:54.047034 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:54.054677 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:54.061409 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:54.065739 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:54.066157 14709 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:03:54.060381 14731 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:54.080480 14709 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:03:54.082383 14727 raft_consensus.cc:1064] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:03:54.084487 14592 raft_consensus.cc:1064] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:03:54.085095 14727 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:54.087968 14709 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:54.068128 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:54.092016 14592 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:54.070919 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:54.100008 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:03:54.049656 14636 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:54.105090 14021 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:54.111022 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:03:54.113495 14636 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:03:54.121301 14559 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.129844 14718 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:03:54.130539 14718 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:03:54.177121 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08001ecb80 after lost signal to thread 12728
W20260709 13:03:54.204490 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08000cb080 after lost signal to thread 13807
W20260709 13:03:54.206416 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08003008e0 after lost signal to thread 13899
W20260709 13:03:54.208689 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08002e28a0 after lost signal to thread 13974
I20260709 13:03:54.138101 14724 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
W20260709 13:03:54.195358 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 50540.2us
I20260709 13:03:54.211516 13875 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.199128 14718 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:54.212127 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:03:54.214314 14045 debug-util.cc:398] Leaking SignalData structure 0x7b0800266620 after lost signal to thread 14048
W20260709 13:03:54.218020 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.221477 14556 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.225165 14736 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:54.230782 13871 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.229737 14556 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.232901 14728 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:54.238955 14723 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:03:54.252166 14614 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:03:54.217301 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08002ecba0 after lost signal to thread 14585
I20260709 13:03:54.277107 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:54.277446 14728 raft_consensus.cc:2955] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.279400 13871 raft_consensus.cc:2955] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.282473 14709 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:54.283829 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:54.288393 13872 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.293131 14709 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:54.298923 14558 raft_consensus.cc:2955] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.308836 14727 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:54.315238 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:54.316416 14709 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:54.311798 13771 catalog_manager.cc:5697] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:54.325922 14723 raft_consensus.cc:2955] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.332310 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:54.335306 13872 raft_consensus.cc:2955] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.343115 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:03:54.342274 14727 raft_consensus.cc:2955] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.358722 14558 raft_consensus.cc:2955] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.348001 13875 raft_consensus.cc:2955] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.366879 14714 consensus_queue.cc:1059] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:54.368018 14714 raft_consensus.cc:3055] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:54.368353 14714 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:54.369210 14714 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:54.370800 14714 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:54.368844 13772 catalog_manager.cc:5697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:54.386620 14559 raft_consensus.cc:2955] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.392472 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:03:54.398896 13945 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:03:54.399932 13971 debug-util.cc:398] Leaking SignalData structure 0x7b0800110800 after lost signal to thread 12728
W20260709 13:03:54.400444 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9960 after lost signal to thread 13757
W20260709 13:03:54.400796 13971 debug-util.cc:398] Leaking SignalData structure 0x7b080012d4e0 after lost signal to thread 13761
W20260709 13:03:54.401059 13971 debug-util.cc:398] Leaking SignalData structure 0x7b080018c780 after lost signal to thread 13763
W20260709 13:03:54.401288 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08001f1fa0 after lost signal to thread 13765
W20260709 13:03:54.401578 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08001dd2c0 after lost signal to thread 13767
W20260709 13:03:54.401804 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08001dd2a0 after lost signal to thread 13768
W20260709 13:03:54.402004 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08000d7920 after lost signal to thread 13769
W20260709 13:03:54.402324 13971 debug-util.cc:398] Leaking SignalData structure 0x7b080013b8a0 after lost signal to thread 13772
W20260709 13:03:54.402583 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08001f1f40 after lost signal to thread 13773
W20260709 13:03:54.402850 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08001f1ec0 after lost signal to thread 13776
W20260709 13:03:54.403466 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08000c1d20 after lost signal to thread 13799
W20260709 13:03:54.403743 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08000c1da0 after lost signal to thread 13801
W20260709 13:03:54.403995 13971 debug-util.cc:398] Leaking SignalData structure 0x7b080018c620 after lost signal to thread 13802
W20260709 13:03:54.404318 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08000c98a0 after lost signal to thread 13807
W20260709 13:03:54.405449 13971 debug-util.cc:398] Leaking SignalData structure 0x7b0800173ac0 after lost signal to thread 13885
W20260709 13:03:54.405745 13971 debug-util.cc:398] Leaking SignalData structure 0x7b080015a5c0 after lost signal to thread 13887
W20260709 13:03:54.406008 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08001d7480 after lost signal to thread 13889
W20260709 13:03:54.407239 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08000b4e00 after lost signal to thread 13890
W20260709 13:03:54.407474 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08001d7300 after lost signal to thread 13891
W20260709 13:03:54.407825 13971 debug-util.cc:398] Leaking SignalData structure 0x7b0800235660 after lost signal to thread 13899
W20260709 13:03:54.409039 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08002b2be0 after lost signal to thread 13974
W20260709 13:03:54.410526 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08002b1fe0 after lost signal to thread 14048
W20260709 13:03:54.410789 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08002ae460 after lost signal to thread 14049
W20260709 13:03:54.411928 13971 debug-util.cc:398] Leaking SignalData structure 0x7b0800306700 after lost signal to thread 14585
I20260709 13:03:54.424702 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:54.425259 14020 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.425896 14023 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 2: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:03:54.429540 14023 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.460439 14714 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:54.461382 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:03:54.462738 14714 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:54.463853 14731 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:54.474574 14731 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:54.478318 14736 raft_consensus.cc:1064] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:03:54.478471 13987 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:54.481108 14728 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:54.481997 14736 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:54.481384 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:03:54.488849 14718 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:54.507032 13945 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:54.516255 13772 catalog_manager.cc:5697] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:54.540669 13871 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:54.540673 14558 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:54.536556 14023 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:54.545280 14724 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:54.547509 14724 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:54.552450 14558 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:54.553365 14614 raft_consensus.cc:2955] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.556079 13875 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:54.554149 14728 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:54.563014 14556 raft_consensus.cc:2955] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.567503 14709 raft_consensus.cc:2955] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.574608 13945 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:54.577589 14719 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:54.582103 14558 raft_consensus.cc:2955] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.590879 13872 raft_consensus.cc:2955] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.611433 13866 raft_consensus.cc:2955] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.619561 14559 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:54.621522 14723 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:54.631011 14559 raft_consensus.cc:2955] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:03:54.638444 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 157133us
I20260709 13:03:54.639393 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:03:54.640699 14728 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:54.643221 13871 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:54.647702 14714 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:54.657748 13769 catalog_manager.cc:5697] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 reported cstate change: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.663965 13773 catalog_manager.cc:5697] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.675313 13937 tablet_service.cc:1558] Processing DeleteTablet for tablet c3f6783295084924801a104ff3faf10e with delete_type TABLET_DATA_TOMBSTONED (TS 3779649036364744823fc035dac892c7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35880
I20260709 13:03:54.634402 14724 raft_consensus.cc:2955] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.682454 14748 tablet_replica.cc:333] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:03:54.690750 13759 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c3f6783295084924801a104ff3faf10e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:54.693769 13757 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ef6c4cc997504124a8bb7fa9c7f10e34 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:54.695778 13757 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 17bd99d4be2c4073bd867c161fa52482 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:03:54.706622 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 56668.8us
I20260709 13:03:54.714720 13872 raft_consensus.cc:2955] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:54.715895 14748 raft_consensus.cc:2243] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:54.732432 14748 raft_consensus.cc:2272] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:54.736246 14748 ts_tablet_manager.cc:1916] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:03:54.746591 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dd240 after lost signal to thread 12725
W20260709 13:03:54.747126 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dd2a0 after lost signal to thread 12728
W20260709 13:03:54.747393 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dd2e0 after lost signal to thread 12729
W20260709 13:03:54.747658 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc320 after lost signal to thread 12730
W20260709 13:03:54.747891 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc300 after lost signal to thread 13753
W20260709 13:03:54.748513 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dff80 after lost signal to thread 13779
W20260709 13:03:54.748865 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfee0 after lost signal to thread 13781
I20260709 13:03:54.748014 13772 catalog_manager.cc:5697] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:54.749150 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dff20 after lost signal to thread 13782
W20260709 13:03:54.749495 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfe40 after lost signal to thread 13786
W20260709 13:03:54.749779 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfe60 after lost signal to thread 13788
W20260709 13:03:54.750046 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfb60 after lost signal to thread 13790
W20260709 13:03:54.750367 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfde0 after lost signal to thread 13793
W20260709 13:03:54.750662 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfca0 after lost signal to thread 13796
W20260709 13:03:54.750917 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfce0 after lost signal to thread 13797
W20260709 13:03:54.751261 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfc60 after lost signal to thread 13804
W20260709 13:03:54.751518 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfc40 after lost signal to thread 13805
W20260709 13:03:54.751776 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfbc0 after lost signal to thread 13807
W20260709 13:03:54.752046 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfbe0 after lost signal to thread 13815
W20260709 13:03:54.752305 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfba0 after lost signal to thread 13817
W20260709 13:03:54.752535 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000df860 after lost signal to thread 13819
W20260709 13:03:54.752818 13896 debug-util.cc:398] Leaking SignalData structure 0x7b0800099860 after lost signal to thread 13821
W20260709 13:03:54.753103 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080009a580 after lost signal to thread 13823
W20260709 13:03:54.753378 13896 debug-util.cc:398] Leaking SignalData structure 0x7b0800096800 after lost signal to thread 13825
W20260709 13:03:54.753679 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9000 after lost signal to thread 13826
W20260709 13:03:54.753935 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9020 after lost signal to thread 13834
W20260709 13:03:54.754551 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c90a0 after lost signal to thread 13840
W20260709 13:03:54.754940 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c90c0 after lost signal to thread 13841
W20260709 13:03:54.755266 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9120 after lost signal to thread 13842
W20260709 13:03:54.755563 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c93a0 after lost signal to thread 13843
W20260709 13:03:54.755868 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9180 after lost signal to thread 13844
W20260709 13:03:54.756202 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9100 after lost signal to thread 13846
W20260709 13:03:54.756528 13896 debug-util.cc:398] Leaking SignalData structure 0x7b0800119640 after lost signal to thread 13847
W20260709 13:03:54.756865 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9200 after lost signal to thread 13849
W20260709 13:03:54.757213 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c91e0 after lost signal to thread 13850
W20260709 13:03:54.757520 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9240 after lost signal to thread 13852
W20260709 13:03:54.757844 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9280 after lost signal to thread 13853
W20260709 13:03:54.758253 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c92a0 after lost signal to thread 13854
W20260709 13:03:54.758569 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9260 after lost signal to thread 13855
W20260709 13:03:54.758896 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9220 after lost signal to thread 13857
W20260709 13:03:54.775671 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9380 after lost signal to thread 13858
W20260709 13:03:54.779925 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9320 after lost signal to thread 13859
I20260709 13:03:54.783501 13871 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:03:54.788189 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9340 after lost signal to thread 13861
I20260709 13:03:54.788326 14709 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:03:54.790748 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9420 after lost signal to thread 13862
W20260709 13:03:54.791528 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9400 after lost signal to thread 13864
W20260709 13:03:54.792237 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08001c5cc0 after lost signal to thread 13865
W20260709 13:03:54.792833 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08001aaa60 after lost signal to thread 13866
W20260709 13:03:54.793437 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08001ccba0 after lost signal to thread 13867
W20260709 13:03:54.794015 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08001aa960 after lost signal to thread 13868
W20260709 13:03:54.794693 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08001c5d80 after lost signal to thread 13871
W20260709 13:03:54.795332 13896 debug-util.cc:398] Leaking SignalData structure 0x7b0800157040 after lost signal to thread 13872
W20260709 13:03:54.795943 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc8a0 after lost signal to thread 13873
W20260709 13:03:54.796619 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc8c0 after lost signal to thread 13874
W20260709 13:03:54.797261 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc8e0 after lost signal to thread 13875
W20260709 13:03:54.797851 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc900 after lost signal to thread 13876
W20260709 13:03:54.801224 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc920 after lost signal to thread 13877
W20260709 13:03:54.801956 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc940 after lost signal to thread 13878
I20260709 13:03:54.802009 14559 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:03:54.803603 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc960 after lost signal to thread 13879
W20260709 13:03:54.804277 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc980 after lost signal to thread 13880
I20260709 13:03:54.804315 14709 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:03:54.810530 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc9a0 after lost signal to thread 13881
W20260709 13:03:54.812917 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc9c0 after lost signal to thread 13882
I20260709 13:03:54.823433 14012 tablet_service.cc:1558] Processing DeleteTablet for tablet ef6c4cc997504124a8bb7fa9c7f10e34 with delete_type TABLET_DATA_TOMBSTONED (TS d1b4fd04200247d99ba9ad1fbeb42b42 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49624
I20260709 13:03:54.842823 13944 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:03:54.828189 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08001fd200 after lost signal to thread 13893
I20260709 13:03:54.863853 14009 tablet_service.cc:1558] Processing DeleteTablet for tablet 17bd99d4be2c4073bd867c161fa52482 with delete_type TABLET_DATA_TOMBSTONED (TS d1b4fd04200247d99ba9ad1fbeb42b42 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49624
W20260709 13:03:54.903316 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000a0a20 after lost signal to thread 13899
W20260709 13:03:54.916026 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080031aa60 after lost signal to thread 13974
I20260709 13:03:54.877948 14718 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
W20260709 13:03:54.964533 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08002eb760 after lost signal to thread 14048
I20260709 13:03:54.853812 14023 raft_consensus.cc:2955] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:03:54.989144 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 138520us
I20260709 13:03:54.907120 14020 raft_consensus.cc:2955] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:03:54.992304 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080024e9e0 after lost signal to thread 14049
W20260709 13:03:55.014658 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08003033e0 after lost signal to thread 14531
W20260709 13:03:55.019344 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08002554c0 after lost signal to thread 14534
W20260709 13:03:55.019724 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080012f6e0 after lost signal to thread 14539
W20260709 13:03:55.020119 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000ec6e0 after lost signal to thread 14551
W20260709 13:03:55.020521 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08002ef380 after lost signal to thread 14565
W20260709 13:03:55.020921 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08002ef240 after lost signal to thread 14577
W20260709 13:03:55.021279 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080030a1a0 after lost signal to thread 14584
W20260709 13:03:55.021533 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080030a140 after lost signal to thread 14585
W20260709 13:03:55.023288 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 83275.3us
W20260709 13:03:55.038137 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 162853us
W20260709 13:03:55.045184 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.054298 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 63182us
W20260709 13:03:55.081403 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.276091 14752 tablet_replica.cc:333] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:03:55.277424 14752 raft_consensus.cc:2243] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:55.301827 14752 raft_consensus.cc:2272] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:55.320569 14748 ts_tablet_manager.cc:1929] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20260709 13:03:55.321302 14748 log.cc:1199] T c3f6783295084924801a104ff3faf10e P 3779649036364744823fc035dac892c7: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-1-root/wals/c3f6783295084924801a104ff3faf10e
I20260709 13:03:55.327762 14752 ts_tablet_manager.cc:1916] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:55.491866 13757 catalog_manager.cc:5028] TS 3779649036364744823fc035dac892c7 (127.12.109.66:43953): tablet c3f6783295084924801a104ff3faf10e (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
W20260709 13:03:55.505789 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 253047us
W20260709 13:03:55.519287 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 240519us
W20260709 13:03:55.502656 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 223651us
I20260709 13:03:55.575094 14752 ts_tablet_manager.cc:1929] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:55.584182 14752 log.cc:1199] T ef6c4cc997504124a8bb7fa9c7f10e34 P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root/wals/ef6c4cc997504124a8bb7fa9c7f10e34
I20260709 13:03:55.608886 14592 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.609589 14592 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:55.619240 14592 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:03:55.625528 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:03:55.631459 14752 tablet_replica.cc:333] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
W20260709 13:03:55.638935 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 110839us
I20260709 13:03:55.640198 14752 raft_consensus.cc:2243] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:55.637437 14556 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:03:55.647384 14769 raft_consensus.cc:2955] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:55.653954 13914 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:03:55.656213 14764 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:03:55.656081 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 85888.5us
I20260709 13:03:55.667016 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:03:55.674825 13945 raft_consensus.cc:2955] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:03:55.680291 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62508.3us
I20260709 13:03:55.661473 14764 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.682148 14764 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:55.668115 14018 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:03:55.673102 13759 catalog_manager.cc:5028] TS d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): tablet ef6c4cc997504124a8bb7fa9c7f10e34 (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:03:55.685503 14556 raft_consensus.cc:2955] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:55.671003 14752 raft_consensus.cc:2272] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:55.740338 14752 ts_tablet_manager.cc:1916] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:55.766954 13771 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:55.785187 14764 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:55.809405 13866 raft_consensus.cc:2955] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:55.844462 14752 ts_tablet_manager.cc:1929] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:55.845880 14752 log.cc:1199] T 17bd99d4be2c4073bd867c161fa52482 P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root/wals/17bd99d4be2c4073bd867c161fa52482
I20260709 13:03:55.849222 13759 catalog_manager.cc:5028] TS d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): tablet 17bd99d4be2c4073bd867c161fa52482 (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:03:55.851630 14764 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:03:55.852811 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:03:55.853771 14558 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:55.858814 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:03:55.859804 14018 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:55.903092 14558 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:03:55.933542 14018 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:03:55.955021 13914 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:03:55.961390 14638 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:55.962164 14638 raft_consensus.cc:697] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:03:55.963239 14638 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:56.019032 14018 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:03:56.027202 13770 catalog_manager.cc:5697] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:56.037256 13945 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:56.037853 14558 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:56.039402 14769 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:56.053517 14771 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:56.071813 14771 raft_consensus.cc:2955] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:56.092918 14558 raft_consensus.cc:2955] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:56.104113 13759 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 208a6eaac64647a59ee173ef6726ae8b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:56.107092 13770 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:56.134763 13860 tablet_service.cc:1558] Processing DeleteTablet for tablet 208a6eaac64647a59ee173ef6726ae8b with delete_type TABLET_DATA_TOMBSTONED (TS e1649107c781428692458f10597c5742 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41672
I20260709 13:03:56.156937 14783 tablet_replica.cc:333] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:03:56.158138 14783 raft_consensus.cc:2243] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:56.158990 14783 raft_consensus.cc:2272] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:56.170125 13947 raft_consensus.cc:2955] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:03:56.182950 14783 ts_tablet_manager.cc:1916] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:56.245159 14783 ts_tablet_manager.cc:1929] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:56.245662 14783 log.cc:1199] T 208a6eaac64647a59ee173ef6726ae8b P e1649107c781428692458f10597c5742: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/wals/208a6eaac64647a59ee173ef6726ae8b
I20260709 13:03:56.249085 13759 catalog_manager.cc:5028] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): tablet 208a6eaac64647a59ee173ef6726ae8b (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:03:56.438259 14022 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:56.440300 14714 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:56.499495 14558 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:56.501870 14766 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:57.175360 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 54486.9us
W20260709 13:03:57.178452 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 189453us
I20260709 13:03:57.525305 14024 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:57.549871 13872 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:57.551733 13988 consensus_peers.cc:597] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4daef29e1ece473f9845687b3c7eae60. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:57.552218 13945 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:57.554374 14736 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:57.556365 14728 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:57.641322 14791 raft_consensus.cc:2955] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:57.661684 13945 raft_consensus.cc:2955] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:57.666524 13769 catalog_manager.cc:5697] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:57.706987 14019 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:03:57.732300 13988 consensus_peers.cc:597] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9f8599b3d11641199dc0043a592857fb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:57.740693 13871 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:57.742539 14736 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:57.767506 13871 raft_consensus.cc:2955] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:57.779181 14790 raft_consensus.cc:2955] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
W20260709 13:03:57.820866 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51127.1us
I20260709 13:03:57.827330 13949 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:57.813592 13872 raft_consensus.cc:2955] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:57.879737 13769 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:57.915627 14771 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:58.010748 13949 raft_consensus.cc:2955] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
W20260709 13:03:58.089561 14801 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:03:58.264873 14802 ts_tablet_manager.cc:933] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:58.344898 14804 ts_tablet_manager.cc:933] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:58.350968 14802 tablet_copy_client.cc:323] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.67:46599
I20260709 13:03:58.356945 14804 tablet_copy_client.cc:323] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.67:46599
I20260709 13:03:58.437968 14022 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:03:58.454718 13988 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d6d5ae5feb04bda8f546e7e4ca80503. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:58.467170 13869 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:58.481236 14736 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:58.592502 14736 raft_consensus.cc:2955] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:58.595378 13869 raft_consensus.cc:2955] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
W20260709 13:03:58.611222 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 124483us
I20260709 13:03:58.633102 13769 catalog_manager.cc:5697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:58.670168 13946 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:58.678171 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55438.2us
I20260709 13:03:58.693284 13950 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:58.721513 14034 tablet_copy_service.cc:140] P d1b4fd04200247d99ba9ad1fbeb42b42: Received BeginTabletCopySession request for tablet 4daef29e1ece473f9845687b3c7eae60 from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:55076)
I20260709 13:03:58.722203 14034 tablet_copy_service.cc:161] P d1b4fd04200247d99ba9ad1fbeb42b42: Beginning new tablet copy session on tablet 4daef29e1ece473f9845687b3c7eae60 from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:55076: session id = 4dd265d61fc141a48ca9d3d5c8f52921-4daef29e1ece473f9845687b3c7eae60
I20260709 13:03:58.728742 14033 tablet_copy_service.cc:140] P d1b4fd04200247d99ba9ad1fbeb42b42: Received BeginTabletCopySession request for tablet 9f8599b3d11641199dc0043a592857fb from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:55076)
I20260709 13:03:58.734521 14033 tablet_copy_service.cc:161] P d1b4fd04200247d99ba9ad1fbeb42b42: Beginning new tablet copy session on tablet 9f8599b3d11641199dc0043a592857fb from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:55076: session id = 4dd265d61fc141a48ca9d3d5c8f52921-9f8599b3d11641199dc0043a592857fb
I20260709 13:03:58.739151 14736 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:58.789603 14033 tablet_copy_source_session.cc:215] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:58.776964 14034 tablet_copy_source_session.cc:215] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:03:58.793068 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 56757.7us
W20260709 13:03:58.829007 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 70946us
W20260709 13:03:58.830120 13914 consensus_peers.cc:597] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 05adb1eba4fe476d8504a36537b738db. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:58.846623 13869 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:58.859304 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 120634us
I20260709 13:03:58.869304 14820 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:58.872561 14802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4daef29e1ece473f9845687b3c7eae60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:58.887012 13945 raft_consensus.cc:2955] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:58.902567 14804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8599b3d11641199dc0043a592857fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:58.917919 14021 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:58.925719 14802 tablet_copy_client.cc:806] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:58.926442 14802 tablet_copy_client.cc:670] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:58.928966 14772 raft_consensus.cc:2955] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:58.934646 14804 tablet_copy_client.cc:806] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:58.935344 14804 tablet_copy_client.cc:670] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:58.956801 13869 raft_consensus.cc:2955] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:58.970425 14820 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:03:58.996992 14802 tablet_copy_client.cc:538] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:58.998426 14804 tablet_copy_client.cc:538] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:59.052778 14021 raft_consensus.cc:2955] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:03:59.094111 14826 ts_tablet_manager.cc:933] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:03:59.108140 14802 tablet_bootstrap.cc:492] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:03:59.114943 14804 tablet_bootstrap.cc:492] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:03:59.118736 14826 tablet_copy_client.cc:323] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.67:46599
I20260709 13:03:59.134951 14802 tablet_bootstrap.cc:492] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:59.136785 14802 tablet_bootstrap.cc:492] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:03:59.137881 14802 ts_tablet_manager.cc:1403] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.030s	user 0.024s	sys 0.000s
I20260709 13:03:59.142815 14802 raft_consensus.cc:359] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:59.144174 14802 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:03:59.146366 14802 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:59.149482 14802 ts_tablet_manager.cc:1434] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 13:03:59.155767 14804 tablet_bootstrap.cc:492] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:59.156922 14804 tablet_bootstrap.cc:492] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:03:59.157609 14804 ts_tablet_manager.cc:1403] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.043s	user 0.022s	sys 0.009s
I20260709 13:03:59.161198 14804 raft_consensus.cc:359] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:59.162041 14804 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:03:59.162837 14804 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:03:59.165819 14804 ts_tablet_manager.cc:1434] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
W20260709 13:03:59.248035 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 194442us
I20260709 13:03:59.254562 13769 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:59.345053 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 49692.6us
W20260709 13:03:59.345232 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 53246.3us
I20260709 13:03:59.436053 14558 raft_consensus.cc:1217] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:03:59.437880 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 62872.3us
I20260709 13:03:59.453999 14841 ts_tablet_manager.cc:933] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:03:59.458384 14841 tablet_copy_client.cc:323] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.66:43953
I20260709 13:03:59.516853 13960 tablet_copy_service.cc:140] P 3779649036364744823fc035dac892c7: Received BeginTabletCopySession request for tablet 05adb1eba4fe476d8504a36537b738db from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:58096)
I20260709 13:03:59.518272 13960 tablet_copy_service.cc:161] P 3779649036364744823fc035dac892c7: Beginning new tablet copy session on tablet 05adb1eba4fe476d8504a36537b738db from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:58096: session id = 4dd265d61fc141a48ca9d3d5c8f52921-05adb1eba4fe476d8504a36537b738db
I20260709 13:03:59.522509 14033 tablet_copy_service.cc:140] P d1b4fd04200247d99ba9ad1fbeb42b42: Received BeginTabletCopySession request for tablet 6d6d5ae5feb04bda8f546e7e4ca80503 from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:55076)
I20260709 13:03:59.523684 14033 tablet_copy_service.cc:161] P d1b4fd04200247d99ba9ad1fbeb42b42: Beginning new tablet copy session on tablet 6d6d5ae5feb04bda8f546e7e4ca80503 from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:55076: session id = 4dd265d61fc141a48ca9d3d5c8f52921-6d6d5ae5feb04bda8f546e7e4ca80503
I20260709 13:03:59.530891 14034 tablet_copy_service.cc:342] P d1b4fd04200247d99ba9ad1fbeb42b42: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-4daef29e1ece473f9845687b3c7eae60 received from {username='slave'} at 127.0.0.1:55076
I20260709 13:03:59.531874 14034 tablet_copy_service.cc:434] P d1b4fd04200247d99ba9ad1fbeb42b42: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-4daef29e1ece473f9845687b3c7eae60 on tablet 4daef29e1ece473f9845687b3c7eae60 with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:03:59.534687 14032 tablet_copy_service.cc:342] P d1b4fd04200247d99ba9ad1fbeb42b42: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-9f8599b3d11641199dc0043a592857fb received from {username='slave'} at 127.0.0.1:55076
I20260709 13:03:59.535451 14032 tablet_copy_service.cc:434] P d1b4fd04200247d99ba9ad1fbeb42b42: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-9f8599b3d11641199dc0043a592857fb on tablet 9f8599b3d11641199dc0043a592857fb with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:03:59.548743 14556 raft_consensus.cc:1217] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:03:59.707048 14033 tablet_copy_source_session.cc:215] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:59.718405 13960 tablet_copy_source_session.cc:215] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:59.724360 14841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05adb1eba4fe476d8504a36537b738db. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:59.728408 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 112792us
W20260709 13:03:59.731216 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 176348us
I20260709 13:03:59.744148 14826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d6d5ae5feb04bda8f546e7e4ca80503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:59.769380 14841 tablet_copy_client.cc:806] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:59.770390 14841 tablet_copy_client.cc:670] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:59.777009 14841 tablet_copy_client.cc:538] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:59.821377 14841 tablet_bootstrap.cc:492] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:03:59.826864 14845 raft_consensus.cc:1064] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:03:59.832964 14845 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:03:59.915496 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 60315us
W20260709 13:03:59.925329 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 70345.8us
I20260709 13:03:59.938782 14826 tablet_copy_client.cc:806] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:59.948995 14826 tablet_copy_client.cc:670] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:59.984104 13950 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:59.993589 14556 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:00.000948 14826 tablet_copy_client.cc:538] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:00.002872 14791 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:00.015666 14791 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:04:00.024612 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 69776.6us
I20260709 13:04:00.026877 13875 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:00.055776 14841 tablet_bootstrap.cc:492] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:00.057083 14841 tablet_bootstrap.cc:492] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:04:00.058432 14841 ts_tablet_manager.cc:1403] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.237s	user 0.013s	sys 0.021s
I20260709 13:04:00.070102 14835 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:00.087882 14841 raft_consensus.cc:359] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:00.090750 14841 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:00.091563 14841 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:00.209394 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62866.8us
I20260709 13:04:00.211089 14841 ts_tablet_manager.cc:1434] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.132s	user 0.008s	sys 0.014s
I20260709 13:04:00.228343 14835 raft_consensus.cc:2955] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.235827 13960 tablet_copy_service.cc:342] P 3779649036364744823fc035dac892c7: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-05adb1eba4fe476d8504a36537b738db received from {username='slave'} at 127.0.0.1:58096
I20260709 13:04:00.236560 13960 tablet_copy_service.cc:434] P 3779649036364744823fc035dac892c7: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-05adb1eba4fe476d8504a36537b738db on tablet 05adb1eba4fe476d8504a36537b738db with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:00.241617 13866 raft_consensus.cc:2955] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.212634 14837 raft_consensus.cc:1064] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:00.266712 14826 tablet_bootstrap.cc:492] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:04:00.268643 14837 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:00.297827 14561 raft_consensus.cc:2955] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:00.313162 14582 debug-util.cc:398] Leaking SignalData structure 0x7b08002aae20 after lost signal to thread 12728
W20260709 13:04:00.314810 14582 debug-util.cc:398] Leaking SignalData structure 0x7b08002a5b60 after lost signal to thread 13807
W20260709 13:04:00.317288 14582 debug-util.cc:398] Leaking SignalData structure 0x7b080032b260 after lost signal to thread 13899
W20260709 13:04:00.318976 14582 debug-util.cc:398] Leaking SignalData structure 0x7b08002e7a80 after lost signal to thread 13974
W20260709 13:04:00.326032 14582 debug-util.cc:398] Leaking SignalData structure 0x7b08002f7e00 after lost signal to thread 14048
I20260709 13:04:00.330262 13944 raft_consensus.cc:2955] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:00.338745 14582 debug-util.cc:398] Leaking SignalData structure 0x7b08002c61a0 after lost signal to thread 14585
I20260709 13:04:00.369865 13771 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:00.474273 14561 raft_consensus.cc:1217] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:00.503396 14561 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:00.505637 14819 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:00.508221 13944 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:00.518409 14837 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:04:00.520728 13867 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:00.533133 14819 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:00.601086 14826 tablet_bootstrap.cc:492] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:00.602285 14826 tablet_bootstrap.cc:492] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:04:00.603096 14826 ts_tablet_manager.cc:1403] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.337s	user 0.014s	sys 0.019s
I20260709 13:04:00.608619 14819 raft_consensus.cc:2955] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.613171 13875 raft_consensus.cc:2955] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.616231 13949 raft_consensus.cc:2955] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.650707 14826 raft_consensus.cc:359] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:00.651755 14826 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:00.652527 14826 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:00.657397 14826 ts_tablet_manager.cc:1434] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.054s	user 0.005s	sys 0.003s
I20260709 13:04:00.659614 14033 tablet_copy_service.cc:342] P d1b4fd04200247d99ba9ad1fbeb42b42: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-6d6d5ae5feb04bda8f546e7e4ca80503 received from {username='slave'} at 127.0.0.1:55076
I20260709 13:04:00.660128 14033 tablet_copy_service.cc:434] P d1b4fd04200247d99ba9ad1fbeb42b42: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-6d6d5ae5feb04bda8f546e7e4ca80503 on tablet 6d6d5ae5feb04bda8f546e7e4ca80503 with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:00.692354 14561 raft_consensus.cc:2955] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.775267 14018 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:00.785615 14561 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:00.787490 14819 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:00.786885 13769 catalog_manager.cc:5697] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.789669 13949 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:00.803377 14845 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:00.804491 14728 raft_consensus.cc:2955] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.812179 14556 raft_consensus.cc:2955] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.831332 13949 raft_consensus.cc:2955] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.835335 14018 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:00.848021 14561 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:04:00.849803 14845 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:00.850140 13867 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:04:00.853685 14835 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:00.858789 14837 raft_consensus.cc:2955] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.881579 14848 raft_consensus.cc:1064] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:00.884778 14848 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:00.888429 13867 raft_consensus.cc:2955] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.869063 14558 raft_consensus.cc:2955] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:00.922555 13759 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4daef29e1ece473f9845687b3c7eae60 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:00.925194 14561 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:00.927328 14849 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:00.946852 13875 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:00.949012 14849 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:00.966213 13759 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9f8599b3d11641199dc0043a592857fb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:01.025545 14561 raft_consensus.cc:1217] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:01.042876 14021 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:01.050832 13772 catalog_manager.cc:5697] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:01.053824 13769 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.095674 13860 tablet_service.cc:1558] Processing DeleteTablet for tablet 9f8599b3d11641199dc0043a592857fb with delete_type TABLET_DATA_TOMBSTONED (TS e1649107c781428692458f10597c5742 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41672
I20260709 13:04:01.139870 13937 tablet_service.cc:1558] Processing DeleteTablet for tablet 4daef29e1ece473f9845687b3c7eae60 with delete_type TABLET_DATA_TOMBSTONED (TS 3779649036364744823fc035dac892c7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35880
I20260709 13:04:01.159909 14876 tablet_replica.cc:333] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:04:01.195425 14714 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:04:01.209050 14872 tablet_replica.cc:333] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:04:01.210335 14872 raft_consensus.cc:2243] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.216658 14872 raft_consensus.cc:2272] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.232673 14876 raft_consensus.cc:2243] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.233880 14876 raft_consensus.cc:2272] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.239079 14872 ts_tablet_manager.cc:1916] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:01.242939 14876 ts_tablet_manager.cc:1916] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:04:01.251868 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 44592.9us
I20260709 13:04:01.317309 14873 raft_consensus.cc:2955] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.351639 14556 raft_consensus.cc:2955] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.381660 13869 raft_consensus.cc:2955] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.389490 13771 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.429440 14876 ts_tablet_manager.cc:1929] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:04:01.433308 13950 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:01.463034 14558 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:01.465101 14871 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:01.465674 14876 log.cc:1199] T 4daef29e1ece473f9845687b3c7eae60 P 3779649036364744823fc035dac892c7: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-1-root/wals/4daef29e1ece473f9845687b3c7eae60
I20260709 13:04:01.475268 14847 raft_consensus.cc:2955] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.479736 14872 ts_tablet_manager.cc:1929] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:01.480334 14872 log.cc:1199] T 9f8599b3d11641199dc0043a592857fb P e1649107c781428692458f10597c5742: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/wals/9f8599b3d11641199dc0043a592857fb
I20260709 13:04:01.480383 14558 raft_consensus.cc:2955] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.491267 14870 raft_consensus.cc:1064] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:01.495505 14870 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:01.503408 13875 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:01.515317 13757 catalog_manager.cc:5028] TS 3779649036364744823fc035dac892c7 (127.12.109.66:43953): tablet 4daef29e1ece473f9845687b3c7eae60 (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:04:01.516396 14873 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:01.522254 13875 raft_consensus.cc:2955] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.530912 13759 catalog_manager.cc:5028] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): tablet 9f8599b3d11641199dc0043a592857fb (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:04:01.569617 13773 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.587651 14012 tablet_service.cc:1558] Processing DeleteTablet for tablet 05adb1eba4fe476d8504a36537b738db with delete_type TABLET_DATA_TOMBSTONED (TS d1b4fd04200247d99ba9ad1fbeb42b42 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49624
I20260709 13:04:01.617403 14882 tablet_replica.cc:333] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:04:01.622860 13950 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:01.623272 14558 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:01.627705 14869 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:01.632371 13757 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 05adb1eba4fe476d8504a36537b738db with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:01.644670 13875 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:01.645808 14881 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:01.651548 14869 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:01.654862 14882 raft_consensus.cc:2243] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.664630 14882 pending_rounds.cc:70] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42: Trying to abort 1 pending ops.
I20260709 13:04:01.665115 14882 pending_rounds.cc:77] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305634778656292864 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "05adb1eba4fe476d8504a36537b738db" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } } }
I20260709 13:04:01.678649 14882 raft_consensus.cc:2917] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 13:04:01.679425 14882 raft_consensus.cc:2889] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:04:01.679929 14882 raft_consensus.cc:2272] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.701119 14882 ts_tablet_manager.cc:1916] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:01.750893 14881 raft_consensus.cc:2955] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.753125 14558 raft_consensus.cc:2955] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.771777 13949 raft_consensus.cc:2955] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.791208 14882 ts_tablet_manager.cc:1929] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:01.791814 14882 log.cc:1199] T 05adb1eba4fe476d8504a36537b738db P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root/wals/05adb1eba4fe476d8504a36537b738db
I20260709 13:04:01.798735 13759 catalog_manager.cc:5028] TS d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): tablet 05adb1eba4fe476d8504a36537b738db (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:04:01.799858 13773 catalog_manager.cc:5697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.800184 13875 raft_consensus.cc:2955] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.836267 14018 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:01.877844 14558 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:01.881873 14881 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:01.889765 14870 raft_consensus.cc:2955] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.898257 13870 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:01.904440 14556 raft_consensus.cc:2955] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:01.920111 13759 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6d6d5ae5feb04bda8f546e7e4ca80503 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:01.921547 14870 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:01.927834 13870 raft_consensus.cc:2955] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:01.947253 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 65978.2us
I20260709 13:04:02.004576 13773 catalog_manager.cc:5697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:02.027925 13937 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d6d5ae5feb04bda8f546e7e4ca80503 with delete_type TABLET_DATA_TOMBSTONED (TS 3779649036364744823fc035dac892c7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35880
I20260709 13:04:02.040077 14887 tablet_replica.cc:333] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:04:02.041520 14887 raft_consensus.cc:2243] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:02.058499 14887 raft_consensus.cc:2272] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:02.067309 14887 ts_tablet_manager.cc:1916] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:02.127252 14887 ts_tablet_manager.cc:1929] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:02.127741 14887 log.cc:1199] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 3779649036364744823fc035dac892c7: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-1-root/wals/6d6d5ae5feb04bda8f546e7e4ca80503
I20260709 13:04:02.129438 13757 catalog_manager.cc:5028] TS 3779649036364744823fc035dac892c7 (127.12.109.66:43953): tablet 6d6d5ae5feb04bda8f546e7e4ca80503 (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:04:03.433890 14018 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:03.506664 13988 consensus_peers.cc:597] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 45103ab867574475a587979f8ea5fa4e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:03.519344 13870 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:03.521090 14869 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:03.524390 13950 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:03.541296 14869 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:03.555305 14837 raft_consensus.cc:2955] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:03.558882 13869 raft_consensus.cc:2955] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:03.571198 13945 raft_consensus.cc:2955] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:03.583964 13773 catalog_manager.cc:5697] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:03.599694 14018 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:03.688918 13988 consensus_peers.cc:597] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a2c65bca5ce45d5af2ace962fa0a37f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:03.700785 13869 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:03.702716 14869 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:03.734754 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 45004.2us
I20260709 13:04:03.743288 14837 raft_consensus.cc:2955] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:03.755689 13875 raft_consensus.cc:2955] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:03.783970 13945 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:03.803237 14870 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:03.839272 13945 raft_consensus.cc:2955] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:03.844837 13875 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:03.868372 13950 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:03.869014 13839 consensus_peers.cc:597] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 72ee2fe56c0b4dd5bf2a9c07597c79ba. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:03.870779 14886 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:03.905014 13771 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:03.940091 14024 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:03.943859 14902 ts_tablet_manager.cc:933] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:03.956302 14886 raft_consensus.cc:2955] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:03.961213 13949 raft_consensus.cc:2955] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:03.993700 14902 tablet_copy_client.cc:323] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.67:46599
I20260709 13:04:03.993878 14844 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:03.996160 14033 tablet_copy_service.cc:140] P d1b4fd04200247d99ba9ad1fbeb42b42: Received BeginTabletCopySession request for tablet 45103ab867574475a587979f8ea5fa4e from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:55076)
I20260709 13:04:03.996793 14033 tablet_copy_service.cc:161] P d1b4fd04200247d99ba9ad1fbeb42b42: Beginning new tablet copy session on tablet 45103ab867574475a587979f8ea5fa4e from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:55076: session id = 4dd265d61fc141a48ca9d3d5c8f52921-45103ab867574475a587979f8ea5fa4e
I20260709 13:04:04.010110 13773 catalog_manager.cc:5697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:04.016265 14033 tablet_copy_source_session.cc:215] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:04.025274 13867 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:04.046623 14902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45103ab867574475a587979f8ea5fa4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:04.047550 14024 raft_consensus.cc:2955] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
W20260709 13:04:04.124305 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 83884.6us
I20260709 13:04:04.134392 13950 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:04.141002 13839 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 82051b9bd85845d6b73773b21ccc6d6c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:04.145406 14844 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:04.163271 14021 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:04.174157 14906 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:04.181623 14902 tablet_copy_client.cc:806] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:04.182533 14902 tablet_copy_client.cc:670] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:04.195770 14902 tablet_copy_client.cc:538] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:04.220646 14912 ts_tablet_manager.cc:933] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:04.222026 13949 raft_consensus.cc:2955] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:04.220785 14904 raft_consensus.cc:2955] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:04.253044 14021 raft_consensus.cc:2955] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:04.255574 14912 tablet_copy_client.cc:323] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.67:46599
I20260709 13:04:04.240690 14902 tablet_bootstrap.cc:492] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:04:04.257577 14033 tablet_copy_service.cc:140] P d1b4fd04200247d99ba9ad1fbeb42b42: Received BeginTabletCopySession request for tablet 9a2c65bca5ce45d5af2ace962fa0a37f from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:55076)
I20260709 13:04:04.258208 14033 tablet_copy_service.cc:161] P d1b4fd04200247d99ba9ad1fbeb42b42: Beginning new tablet copy session on tablet 9a2c65bca5ce45d5af2ace962fa0a37f from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:55076: session id = 4dd265d61fc141a48ca9d3d5c8f52921-9a2c65bca5ce45d5af2ace962fa0a37f
I20260709 13:04:04.276161 13773 catalog_manager.cc:5697] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:04.284698 14033 tablet_copy_source_session.cc:215] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:04.308218 14912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a2c65bca5ce45d5af2ace962fa0a37f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:04.324129 14914 ts_tablet_manager.cc:933] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer e1649107c781428692458f10597c5742 (127.12.109.65:43607)
I20260709 13:04:04.340209 14902 tablet_bootstrap.cc:492] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:04.341300 14902 tablet_bootstrap.cc:492] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:04:04.346282 14902 ts_tablet_manager.cc:1403] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.107s	user 0.025s	sys 0.006s
I20260709 13:04:04.359236 14902 raft_consensus.cc:359] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:04.360309 14902 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:04.361083 14902 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:04.379757 14912 tablet_copy_client.cc:806] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:04.386612 14912 tablet_copy_client.cc:670] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:04.385470 14902 ts_tablet_manager.cc:1434] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.001s
I20260709 13:04:04.391633 14912 tablet_copy_client.cc:538] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:04.416656 14033 tablet_copy_service.cc:342] P d1b4fd04200247d99ba9ad1fbeb42b42: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-45103ab867574475a587979f8ea5fa4e received from {username='slave'} at 127.0.0.1:55076
I20260709 13:04:04.417171 14033 tablet_copy_service.cc:434] P d1b4fd04200247d99ba9ad1fbeb42b42: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-45103ab867574475a587979f8ea5fa4e on tablet 45103ab867574475a587979f8ea5fa4e with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:04.447458 14912 tablet_bootstrap.cc:492] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:04:04.472911 14914 tablet_copy_client.cc:323] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.65:43607
I20260709 13:04:04.475301 13885 tablet_copy_service.cc:140] P e1649107c781428692458f10597c5742: Received BeginTabletCopySession request for tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:42526)
I20260709 13:04:04.475991 13885 tablet_copy_service.cc:161] P e1649107c781428692458f10597c5742: Beginning new tablet copy session on tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:42526: session id = 4dd265d61fc141a48ca9d3d5c8f52921-72ee2fe56c0b4dd5bf2a9c07597c79ba
I20260709 13:04:04.495551 13885 tablet_copy_source_session.cc:215] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:04.508239 14914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:04.610167 14914 tablet_copy_client.cc:806] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:04.610961 14914 tablet_copy_client.cc:670] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:04.614519 14912 tablet_bootstrap.cc:492] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:04.615634 14912 tablet_bootstrap.cc:492] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:04:04.616434 14912 ts_tablet_manager.cc:1403] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.169s	user 0.026s	sys 0.002s
I20260709 13:04:04.622175 14912 raft_consensus.cc:359] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:04.623108 14912 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:04.623792 14912 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:04.634214 14912 ts_tablet_manager.cc:1434] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:04:04.639086 14033 tablet_copy_service.cc:342] P d1b4fd04200247d99ba9ad1fbeb42b42: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-9a2c65bca5ce45d5af2ace962fa0a37f received from {username='slave'} at 127.0.0.1:55076
I20260709 13:04:04.639591 14033 tablet_copy_service.cc:434] P d1b4fd04200247d99ba9ad1fbeb42b42: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-9a2c65bca5ce45d5af2ace962fa0a37f on tablet 9a2c65bca5ce45d5af2ace962fa0a37f with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:04.646387 14914 tablet_copy_client.cc:538] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:04.688378 14914 tablet_bootstrap.cc:492] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:04:04.730496 14912 ts_tablet_manager.cc:933] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer e1649107c781428692458f10597c5742 (127.12.109.65:43607)
I20260709 13:04:04.741510 14912 tablet_copy_client.cc:323] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.65:43607
I20260709 13:04:04.743858 13885 tablet_copy_service.cc:140] P e1649107c781428692458f10597c5742: Received BeginTabletCopySession request for tablet 82051b9bd85845d6b73773b21ccc6d6c from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:42526)
I20260709 13:04:04.744585 13885 tablet_copy_service.cc:161] P e1649107c781428692458f10597c5742: Beginning new tablet copy session on tablet 82051b9bd85845d6b73773b21ccc6d6c from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:42526: session id = 4dd265d61fc141a48ca9d3d5c8f52921-82051b9bd85845d6b73773b21ccc6d6c
I20260709 13:04:04.754177 13885 tablet_copy_source_session.cc:215] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:04.758316 14912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82051b9bd85845d6b73773b21ccc6d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:04.891048 14914 tablet_bootstrap.cc:492] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:04.892881 14914 tablet_bootstrap.cc:492] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:04:04.893798 14914 ts_tablet_manager.cc:1403] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.206s	user 0.028s	sys 0.006s
I20260709 13:04:04.901305 14912 tablet_copy_client.cc:806] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:04.902122 14912 tablet_copy_client.cc:670] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:04.907456 14912 tablet_copy_client.cc:538] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:04.915861 14561 raft_consensus.cc:1217] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:04.939702 14914 raft_consensus.cc:359] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:04.940730 14914 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:04.941511 14914 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:04.944483 14914 ts_tablet_manager.cc:1434] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 13:04:04.947348 13885 tablet_copy_service.cc:342] P e1649107c781428692458f10597c5742: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-72ee2fe56c0b4dd5bf2a9c07597c79ba received from {username='slave'} at 127.0.0.1:42526
I20260709 13:04:04.947904 13885 tablet_copy_service.cc:434] P e1649107c781428692458f10597c5742: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-72ee2fe56c0b4dd5bf2a9c07597c79ba on tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:05.007257 14912 tablet_bootstrap.cc:492] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:04:05.053544 14912 tablet_bootstrap.cc:492] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:05.054664 14912 tablet_bootstrap.cc:492] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:04:05.055414 14912 ts_tablet_manager.cc:1403] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.048s	user 0.019s	sys 0.005s
I20260709 13:04:05.058921 14912 raft_consensus.cc:359] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:05.059896 14912 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:05.060637 14912 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:05.073036 14912 ts_tablet_manager.cc:1434] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 13:04:05.091755 13885 tablet_copy_service.cc:342] P e1649107c781428692458f10597c5742: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-82051b9bd85845d6b73773b21ccc6d6c received from {username='slave'} at 127.0.0.1:42526
I20260709 13:04:05.092322 13885 tablet_copy_service.cc:434] P e1649107c781428692458f10597c5742: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-82051b9bd85845d6b73773b21ccc6d6c on tablet 82051b9bd85845d6b73773b21ccc6d6c with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:05.107337 14561 raft_consensus.cc:1217] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:05.137338 14561 raft_consensus.cc:1217] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:05.258693 14561 raft_consensus.cc:1217] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:05.459841 14870 raft_consensus.cc:1064] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:05.466444 14870 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:05.505784 14904 raft_consensus.cc:1064] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:05.508869 14904 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:05.566234 14558 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:05.570477 14869 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:05.572906 13945 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:05.580518 14881 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:05.602406 13871 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:05.614238 14909 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:05.624416 13949 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:05.631443 14844 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:05.639117 14018 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:05.657891 14905 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:04:05.710254 14558 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:05.719944 14909 raft_consensus.cc:1064] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:05.723287 14909 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:05.756703 14904 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:05.769481 14869 raft_consensus.cc:2955] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:05.777415 13945 raft_consensus.cc:2955] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:05.788633 13867 raft_consensus.cc:2955] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:05.810918 14558 raft_consensus.cc:2955] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:05.841075 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62819.4us
W20260709 13:04:05.847904 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 55260.6us
I20260709 13:04:05.880218 13769 catalog_manager.cc:5697] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:05.928047 14844 raft_consensus.cc:2955] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:05.953382 14018 raft_consensus.cc:2955] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:05.996534 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 153100us
I20260709 13:04:05.998311 13950 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:05.999181 14558 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:05.999783 14904 raft_consensus.cc:1064] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:06.008575 14904 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:06.012869 13875 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:06.014024 14869 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:06.015132 14870 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:06.017264 14881 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:06.000981 14559 raft_consensus.cc:2955] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.062044 14561 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:06.063951 13945 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:06.064117 14024 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:06.064267 14905 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:04:06.119758 14935 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:06.123432 14929 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:04:06.141963 13945 raft_consensus.cc:2955] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.142318 14909 raft_consensus.cc:2955] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.189162 14561 raft_consensus.cc:2955] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.166613 13946 raft_consensus.cc:2955] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.307255 14844 raft_consensus.cc:2955] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.318358 13871 raft_consensus.cc:2955] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.324720 14559 raft_consensus.cc:2955] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.351403 13946 raft_consensus.cc:2955] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.364038 13772 catalog_manager.cc:5697] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:06.387954 14018 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:06.391849 13771 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.394260 13870 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:06.403378 14559 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:06.424322 14905 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:06.433367 14905 raft_consensus.cc:2955] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.444391 14559 raft_consensus.cc:2955] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.445271 13773 catalog_manager.cc:5697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.457491 14024 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:06.461135 14558 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:06.471938 14837 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:06.479626 14939 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:06.487571 14870 raft_consensus.cc:2955] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.455893 14018 raft_consensus.cc:2955] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.494962 13947 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:06.493270 13759 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 82051b9bd85845d6b73773b21ccc6d6c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:06.492501 14558 raft_consensus.cc:2955] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.505435 14019 raft_consensus.cc:2955] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.513417 14021 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:06.516443 14909 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:06.519106 14561 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:06.520701 14909 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:06.520942 13870 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:06.527789 14837 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:06.531914 14881 raft_consensus.cc:2955] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.533830 13870 raft_consensus.cc:2955] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.547168 13946 raft_consensus.cc:2955] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.549131 13771 catalog_manager.cc:5697] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.539886 14561 raft_consensus.cc:2955] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.575989 13759 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 45103ab867574475a587979f8ea5fa4e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:06.575058 13867 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:06.582391 14559 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:06.584272 14844 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:06.584811 13947 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:06.587420 14844 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:04:06.592723 13937 tablet_service.cc:1558] Processing DeleteTablet for tablet 82051b9bd85845d6b73773b21ccc6d6c with delete_type TABLET_DATA_TOMBSTONED (TS 3779649036364744823fc035dac892c7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35880
I20260709 13:04:06.598848 13759 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9a2c65bca5ce45d5af2ace962fa0a37f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:06.596973 13773 catalog_manager.cc:5697] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 3 to 4, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.602938 14939 raft_consensus.cc:2955] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.622915 13947 raft_consensus.cc:2955] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.632527 14559 raft_consensus.cc:2955] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.625648 13772 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index 3 to 4, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:06.658228 13759 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:06.694679 13940 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a2c65bca5ce45d5af2ace962fa0a37f with delete_type TABLET_DATA_TOMBSTONED (TS 3779649036364744823fc035dac892c7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35880
I20260709 13:04:06.694707 14953 tablet_replica.cc:333] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:04:06.698385 13773 catalog_manager.cc:5697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:06.702389 13860 tablet_service.cc:1558] Processing DeleteTablet for tablet 45103ab867574475a587979f8ea5fa4e with delete_type TABLET_DATA_TOMBSTONED (TS e1649107c781428692458f10597c5742 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41672
I20260709 13:04:06.702615 14953 raft_consensus.cc:2243] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.703547 14953 raft_consensus.cc:2272] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.743288 14012 tablet_service.cc:1558] Processing DeleteTablet for tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba with delete_type TABLET_DATA_TOMBSTONED (TS d1b4fd04200247d99ba9ad1fbeb42b42 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49624
I20260709 13:04:06.761106 14953 ts_tablet_manager.cc:1916] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:04:06.762336 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 54309.9us
I20260709 13:04:06.775789 14955 tablet_replica.cc:333] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:04:06.777902 14955 raft_consensus.cc:2243] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.779026 14955 raft_consensus.cc:2272] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.782251 14954 tablet_replica.cc:333] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:04:06.788537 14954 raft_consensus.cc:2243] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.804522 14954 raft_consensus.cc:2272] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.807701 14955 ts_tablet_manager.cc:1916] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:06.855402 14954 ts_tablet_manager.cc:1916] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:06.892447 14955 ts_tablet_manager.cc:1929] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:06.893294 14955 log.cc:1199] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root/wals/72ee2fe56c0b4dd5bf2a9c07597c79ba
I20260709 13:04:06.913276 14953 ts_tablet_manager.cc:1929] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:06.913892 14953 log.cc:1199] T 82051b9bd85845d6b73773b21ccc6d6c P 3779649036364744823fc035dac892c7: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-1-root/wals/82051b9bd85845d6b73773b21ccc6d6c
I20260709 13:04:06.920861 14953 tablet_replica.cc:333] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:04:06.941012 14954 ts_tablet_manager.cc:1929] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:06.941655 14954 log.cc:1199] T 45103ab867574475a587979f8ea5fa4e P e1649107c781428692458f10597c5742: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/wals/45103ab867574475a587979f8ea5fa4e
I20260709 13:04:06.942449 13759 catalog_manager.cc:5028] TS d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): tablet 72ee2fe56c0b4dd5bf2a9c07597c79ba (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:04:07.078580 13759 catalog_manager.cc:5028] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): tablet 45103ab867574475a587979f8ea5fa4e (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:04:07.661185 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:07.667201 14956 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:07.668157 14956 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:07.673834 14956 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:07.809537 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 147013us
I20260709 13:04:07.843382 14905 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:07.844122 14905 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:07.862331 14905 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:07.899813 14905 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:07.900465 14905 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:07.903522 14953 raft_consensus.cc:2243] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.904484 14953 raft_consensus.cc:2272] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.904589 14905 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:07.900454 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:07.908146 14953 ts_tablet_manager.cc:1916] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:07.919199 14905 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:07.920878 14905 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:07.927359 14905 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:07.931859 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:07.935809 13990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.002s	sys 0.006s
W20260709 13:04:07.936302 13990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.003s	sys 0.006s
I20260709 13:04:07.935701 14905 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:07.937705 14905 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:07.940039 14905 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:07.942866 14904 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:07.956521 14904 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:07.985510 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 72199.6us
W20260709 13:04:07.988854 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48919.9us
I20260709 13:04:07.829614 13757 catalog_manager.cc:5028] TS 3779649036364744823fc035dac892c7 (127.12.109.66:43953): tablet 82051b9bd85845d6b73773b21ccc6d6c (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
W20260709 13:04:08.002579 13757 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 172988us
I20260709 13:04:07.965986 14904 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:08.008463 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.009491 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:08.010568 14904 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:08.013346 14904 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:08.015333 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080030a1c0 after lost signal to thread 12728
I20260709 13:04:08.015672 14904 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:08.016518 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08002529c0 after lost signal to thread 13807
W20260709 13:04:08.017786 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dd080 after lost signal to thread 13899
W20260709 13:04:08.017972 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000dd040 after lost signal to thread 13900
I20260709 13:04:08.019707 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.020529 14837 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:08.020517 14965 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:08.021289 14958 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:08.021591 14965 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.021301 14837 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.022259 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 1204) took 1008 ms. Trace:
I20260709 13:04:08.024314 13947 rpcz_store.cc:276] 0709 13:04:07.014158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.021183 (+1007025us) service_pool.cc:224] Handling call
0709 13:04:08.021540 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.021830 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.021859 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:08.021910 (+    51us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:08.021955 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.022023 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.022196 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:08.025511 14965 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:08.021961 14958 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.025993 14959 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:08.030277 14964 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:04:08.030448 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000df100 after lost signal to thread 13974
I20260709 13:04:08.030618 14837 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:08.031832 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080028c320 after lost signal to thread 14048
W20260709 13:04:08.032886 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08002f82a0 after lost signal to thread 14585
I20260709 13:04:08.030939 14964 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.040390 14964 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:08.047206 14966 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:08.049350 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.050405 14844 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:08.050678 14964 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:08.051334 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.050980 14844 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.052083 14964 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.052485 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.051301 14965 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:08.053249 14965 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:08.055390 14965 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:08.052582 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.050809 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.056710 14844 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:08.026558 14959 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.058985 14959 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:08.059641 14904 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:08.060588 14958 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:08.063406 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.064075 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.069976 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:08.067371 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.066663 14963 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:08.068200 14964 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:08.073549 14904 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.072003 14963 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.078514 14904 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:08.047998 14966 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.074800 14958 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:08.083072 14958 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.086403 14966 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:08.112349 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.125073 14963 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:08.126976 14958 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:08.128334 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.128705 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.133738 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.137789 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.140025 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.141039 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:08.151268 14963 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:08.151221 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.152123 14963 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.153399 13915 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:08.154441 13915 raft_consensus.cc:2686] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:04:08.158301 14963 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:08.164820 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:08.165472 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:08.165104 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:08.166548 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:08.167011 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:08.167755 13869 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:04:08.174368 13945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:08.175258 13945 raft_consensus.cc:606] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:08.182320 13945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:08.182991 13945 raft_consensus.cc:606] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:08.194941 14953 ts_tablet_manager.cc:1929] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:08.195227 14931 raft_consensus.cc:993] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:08.195578 14953 log.cc:1199] T 9a2c65bca5ce45d5af2ace962fa0a37f P 3779649036364744823fc035dac892c7: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-1-root/wals/9a2c65bca5ce45d5af2ace962fa0a37f
I20260709 13:04:08.195753 14931 raft_consensus.cc:1081] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:08.196825 14953 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35880 (request call id 70) took 1508 ms. Trace:
I20260709 13:04:08.198289 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:08.209749 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.216670 13869 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:08.217449 13869 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:08.218021 13869 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.217104 14953 rpcz_store.cc:276] 0709 13:04:06.688176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.694347 (+  6171us) service_pool.cc:224] Handling call
0709 13:04:06.920467 (+226120us) ts_tablet_manager.cc:1151] Deleting tablet 9a2c65bca5ce45d5af2ace962fa0a37f
0709 13:04:06.920523 (+    56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:04:08.020564 (+1100041us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:08.194765 (+174201us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:08.196735 (+  1970us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":978708,"tablet-delete.queue_time_us":225146}
I20260709 13:04:08.219223 13757 catalog_manager.cc:5028] TS 3779649036364744823fc035dac892c7 (127.12.109.66:43953): tablet 9a2c65bca5ce45d5af2ace962fa0a37f (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:04:08.226341 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.227012 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.228104 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.227026 13869 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.235343 13869 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:08.236774 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:08.242473 14024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10b028ec30974d06bd9da921c268a641"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:08.245630 14024 raft_consensus.cc:606] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:08.243466 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.244292 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.244313 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.247857 13989 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:08.248217 13945 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:08.248700 14870 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.248765 13945 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:08.249994 13945 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.251447 13945 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.250015 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:08.255712 13949 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:08.262549 13949 raft_consensus.cc:606] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:08.262684 14870 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.258708 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.264021 13989 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:08.264938 14870 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.266343 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:08.267120 14870 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.270434 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.273576 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.279289 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.279644 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.279062 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.282625 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.284615 13869 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:08.289225 13947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "617d39d9397c4724be821e5efacbde17"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:08.289894 13947 raft_consensus.cc:606] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:08.290766 14965 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.294526 13947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f50263f80db844239948e5e67ef3b987"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:08.284988 13989 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:08.296775 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.298689 14963 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.299193 14963 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:08.299612 14963 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.302560 13840 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.304211 13945 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:08.305682 13840 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.306504 14904 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.295337 13947 raft_consensus.cc:606] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:08.307860 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.309810 13989 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:08.310586 14870 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.311065 14961 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.312258 13838 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:08.313437 14961 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.313918 14961 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:08.314553 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:08.316632 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:08.317216 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.308766 14966 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.318650 14966 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:08.304966 14964 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.322357 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:08.323014 14023 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.318605 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.326427 14964 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.326862 14931 raft_consensus.cc:993] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:08.327422 14931 raft_consensus.cc:1081] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:08.328007 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.331691 14964 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.332059 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:08.334482 13839 consensus_peers.cc:597] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 403f6d16105d4532ae1de441c32f632a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:08.335271 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.333330 14966 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.336620 14870 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:08.336823 14023 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:08.337200 14870 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:08.338176 14904 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.339882 13868 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f50263f80db844239948e5e67ef3b987"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:08.340144 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.340488 13868 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:08.340894 13868 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.341547 13947 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:08.342515 14870 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:08.343566 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:08.347102 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.348229 14904 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.349516 14904 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:08.350813 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.351652 14904 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.351652 14966 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.357162 14021 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:08.358397 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:08.361452 14904 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:08.362948 13868 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.314833 14961 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.368439 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:08.375110 14960 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.382436 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:08.383462 13945 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:08.383884 13945 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:08.387557 13868 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:08.394817 13945 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.402580 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:08.403380 14016 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.405884 13945 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.406183 14963 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.411569 14905 raft_consensus.cc:2955] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:08.415313 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:08.416210 14018 raft_consensus.cc:2955] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:08.421819 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:08.431386 13945 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:08.432843 13838 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:08.433873 14966 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.434608 14966 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:08.435592 14966 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.446178 13946 raft_consensus.cc:2955] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:08.447227 14016 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:08.421790 14963 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:08.462661 13989 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:08.463585 14837 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.472231 13771 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:08.484679 13772 catalog_manager.cc:5697] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:08.501379 14936 raft_consensus.cc:993] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:08.501971 14936 raft_consensus.cc:1081] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:08.511751 13868 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:08.512449 13868 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:08.512830 13868 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.520085 13868 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.530121 13868 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:08.530786 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:08.531561 13945 raft_consensus.cc:3055] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:08.531881 13945 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:08.532948 13945 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.534133 13945 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.534484 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:08.534930 13946 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:08.535211 14023 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.544236 13945 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:08.545585 13838 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:08.546653 14018 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:08.562790 14837 raft_consensus.cc:993] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:08.563316 14881 raft_consensus.cc:1081] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:08.566780 13868 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "10b028ec30974d06bd9da921c268a641"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41684
I20260709 13:04:08.567435 14905 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.567618 13868 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:08.568200 13868 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.570080 14023 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:08.568207 14905 raft_consensus.cc:697] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:08.581441 14905 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.585861 14844 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:08.585939 14904 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:08.586495 14844 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:08.586717 14904 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.588575 14844 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:08.589017 14904 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:08.589989 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:08.591666 14966 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:08.592309 14966 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.594874 14966 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:08.595240 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.597218 13868 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.603631 13868 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:08.628328 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 55237.9us
I20260709 13:04:08.649408 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:08.650908 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.653605 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.655465 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:08.656122 14018 raft_consensus.cc:3055] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:08.656044 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:08.656541 14018 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:08.658824 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:08.659385 13949 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.664618 13838 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.673403 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:08.674717 14844 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.675376 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:08.679243 14018 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.681205 14018 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.683318 14848 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:08.684563 14844 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.699944 14018 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
W20260709 13:04:08.710073 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 60481.7us
I20260709 13:04:08.711544 14918 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:08.712154 14918 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:08.713303 14918 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:08.715018 14918 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.720836 13949 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:08.726698 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:08.676913 14904 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:08.760969 14976 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:04:08.758301 14981 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.771814 14981 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:08.772909 14981 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:08.776697 14977 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.067s	user 0.000s	sys 0.003s
W20260709 13:04:08.822746 13914 consensus_peers.cc:597] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 567aed48a57a41e5820f0b937f2de9be. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:08.831539 13868 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:08.854393 13949 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.854492 13946 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.858795 14980 raft_consensus.cc:993] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: : Instructing follower 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:08.859414 14904 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:08.861029 14981 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:08.868739 14918 raft_consensus.cc:1081] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:08.868865 14018 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:08.874166 14559 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:59646
I20260709 13:04:08.874811 14559 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:08.875198 14559 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.878921 14980 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:08.886955 14559 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:08.895140 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:08.895931 13947 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:08.896251 13947 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:08.897187 13947 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:08.898487 13947 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.899272 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:08.910580 13872 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.894152 14559 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:08.939477 13872 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:08.922235 14018 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.934316 14918 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:08.958037 14956 raft_consensus.cc:993] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:08.957798 14524 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:08.946189 14904 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:04:08.962379 14980 raft_consensus.cc:1081] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:08.964017 13947 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:08.998386 13769 catalog_manager.cc:5697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:04:09.003665 14023 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:09.005259 14021 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:09.016561 14991 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:04:09.018903 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 73215.2us
W20260709 13:04:09.021565 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 68538.1us
I20260709 13:04:09.025986 13868 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "617d39d9397c4724be821e5efacbde17"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:09.026657 13868 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:09.027076 13868 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:09.026417 13771 catalog_manager.cc:5697] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:09.028791 13771 catalog_manager.cc:5697] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:09.033727 14981 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:09.035328 14981 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:09.057696 13868 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:09.065249 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:09.066092 13945 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:09.066542 13945 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:09.067540 13945 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:09.068423 13945 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:09.058180 14991 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:09.070667 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:09.071362 14023 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:09.066184 13868 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:09.070978 14991 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:09.079041 13945 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:09.081382 13838 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:09.082520 14961 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:09.084184 14999 ts_tablet_manager.cc:933] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer e1649107c781428692458f10597c5742 (127.12.109.65:43607)
I20260709 13:04:09.084295 14961 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:09.085376 14961 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:09.121932 15005 ts_tablet_manager.cc:933] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:09.128750 14023 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
W20260709 13:04:09.135180 14995 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:04:09.135394 14996 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:04:09.167100 14018 raft_consensus.cc:2955] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:09.151437 14999 tablet_copy_client.cc:323] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.65:43607
I20260709 13:04:09.184095 13885 tablet_copy_service.cc:140] P e1649107c781428692458f10597c5742: Received BeginTabletCopySession request for tablet 403f6d16105d4532ae1de441c32f632a from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:42526)
I20260709 13:04:09.184815 13885 tablet_copy_service.cc:161] P e1649107c781428692458f10597c5742: Beginning new tablet copy session on tablet 403f6d16105d4532ae1de441c32f632a from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:42526: session id = 4dd265d61fc141a48ca9d3d5c8f52921-403f6d16105d4532ae1de441c32f632a
I20260709 13:04:09.200152 14931 raft_consensus.cc:2955] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:09.204903 13947 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:09.204270 13872 raft_consensus.cc:2955] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:09.211826 13769 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 2 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:09.220368 14904 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:09.234869 13769 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:09.269244 13885 tablet_copy_source_session.cc:215] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:09.269738 13771 catalog_manager.cc:5697] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:09.287905 14016 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:09.295161 14961 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:09.303612 13945 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:09.306195 14999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 403f6d16105d4532ae1de441c32f632a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:09.319015 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:09.307256 14961 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:04:09.324301 15005 tablet_copy_client.cc:323] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.66:43953
I20260709 13:04:09.342192 13960 tablet_copy_service.cc:140] P 3779649036364744823fc035dac892c7: Received BeginTabletCopySession request for tablet 567aed48a57a41e5820f0b937f2de9be from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:58096)
I20260709 13:04:09.342921 13960 tablet_copy_service.cc:161] P 3779649036364744823fc035dac892c7: Beginning new tablet copy session on tablet 567aed48a57a41e5820f0b937f2de9be from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:58096: session id = 4dd265d61fc141a48ca9d3d5c8f52921-567aed48a57a41e5820f0b937f2de9be
I20260709 13:04:09.331312 13947 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:09.357497 13960 tablet_copy_source_session.cc:215] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:09.403728 13872 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:09.411099 15005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 567aed48a57a41e5820f0b937f2de9be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:09.414880 14848 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:09.433355 13914 consensus_peers.cc:597] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: efeae9bc4fd84c4487b09362929c2fa6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.455180 14021 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:09.487413 14848 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:09.510566 15012 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:04:09.512784 14999 tablet_copy_client.cc:806] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:09.513479 14999 tablet_copy_client.cc:670] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:09.521255 14999 tablet_copy_client.cc:538] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:09.535089 13946 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:09.540443 14991 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:09.581339 14999 tablet_bootstrap.cc:492] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
W20260709 13:04:09.588477 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 109153us
I20260709 13:04:09.597462 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:09.601934 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 123720us
I20260709 13:04:09.632124 14016 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:09.637583 14964 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:09.660910 13868 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:09.670552 14991 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:04:09.666006 15005 tablet_copy_client.cc:806] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:09.698755 14931 raft_consensus.cc:2955] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:09.733758 15005 tablet_copy_client.cc:670] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:09.813251 15005 tablet_copy_client.cc:538] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:09.886021 13947 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:09.915099 14966 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:09.918644 14848 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.919673 14848 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:09.904302 13944 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:09.926575 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 68388.2us
I20260709 13:04:09.943703 14848 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:09.965014 13868 raft_consensus.cc:2955] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:09.971167 13949 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:09.974292 13949 raft_consensus.cc:606] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:09.977658 14023 raft_consensus.cc:2955] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
W20260709 13:04:10.018322 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 140890us
I20260709 13:04:10.035419 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:10.043021 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:10.049782 14024 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:10.060591 14966 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:10.061684 14966 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:10.069269 13914 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 435cf49d11154354987bf559d46d3c3a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:10.087252 14966 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:10.103940 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:10.104791 13947 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:10.109448 15032 ts_tablet_manager.cc:933] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:10.121929 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:10.127972 14981 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:10.128964 14981 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:10.129499 14981 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.138515 13772 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:10.161988 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:10.163236 14016 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 2: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:10.141834 15005 tablet_bootstrap.cc:492] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:04:10.187887 15032 tablet_copy_client.cc:323] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.66:43953
I20260709 13:04:10.192883 13960 tablet_copy_service.cc:140] P 3779649036364744823fc035dac892c7: Received BeginTabletCopySession request for tablet efeae9bc4fd84c4487b09362929c2fa6 from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:58096)
I20260709 13:04:10.193595 13960 tablet_copy_service.cc:161] P 3779649036364744823fc035dac892c7: Beginning new tablet copy session on tablet efeae9bc4fd84c4487b09362929c2fa6 from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:58096: session id = 4dd265d61fc141a48ca9d3d5c8f52921-efeae9bc4fd84c4487b09362929c2fa6
I20260709 13:04:10.178195 14981 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:10.221961 13872 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:10.224002 14023 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:10.232205 14999 tablet_bootstrap.cc:492] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.233438 14999 tablet_bootstrap.cc:492] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:04:10.235980 14981 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:10.237522 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:10.238763 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:10.239629 14016 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 2: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:10.240471 14984 raft_consensus.cc:993] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: : Instructing follower 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:10.240990 14848 raft_consensus.cc:1081] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:10.242954 14559 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:59646
I20260709 13:04:10.243535 14559 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:10.243896 14559 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.254979 14999 ts_tablet_manager.cc:1403] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.674s	user 0.028s	sys 0.024s
I20260709 13:04:10.258304 13960 tablet_copy_source_session.cc:215] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:10.261142 14559 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:10.259341 14999 raft_consensus.cc:359] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:10.261711 13944 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:10.271417 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:10.272104 13946 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:10.273010 13946 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:10.274416 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:10.276823 14905 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:10.277367 14905 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:10.275678 13946 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:10.277872 15032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efeae9bc4fd84c4487b09362929c2fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.282138 13944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:10.281405 14905 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:10.282833 13944 raft_consensus.cc:606] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:10.282943 13946 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.287096 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:10.283205 13944 raft_consensus.cc:614] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:10.273170 14999 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:10.287954 14559 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:10.288595 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:10.292833 14016 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.306012 14931 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:10.299247 14999 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:10.311425 13946 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:10.322865 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:10.328651 15028 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:10.329270 15028 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:10.330241 15028 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:10.323478 15025 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:04:10.319517 14931 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.342368 15016 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:04:10.344913 14931 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:10.370450 14999 ts_tablet_manager.cc:1434] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.115s	user 0.004s	sys 0.007s
I20260709 13:04:10.376124 14016 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:10.375811 13885 tablet_copy_service.cc:342] P e1649107c781428692458f10597c5742: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-403f6d16105d4532ae1de441c32f632a received from {username='slave'} at 127.0.0.1:42526
I20260709 13:04:10.383805 13885 tablet_copy_service.cc:434] P e1649107c781428692458f10597c5742: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-403f6d16105d4532ae1de441c32f632a on tablet 403f6d16105d4532ae1de441c32f632a with peer 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:04:10.386914 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55356.7us
I20260709 13:04:10.414381 13946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:10.415122 13946 raft_consensus.cc:606] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:10.443243 13771 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 2 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:10.444686 14931 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:10.469126 14931 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:10.484650 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:10.484905 13946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:10.485494 13946 raft_consensus.cc:606] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:10.489464 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:10.490248 14023 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:10.504077 14023 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:10.553169 13871 raft_consensus.cc:2955] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:10.555323 14848 raft_consensus.cc:2955] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:10.556578 14016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:10.557322 14016 raft_consensus.cc:606] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Received request to transfer leadership to TS 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:10.562321 15005 tablet_bootstrap.cc:492] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.567749 13773 catalog_manager.cc:5697] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:10.570835 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:10.577025 14997 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:10.577634 14997 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:10.578639 14997 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:10.580533 13949 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:10.581183 13949 raft_consensus.cc:606] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:10.565693 15032 tablet_copy_client.cc:806] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:10.583874 15005 tablet_bootstrap.cc:492] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:04:10.588847 15032 tablet_copy_client.cc:670] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:10.608918 15032 tablet_copy_client.cc:538] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:10.634974 14999 ts_tablet_manager.cc:933] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:10.643787 15005 ts_tablet_manager.cc:1403] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.503s	user 0.028s	sys 0.023s
I20260709 13:04:10.648440 14931 raft_consensus.cc:993] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:10.649199 14984 raft_consensus.cc:1081] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:10.652410 15032 tablet_bootstrap.cc:492] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:04:10.655296 14931 raft_consensus.cc:993] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:10.655786 14931 raft_consensus.cc:1081] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:10.658032 13872 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:10.658172 13867 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:10.658867 13867 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:10.659274 13867 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.658794 13872 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:10.663926 13872 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.674245 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:10.683357 14023 raft_consensus.cc:2955] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:10.687784 14559 raft_consensus.cc:1217] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:10.695549 13872 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:10.699450 15005 raft_consensus.cc:359] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:10.709087 15005 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:10.703639 13949 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:10.712453 15005 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:10.714811 14904 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:10.715662 14904 consensus_queue.cc:1243] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:04:10.717540 14904 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:10.717867 14904 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:10.721728 14904 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:10.723448 14904 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:10.730971 13949 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:10.732573 15028 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:10.731041 15005 ts_tablet_manager.cc:1434] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.087s	user 0.010s	sys 0.000s
I20260709 13:04:10.736793 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:10.742422 13947 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:10.742786 13947 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:10.743636 13947 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:10.744783 13947 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.739281 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:10.739307 14881 raft_consensus.cc:993] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:10.707086 13867 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:10.749068 13872 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:10.759951 14881 raft_consensus.cc:1081] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Signalling peer 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:10.761602 13960 tablet_copy_service.cc:342] P 3779649036364744823fc035dac892c7: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-567aed48a57a41e5820f0b937f2de9be received from {username='slave'} at 127.0.0.1:58096
I20260709 13:04:10.762295 13960 tablet_copy_service.cc:434] P 3779649036364744823fc035dac892c7: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-567aed48a57a41e5820f0b937f2de9be on tablet 567aed48a57a41e5820f0b937f2de9be with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:10.766940 14019 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:10.769520 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:10.779098 14560 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:59650
I20260709 13:04:10.780006 14560 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:10.780515 14560 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.781993 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:10.782783 13946 raft_consensus.cc:3055] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:10.783149 13946 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:10.784197 13946 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:10.787274 13946 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.799659 13867 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:10.757817 13771 catalog_manager.cc:5697] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 3, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 3 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:10.810343 14999 tablet_copy_client.cc:323] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.66:43953
I20260709 13:04:10.812456 13960 tablet_copy_service.cc:140] P 3779649036364744823fc035dac892c7: Received BeginTabletCopySession request for tablet 435cf49d11154354987bf559d46d3c3a from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:58096)
I20260709 13:04:10.813076 13960 tablet_copy_service.cc:161] P 3779649036364744823fc035dac892c7: Beginning new tablet copy session on tablet 435cf49d11154354987bf559d46d3c3a from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:58096: session id = 4dd265d61fc141a48ca9d3d5c8f52921-435cf49d11154354987bf559d46d3c3a
I20260709 13:04:10.807035 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:10.815673 14021 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.818342 13946 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:10.819347 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:10.829562 14961 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:10.830224 14961 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:10.831246 14961 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:10.808847 13771 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:10.877463 14931 raft_consensus.cc:993] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:10.878172 14931 raft_consensus.cc:1081] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:10.880904 13867 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:10.892555 13867 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:10.894507 13867 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.850904 14560 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:10.903203 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:10.903846 13868 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.910490 13947 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:10.915482 14560 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:10.916225 14021 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:10.921058 13867 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:10.927644 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:10.928349 13946 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:10.929219 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:10.929870 14019 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:10.930210 14019 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:10.930579 13960 tablet_copy_source_session.cc:215] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:10.933648 13838 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:10.935179 14904 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:10.935696 14904 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:10.937091 14023 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:10.936655 14904 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:10.940209 13867 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:10.928705 13946 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:10.956736 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:10.957463 14021 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.931166 14019 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:10.972358 14021 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:10.974493 14019 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.967057 13946 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:10.980276 13868 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:10.990562 13946 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.004613 14558 raft_consensus.cc:1217] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:11.005139 13840 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:11.011897 14999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435cf49d11154354987bf559d46d3c3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:11.014423 14966 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:11.015048 14966 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:11.020071 14019 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:11.025228 13946 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:11.016078 14966 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:11.091071 15054 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:11.145848 14019 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:04:11.161280 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 143598us
I20260709 13:04:11.175988 15016 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:11.182399 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.192287 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:11.193482 15055 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:11.194017 15055 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:11.195091 15055 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:11.223196 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 91216.2us
W20260709 13:04:11.246693 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 67329.3us
I20260709 13:04:11.279448 13867 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:04:11.280112 13867 pending_rounds.cc:85] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 1
I20260709 13:04:11.280480 13867 pending_rounds.cc:107] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:04:11.280844 13867 raft_consensus.cc:2889] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:11.282894 13947 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:11.287087 14904 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:04:11.310406 14984 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:11.363971 13839 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 435cf49d11154354987bf559d46d3c3a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:11.380726 15032 tablet_bootstrap.cc:492] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.393807 15032 tablet_bootstrap.cc:492] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:04:11.410228 14018 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:11.419024 14905 raft_consensus.cc:1064] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:11.423314 13773 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 2 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.428562 15032 ts_tablet_manager.cc:1403] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 0.776s	user 0.043s	sys 0.053s
I20260709 13:04:11.464855 14848 raft_consensus.cc:1064] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:11.421895 14905 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:11.456889 13772 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:11.499413 13772 catalog_manager.cc:5697] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:11.511749 13772 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:11.536482 15032 raft_consensus.cc:359] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:11.537678 15032 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:11.602245 15032 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:11.615710 14848 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:11.623183 14999 tablet_copy_client.cc:806] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:11.624064 14999 tablet_copy_client.cc:670] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
W20260709 13:04:11.638118 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 109091us
W20260709 13:04:11.690680 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 224525us
W20260709 13:04:11.698288 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 57698.1us
I20260709 13:04:11.723384 13871 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:11.697393 14560 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Advancing to term 2
I20260709 13:04:11.742542 14558 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:11.736454 15054 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:04:11.825469 14019 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.804409 15032 ts_tablet_manager.cc:1434] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.371s	user 0.010s	sys 0.003s
W20260709 13:04:11.839265 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 104903us
I20260709 13:04:11.911306 14558 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:04:11.917656 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.920557 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:11.927685 14019 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:11.946996 13949 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:04:11.953028 15065 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.253s	user 0.007s	sys 0.000s
W20260709 13:04:11.953758 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 222810us
I20260709 13:04:11.955749 13945 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:11.899114 14966 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:04:11.965675 14999 tablet_copy_client.cc:538] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:11.966244 13946 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:11.914902 15071 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:04:11.986336 14019 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:04:12.018545 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 89262.6us
I20260709 13:04:12.049275 14560 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEARNER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:12.042289 15078 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 13:04:12.133957 15081 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
W20260709 13:04:12.128073 15076 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 13:04:12.127713 15083 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:04:12.268625 14999 tablet_bootstrap.cc:492] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:04:12.334980 13871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41672
I20260709 13:04:12.335865 13871 raft_consensus.cc:606] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:04:12.370718 13960 tablet_copy_service.cc:342] P 3779649036364744823fc035dac892c7: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-efeae9bc4fd84c4487b09362929c2fa6 received from {username='slave'} at 127.0.0.1:58096
I20260709 13:04:12.372753 13960 tablet_copy_service.cc:434] P 3779649036364744823fc035dac892c7: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-efeae9bc4fd84c4487b09362929c2fa6 on tablet efeae9bc4fd84c4487b09362929c2fa6 with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:12.372413 15084 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:12.400718 13871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41672
I20260709 13:04:12.401991 13871 raft_consensus.cc:606] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Received request to transfer leadership
W20260709 13:04:12.400848 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 424489us
W20260709 13:04:12.400422 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 380905us
W20260709 13:04:12.409615 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 388423us
I20260709 13:04:12.417689 14963 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:12.422957 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 434679us
I20260709 13:04:12.424767 14904 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:12.418388 14963 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:12.431787 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 396919us
I20260709 13:04:12.439253 15081 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:12.442653 13867 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:12.401664 15084 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:12.436993 13759 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for efeae9bc4fd84c4487b09362929c2fa6 already in progress [suppressed 3 similar messages]
W20260709 13:04:12.450034 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.454289 15093 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:04:12.459784 15059 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:04:12.463939 15084 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:12.468798 14961 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:12.469738 14963 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:12.469429 14961 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:12.502431 15039 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:04:12.571097 14021 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:12.579739 14961 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:12.587402 14023 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:12.599156 14024 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:04:12.612092 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 165276us
I20260709 13:04:12.618185 14999 tablet_bootstrap.cc:700] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent opening tablet: real 0.117s	user 0.002s	sys 0.005s
I20260709 13:04:12.624694 15085 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.625437 15085 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:12.580834 13759 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 154620us
I20260709 13:04:12.655411 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:12.665468 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 53046.7us
I20260709 13:04:12.611361 15055 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:04:12.691442 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:12.722033 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 46249.2us
I20260709 13:04:12.702984 15085 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:12.735368 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:12.736385 14024 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:12.756095 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:12.759217 15071 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:12.760504 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:12.759442 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:12.728228 15080 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:12.765559 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:12.771677 14905 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:12.770784 14961 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:04:12.780622 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:12.787681 13989 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:12.793095 13874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41672
I20260709 13:04:12.793891 13874 raft_consensus.cc:606] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:04:12.787590 15080 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:12.830484 15080 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:12.848258 15095 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.848946 15095 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:12.880498 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:12.885279 15087 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:12.873740 14963 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:12.898485 15029 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:12.900390 15017 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:12.901047 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:12.901238 15017 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:12.905092 15017 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:12.885993 15087 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:12.908648 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:12.923906 15078 raft_consensus.cc:993] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:12.924518 15078 raft_consensus.cc:1081] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:12.908994 14963 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:12.928311 15061 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.928894 15061 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:12.928376 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:12.942430 14963 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:12.945138 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:12.945569 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:12.946982 15092 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:12.947593 15092 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:12.957886 15087 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:12.962885 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:12.964679 15075 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:12.965443 15075 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:12.970676 15061 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:12.982594 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:13.007778 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:13.002916 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:13.059924 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:13.072499 14904 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:13.073274 14904 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:13.071331 15092 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:13.083886 14904 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:13.082885 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:13.098287 14016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17624553356541c69c1644ca409df1e9"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:13.099277 14016 raft_consensus.cc:606] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:04:13.055485 15084 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:13.008913 15096 raft_consensus.cc:2955] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:13.179417 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 74113.4us
I20260709 13:04:13.026441 15095 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:13.227758 13839 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:13.211079 14016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
W20260709 13:04:13.237995 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 182030us
I20260709 13:04:13.241775 15081 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:13.285424 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 213313us
I20260709 13:04:13.286815 13838 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:13.252345 14016 raft_consensus.cc:606] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
W20260709 13:04:13.300194 15114 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:04:13.300292 15080 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.296756 15062 raft_consensus.cc:1064] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:13.236521 15084 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:13.340502 15062 raft_consensus.cc:1068] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: Unable to promote non-voter 4dd265d61fc141a48ca9d3d5c8f52921: Service unavailable: leader transfer in progress
W20260709 13:04:13.358350 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 92910.2us
I20260709 13:04:13.370921 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:13.366298 14999 tablet_bootstrap.cc:492] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:04:13.383978 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 92595.3us
I20260709 13:04:13.374441 15124 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:13.402606 15127 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:13.403017 13821 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:04:13.403528 13821 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for efeae9bc4fd84c4487b09362929c2fa6 already in progress
W20260709 13:04:13.403896 13821 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for efeae9bc4fd84c4487b09362929c2fa6 already in progress
I20260709 13:04:13.424373 15016 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:13.425311 15016 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:13.428781 15016 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:13.432932 15084 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:13.436110 14999 tablet_bootstrap.cc:492] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:04:13.437049 14999 ts_tablet_manager.cc:1403] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 1.169s	user 0.039s	sys 0.038s
I20260709 13:04:13.445652 14981 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:13.431348 15124 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:13.472286 15068 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:13.453599 15121 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:13.473776 14556 raft_consensus.cc:2955] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:13.441570 14999 raft_consensus.cc:359] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:13.478791 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46267.2us
I20260709 13:04:13.475627 15068 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:13.492682 14021 raft_consensus.cc:2955] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:13.491196 13838 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:13.481225 14016 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49628
I20260709 13:04:13.476127 15080 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:13.504925 14016 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:13.505513 14016 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.490237 14999 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:13.513161 14999 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:13.516618 14999 ts_tablet_manager.cc:1434] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.079s	user 0.007s	sys 0.002s
I20260709 13:04:13.519767 14837 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:13.520360 14837 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:13.522322 14837 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:13.510195 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:13.477313 14981 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:13.507798 15080 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:13.499749 15078 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.511643 15075 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:13.574954 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:13.576210 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:13.574373 15068 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:13.497929 15121 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:13.631639 14018 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3f6783295084924801a104ff3faf10e"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:13.633934 14018 raft_consensus.cc:606] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Received request to transfer leadership to TS 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:04:13.628973 15111 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 13:04:13.639740 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 259426us
I20260709 13:04:13.647475 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:13.649972 15077 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.627955 15124 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:13.653694 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:13.653214 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:13.657367 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:13.663232 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:13.673121 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:13.690999 15131 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:04:13.693490 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:13.695335 15098 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.696547 15080 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:13.684733 15017 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.691097 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:13.711424 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:13.715663 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:13.703019 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:13.718423 15126 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.734632 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:13.735467 14022 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:13.656373 14016 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:13.721962 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:13.825500 15077 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:13.826193 15077 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:13.829401 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 72679.5us
I20260709 13:04:13.830386 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:13.723137 14981 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:13.843820 14016 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:13.782792 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:13.863317 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:13.871989 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:13.875474 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:13.879027 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:13.881477 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:13.915977 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 64868us
I20260709 13:04:13.864542 15126 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:13.873814 15094 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:13.916517 13950 raft_consensus.cc:2955] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:13.936475 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:13.917374 13960 tablet_copy_service.cc:342] P 3779649036364744823fc035dac892c7: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-435cf49d11154354987bf559d46d3c3a received from {username='slave'} at 127.0.0.1:58096
I20260709 13:04:13.937192 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:13.937454 13960 tablet_copy_service.cc:434] P 3779649036364744823fc035dac892c7: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-435cf49d11154354987bf559d46d3c3a on tablet 435cf49d11154354987bf559d46d3c3a with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:13.938172 15072 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.917481 15080 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:13.946280 13947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:13.922717 15121 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:13.938534 15134 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:13.949589 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 70677.8us
I20260709 13:04:13.948055 13947 raft_consensus.cc:606] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:13.958559 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:13.970019 15141 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:04:13.978598 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:13.981786 15139 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:13.939842 15126 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:13.948319 15094 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:13.990988 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 694) took 1336 ms. Trace:
I20260709 13:04:14.007709 15068 raft_consensus.cc:993] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:14.018543 15068 raft_consensus.cc:1081] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:14.031119 15093 raft_consensus.cc:1064] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
W20260709 13:04:14.031706 15093 raft_consensus.cc:1068] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: Unable to promote non-voter 4dd265d61fc141a48ca9d3d5c8f52921: Service unavailable: leader transfer in progress
I20260709 13:04:13.997624 13944 rpcz_store.cc:276] 0709 13:04:12.654773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.658981 (+  4208us) service_pool.cc:224] Handling call
0709 13:04:12.659369 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:12.715814 (+ 56445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.715860 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:12.718441 (+  2581us) log.cc:844] Serialized 26 byte log entry
0709 13:04:13.447366 (+728925us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b18000fb260. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:04:13.637249 (+189883us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:13.637304 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.638472 (+  1168us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:13.988726 (+350254us) raft_consensus.cc:1685] finished
0709 13:04:13.988761 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.990873 (+  2112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21851,"raft.queue_time_us":58037,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":8322,"spinlock_wait_cycles":5539584,"thread_start_us":189648,"threads_started":1,"wal-append.queue_time_us":1133153}
I20260709 13:04:13.996157 15134 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:14.042274 15135 raft_consensus.cc:993] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:14.042905 15135 raft_consensus.cc:1081] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Signalling peer 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:14.037897 15134 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:14.035455 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:14.059203 15126 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:14.068205 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:14.046793 15094 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:14.075749 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:14.048372 14555 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3f6783295084924801a104ff3faf10e"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:59650
I20260709 13:04:14.079144 13871 raft_consensus.cc:2955] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:14.083855 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:14.084340 13989 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:14.085632 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:14.087158 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:14.090884 15017 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.091275 13866 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:14.091712 13866 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:14.092818 13866 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:14.095639 13866 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.098848 15135 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.099152 15127 raft_consensus.cc:993] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:14.099794 15127 raft_consensus.cc:1081] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:14.065742 15134 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.111274 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:14.112994 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:14.115511 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:14.083315 14555 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:14.120217 14555 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.122421 15095 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.124773 13866 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:14.126905 15097 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.127877 13950 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:14.131924 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.078528 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 45266.7us
I20260709 13:04:14.136432 13947 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17624553356541c69c1644ca409df1e9"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:35894
I20260709 13:04:14.137068 13947 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:14.137419 13947 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.139264 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:14.140614 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:14.143492 13840 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:14.145138 13950 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:14.143415 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:14.157132 15077 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:14.170189 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:14.176019 13947 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:14.176625 15146 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:14.178283 13944 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:14.124608 14554 raft_consensus.cc:2955] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:14.179816 15089 raft_consensus.cc:993] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: : Instructing follower 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:14.183473 15095 raft_consensus.cc:1081] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:14.180740 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:14.185668 14558 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:59646
I20260709 13:04:14.186146 15097 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.186303 14558 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:14.186825 14558 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.196456 13989 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:14.203644 15077 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:14.201184 15127 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:14.201655 14558 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:14.211522 13947 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:14.214243 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:14.214929 13873 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.208909 15127 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:14.217360 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:14.217995 13874 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.218900 15133 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.218474 15127 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:14.219831 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:14.221405 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:14.222499 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:14.231446 14558 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:14.233337 15119 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:14.233867 15146 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:14.237166 15071 raft_consensus.cc:2955] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:14.239975 15119 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:14.241217 13944 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:35894
I20260709 13:04:14.244933 13944 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:14.244946 13874 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:14.245455 13944 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.247996 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:14.244171 15146 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.259755 15144 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:14.260367 15144 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:14.262254 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:14.263827 14904 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.247695 15016 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:14.264694 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:14.247964 15119 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:14.268646 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:14.277509 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:14.278321 14015 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:14.278651 14015 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:14.279553 14015 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:14.274755 15081 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.260784 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:14.282181 13941 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:14.287515 14015 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.282847 15016 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:14.293515 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:14.293828 13944 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:14.299154 13987 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:14.278944 15144 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:14.300725 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:14.306300 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:14.308420 14837 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.310739 13769 catalog_manager.cc:5697] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:14.287842 13941 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:14.316867 13944 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:14.319519 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:14.324826 15075 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.331413 14015 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:14.307889 15127 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.307124 15098 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.334256 13873 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:14.309091 15119 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:14.336447 15119 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.352010 13941 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:14.354496 13941 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.367854 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:14.368942 13873 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.378232 13873 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:14.386757 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:14.389456 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:14.392000 15016 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:14.390118 15075 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:14.392654 15016 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:14.392778 15075 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:14.393195 15016 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:14.393898 15075 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:14.338291 14555 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:14.420053 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:14.420728 14021 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:14.421309 14021 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:14.422942 13873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f50263f80db844239948e5e67ef3b987"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:14.422150 14021 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:14.423700 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:14.424293 14021 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.420648 14022 raft_consensus.cc:2955] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:14.431672 13873 raft_consensus.cc:606] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:04:14.435369 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:14.437870 14558 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:14.442930 14021 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:14.440164 15097 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.452497 13947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:14.453313 13947 raft_consensus.cc:606] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:14.422374 15016 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:14.454846 14555 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:14.466503 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:14.467396 13873 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.434949 15146 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:14.478197 15061 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:14.479241 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:14.479492 15061 raft_consensus.cc:3055] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:14.479920 15061 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:14.483477 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:14.485483 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:14.480940 15061 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:14.492573 13941 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:14.495124 13874 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:14.501168 15045 raft_consensus.cc:993] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: : Instructing follower 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:14.501784 15045 raft_consensus.cc:1081] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:14.511986 14558 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:59646
I20260709 13:04:14.512939 14558 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:14.513479 14558 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.522421 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:14.524953 15146 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:14.524859 15061 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.526873 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:14.531956 13838 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:04:14.535389 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 47404.5us
I20260709 13:04:14.539027 15119 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.555269 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:14.561100 13873 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:14.562312 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:14.566166 15144 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:14.566768 15144 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:14.567799 15144 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:14.575925 14558 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:14.578660 15146 raft_consensus.cc:993] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:14.579401 15146 raft_consensus.cc:1081] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:14.581444 13947 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f50263f80db844239948e5e67ef3b987"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:35884
I20260709 13:04:14.582011 13947 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:14.582399 13947 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.589052 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:14.589709 13870 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.592087 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:14.592839 13941 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:14.593166 13941 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:14.594031 13941 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:14.602465 14558 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:14.604664 13941 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.610038 13870 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:14.612497 14524 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:14.614260 15147 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:14.614766 15147 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:14.615653 15147 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:14.619490 14022 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
W20260709 13:04:14.626634 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 138239us
I20260709 13:04:14.631712 14558 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEARNER]: Advancing to term 3
I20260709 13:04:14.636475 13947 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:14.641920 13947 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:14.647365 15075 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:14.648608 15075 raft_consensus.cc:3055] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:14.648981 15075 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:14.649844 15075 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:14.660251 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:14.661012 13873 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:14.661314 13873 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:14.662145 13873 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:14.663504 13873 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.664274 14558 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:14.666349 15075 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.670794 15061 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:14.671840 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:14.672479 14015 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.680778 15122 raft_consensus.cc:1064] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
W20260709 13:04:14.681375 15122 raft_consensus.cc:1068] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Unable to promote non-voter 4dd265d61fc141a48ca9d3d5c8f52921: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:04:14.682075 14015 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:14.682710 13873 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:14.687433 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:14.688640 15072 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:14.689217 15072 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:14.690286 15072 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:14.654742 13941 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:14.719758 13772 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 3, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.737370 13870 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:14.751808 14558 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:14.752628 13941 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:14.752801 13771 catalog_manager.cc:5697] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:14.754659 15145 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:14.755858 15060 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:14.764575 13941 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:14.766892 15094 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:14.780481 13870 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:14.782226 15094 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:14.783783 15060 raft_consensus.cc:2955] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:14.786969 14555 raft_consensus.cc:2955] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:14.784819 13769 catalog_manager.cc:5697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 2 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:14.789206 13941 raft_consensus.cc:2955] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:14.789698 13769 catalog_manager.cc:5697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 2 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:14.800894 13757 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 567aed48a57a41e5820f0b937f2de9be with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 3779649036364744823fc035dac892c7 (127.12.109.66:43953) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3779649036364744823fc035dac892c7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:14.793380 13769 catalog_manager.cc:5697] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 2 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:14.821081 13759 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 403f6d16105d4532ae1de441c32f632a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:14.869248 13947 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:14.870772 15147 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:14.905818 13771 catalog_manager.cc:5697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.907898 13771 catalog_manager.cc:5697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 3, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 3 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.909505 13771 catalog_manager.cc:5697] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.912824 13769 catalog_manager.cc:5697] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 3 to 4, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New cstate: current_term: 1 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:14.926515 13759 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 403f6d16105d4532ae1de441c32f632a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader e1649107c781428692458f10597c5742 (127.12.109.65:43607) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 13:04:14.951678 14023 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:14.954164 13873 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:14.957060 14023 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:14.957558 15072 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:14.961864 15072 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:14.975772 15094 raft_consensus.cc:1064] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:14.987622 15147 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:04:15.034221 13873 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:15.091959 15170 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:04:15.073731 15077 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:04:15.106760 13873 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:15.081480 15094 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:15.110558 15072 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:15.114015 13868 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:15.120630 13867 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:15.129911 14555 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:15.132629 15122 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:15.136340 13950 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:04:15.145958 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 92887us
I20260709 13:04:15.153282 15098 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:04:15.153064 15144 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
W20260709 13:04:15.294191 15168 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:04:15.305799 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 166306us
W20260709 13:04:15.404696 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 91008.4us
I20260709 13:04:15.435694 15179 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:15.453013 13872 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:15.455500 14023 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:04:15.494732 15174 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:04:15.500662 15183 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:15.505316 15061 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:04:15.490451 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:15.626232 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 398556us
W20260709 13:04:15.636770 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 78089.9us
I20260709 13:04:15.653995 14022 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:04:15.669052 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 48140us
I20260709 13:04:15.627380 15144 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:15.713063 15189 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:04:15.716125 14012 tablet_service.cc:1558] Processing DeleteTablet for tablet 403f6d16105d4532ae1de441c32f632a with delete_type TABLET_DATA_TOMBSTONED (TS d1b4fd04200247d99ba9ad1fbeb42b42 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49624
I20260709 13:04:15.719081 15187 raft_consensus.cc:2955] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:15.724442 13944 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:15.731782 15180 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:15.710758 15195 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:04:15.693764 15144 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:15.749987 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:15.751209 13871 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 2: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:15.773620 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:15.774698 13947 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:15.752935 15144 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:15.841941 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 104314us
W20260709 13:04:15.842872 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 139543us
I20260709 13:04:15.833355 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:15.894208 15203 tablet_replica.cc:333] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:04:15.874866 15198 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:15.906455 15198 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:15.906876 13870 raft_consensus.cc:2955] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:15.932420 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48648.3us
I20260709 13:04:15.913686 15198 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:15.962679 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:15.966292 15144 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.970077 13945 raft_consensus.cc:2955] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:15.981238 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 97853.6us
I20260709 13:04:15.982784 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:15.983599 13874 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:15.984668 13989 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:15.978955 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:15.996121 15126 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:15.996830 15126 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:15.996219 13941 raft_consensus.cc:2393] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 3: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:16.002202 15188 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:16.002681 15188 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:16.003082 15188 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.003683 14555 raft_consensus.cc:2955] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:16.006714 15126 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:16.020750 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:16.021535 13946 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:16.024392 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:16.025388 13866 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 2: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:04:16.030458 13757 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 567aed48a57a41e5820f0b937f2de9be with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 3779649036364744823fc035dac892c7 (127.12.109.66:43953) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3779649036364744823fc035dac892c7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:16.046783 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:16.080277 15203 raft_consensus.cc:2243] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:16.081445 15203 raft_consensus.cc:2272] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:16.089973 15199 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.161s	user 0.003s	sys 0.000s
I20260709 13:04:16.104993 15188 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:16.109257 15203 ts_tablet_manager.cc:1916] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:16.137576 13870 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:16.162484 15016 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:16.172436 15188 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:16.177418 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:16.178385 13945 raft_consensus.cc:2393] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 3: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:16.212587 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:04:16.306571 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 49893us
I20260709 13:04:16.313623 15212 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.323668 15212 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:16.377791 13769 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index 2 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:16.393966 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:16.395691 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:16.417053 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:16.417837 14015 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:16.422219 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:16.423623 13950 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:16.435864 13838 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:16.419658 15144 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:16.391717 15212 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:16.453841 14558 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:16.443961 15119 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:16.474671 13866 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:16.458470 15119 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.493193 15119 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.521574 15203 ts_tablet_manager.cc:1929] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:16.522400 15203 log.cc:1199] T 403f6d16105d4532ae1de441c32f632a P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root/wals/403f6d16105d4532ae1de441c32f632a
I20260709 13:04:16.528389 13759 catalog_manager.cc:5028] TS d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): tablet 403f6d16105d4532ae1de441c32f632a (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:04:16.528349 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:16.533803 15188 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:16.534390 15188 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:16.534781 15188 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:16.558811 15119 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:16.584856 15119 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:16.587970 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:16.588964 13946 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:16.593170 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 141018us
I20260709 13:04:16.595029 13989 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:16.596481 14837 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:16.596989 14837 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:16.598042 14837 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:16.606977 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:16.610458 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:16.621544 14554 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.618855 15188 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:16.634877 14015 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.649196 13946 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:16.681389 15217 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.657270 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:16.678480 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:16.682158 15217 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:16.683274 13950 raft_consensus.cc:2368] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3.
I20260709 13:04:16.684473 13870 raft_consensus.cc:2368] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3.
I20260709 13:04:16.685945 13989 leader_election.cc:400] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3.
I20260709 13:04:16.686327 13989 leader_election.cc:403] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:04:16.684850 15217 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:16.651479 14015 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:16.662917 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:16.694164 14837 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3.
I20260709 13:04:16.696517 14554 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:16.699244 13840 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:16.700947 14964 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:16.701542 14964 raft_consensus.cc:697] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:16.703105 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:16.696760 15188 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:16.709314 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:16.710028 13872 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:16.712620 13987 leader_election.cc:400] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3.
I20260709 13:04:16.715090 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:16.707108 14964 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:16.719425 15217 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:16.719946 15217 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.720371 15217 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.754344 15217 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:16.763785 15217 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:16.765059 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:16.769754 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:16.774937 13872 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.778280 14015 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:16.784148 13872 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:16.795296 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:16.800901 15218 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:16.806219 13946 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:16.806329 15218 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:16.808314 15218 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:16.809921 14558 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:16.813063 14837 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:16.816745 15188 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:16.827956 15209 raft_consensus.cc:2955] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:16.834913 14558 raft_consensus.cc:2955] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:16.841236 13946 raft_consensus.cc:2955] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:16.856922 13769 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 3, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:16.899304 13872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:16.900058 13872 raft_consensus.cc:606] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 2 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:16.905344 13950 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:16.905992 13950 raft_consensus.cc:606] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:16.913954 13950 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:16.916854 13950 raft_consensus.cc:606] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:16.920068 14024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "765348f2471a493d92e998132c0c746b"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:16.920835 14024 raft_consensus.cc:606] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:04:16.925968 14024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:16.926774 14024 raft_consensus.cc:606] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:04:16.928196 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:16.930311 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:16.930337 13771 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 3 to 4, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 4 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:16.913672 13772 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 5 to 6, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:16.954595 13759 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet efeae9bc4fd84c4487b09362929c2fa6 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:16.957247 14024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49624
I20260709 13:04:16.958076 14024 raft_consensus.cc:606] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Received request to transfer leadership
I20260709 13:04:16.964473 13860 tablet_service.cc:1558] Processing DeleteTablet for tablet efeae9bc4fd84c4487b09362929c2fa6 with delete_type TABLET_DATA_TOMBSTONED (TS e1649107c781428692458f10597c5742 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41672
I20260709 13:04:17.007647 13771 catalog_manager.cc:5697] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 3, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:17.046161 15228 tablet_replica.cc:333] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:04:17.047448 15228 raft_consensus.cc:2243] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:17.048432 15228 raft_consensus.cc:2272] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:17.048612 15218 raft_consensus.cc:993] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:17.049203 15202 raft_consensus.cc:1081] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:17.051271 13866 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:17.051941 13866 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:17.052307 13866 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.056542 15228 ts_tablet_manager.cc:1916] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:17.066082 13872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41672
I20260709 13:04:17.066793 13872 raft_consensus.cc:606] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 LEADER]: Received request to transfer leadership
I20260709 13:04:17.084048 13872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41672
I20260709 13:04:17.084703 13872 raft_consensus.cc:606] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 LEADER]: Received request to transfer leadership
I20260709 13:04:17.108791 13872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41672
I20260709 13:04:17.109442 13872 raft_consensus.cc:606] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 LEADER]: Received request to transfer leadership
W20260709 13:04:17.115736 13821 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:04:17.116175 13821 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 567aed48a57a41e5820f0b937f2de9be already in progress
I20260709 13:04:17.116272 14558 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEARNER]: Advancing to term 3
W20260709 13:04:17.116379 13821 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 567aed48a57a41e5820f0b937f2de9be already in progress
I20260709 13:04:17.110880 13866 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.126530 15218 raft_consensus.cc:993] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:17.127483 15016 raft_consensus.cc:1081] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:17.129292 13870 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:17.129899 13870 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:17.130316 13870 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.139791 15228 ts_tablet_manager.cc:1929] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:04:17.142575 15228 log.cc:1199] T efeae9bc4fd84c4487b09362929c2fa6 P e1649107c781428692458f10597c5742: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/wals/efeae9bc4fd84c4487b09362929c2fa6
I20260709 13:04:17.145680 13759 catalog_manager.cc:5028] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): tablet efeae9bc4fd84c4487b09362929c2fa6 (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:04:17.139582 13870 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.148241 13870 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:17.149458 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:17.150231 14015 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.154798 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:17.155555 13950 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:17.155860 13950 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:17.156766 13950 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.159574 13866 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:17.149739 14558 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:17.160449 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:17.160764 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:17.161073 13946 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:17.161413 14024 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.161446 13946 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:17.157986 13950 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.205843 13870 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:17.221203 14021 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:17.225543 14024 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:17.163364 15188 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:17.208734 13946 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.230826 13941 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:17.249832 15122 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:17.250496 13946 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.251758 15122 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:17.231563 13950 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:17.252193 15122 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:17.253302 13838 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:17.253088 15122 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:17.254222 15119 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:17.208230 14015 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:17.239823 14558 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:17.254719 15119 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:17.245505 14964 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:17.255650 15119 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.261713 13946 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:17.262601 15059 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:17.266770 13838 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:17.260613 15122 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:17.278509 15184 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:17.279808 15212 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:17.281116 15212 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:17.282331 15212 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.315907 15059 raft_consensus.cc:993] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742: : Instructing follower 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:17.344842 15059 raft_consensus.cc:1081] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 LEADER]: Signalling peer 4dd265d61fc141a48ca9d3d5c8f52921 to start an election
I20260709 13:04:17.358839 15212 raft_consensus.cc:993] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:17.359411 15212 raft_consensus.cc:1081] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 2 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:17.362478 14555 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
 from {username='slave'} at 127.0.0.1:59630
I20260709 13:04:17.363336 14555 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:17.363827 14555 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:17.364401 15236 raft_consensus.cc:993] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:17.366830 15208 raft_consensus.cc:1081] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:17.375819 14022 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49628
I20260709 13:04:17.376576 14022 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:17.376994 14022 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:17.379261 13950 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:35894
I20260709 13:04:17.379969 13950 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:17.380499 13950 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.387821 15236 raft_consensus.cc:993] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:17.388499 15236 raft_consensus.cc:1081] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:17.401649 13941 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "765348f2471a493d92e998132c0c746b"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:35894
I20260709 13:04:17.402530 13941 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:17.402994 13941 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.407605 14554 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEARNER]: Advancing to term 4
I20260709 13:04:17.425202 14554 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEARNER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:04:17.425697 14554 pending_rounds.cc:85] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting all ops after (but not including) 3
I20260709 13:04:17.438164 13941 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.453369 14022 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.448889 13771 catalog_manager.cc:5697] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:17.456985 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:17.457599 13872 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:17.457931 13872 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:17.462394 14021 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:04:17.463064 14021 pending_rounds.cc:85] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 3
I20260709 13:04:17.463339 14021 pending_rounds.cc:107] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:04:17.463577 14021 raft_consensus.cc:2889] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:17.464213 14555 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:17.466657 13872 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.468122 13872 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:17.470139 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:17.470803 13866 raft_consensus.cc:3055] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:17.471120 13866 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:17.470286 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:17.471853 13944 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:17.473855 13866 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:17.474399 13950 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.475054 13866 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:17.475160 13870 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:17.486789 13950 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:17.487067 14022 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:17.492332 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:17.493125 15202 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:17.493094 13950 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:17.495710 15016 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:17.494902 13866 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:17.499684 13941 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:17.425997 14554 pending_rounds.cc:107] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:04:17.504220 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:17.504940 14554 raft_consensus.cc:2889] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:17.505059 14021 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:17.505556 14021 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:17.506460 14021 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.506789 13944 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:17.509095 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:17.509847 14022 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:17.510272 14022 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:17.511405 14522 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:17.512485 15178 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:17.513041 15178 raft_consensus.cc:697] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:17.511338 14021 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.518321 13950 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:17.518484 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:17.519347 13870 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.521103 13872 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:17.523687 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:17.524314 13868 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.514104 15178 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:17.523116 13989 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:17.533429 15198 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:17.529775 13771 catalog_manager.cc:5697] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:17.534106 15198 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:17.535216 15198 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.539575 13868 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:17.549767 14555 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:17.551530 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:17.546376 14022 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.562795 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:17.567195 13870 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:17.568969 13771 catalog_manager.cc:5697] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 3 to 4, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 4 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:17.621399 14022 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.571318 15138 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:17.643303 14022 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
W20260709 13:04:17.644733 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.571527 15016 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:04:17.655818 14021 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:17.648608 15138 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:17.695513 15138 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.697815 13941 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:17.703238 15212 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:17.719496 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 151847us
I20260709 13:04:17.721272 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:17.748453 15242 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:17.773627 13771 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 3, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:17.798115 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 67902.6us
I20260709 13:04:17.782631 15242 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:17.816237 15242 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:17.825608 14015 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:17.866633 15222 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:17.879124 14015 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:17.888809 13944 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:17.901226 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 567aed48a57a41e5820f0b937f2de9be with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:04:17.918465 15222 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:17.921034 15222 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:17.921801 15244 raft_consensus.cc:1064] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:17.928349 15244 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:17.931355 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:17.964366 14016 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:17.973799 14016 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:18.029553 14555 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEARNER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:18.013417 15249 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:04:18.046869 15138 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:18.052922 13941 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:18.055835 15110 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:18.052857 13771 catalog_manager.cc:5697] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:18.069715 13771 catalog_manager.cc:5697] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:18.119284 13872 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:18.120311 14016 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:18.121132 13941 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:18.130880 13870 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:18.132616 15249 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:18.136948 15209 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:18.140291 13869 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:18.151006 15127 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:04:18.274801 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 77745.9us
W20260709 13:04:18.275606 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 103426us
W20260709 13:04:18.279253 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 96946.9us
I20260709 13:04:18.280802 15270 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:04:18.276661 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 77798.6us
I20260709 13:04:18.292666 14021 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:18.321915 15218 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:18.357048 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:18.377092 13870 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:18.398217 15270 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:18.403059 15218 raft_consensus.cc:2955] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:18.466279 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 63986.5us
I20260709 13:04:18.484050 14015 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:18.492214 14021 raft_consensus.cc:2955] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:18.501446 13870 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:18.519516 15246 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:18.529112 15244 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:18.540357 14554 raft_consensus.cc:2955] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:18.558319 13769 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index 2 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:18.607414 13946 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:18.632499 14554 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:18.634591 15270 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:18.649431 15218 raft_consensus.cc:2955] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:18.649631 13873 raft_consensus.cc:2955] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:18.652467 15244 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:18.651825 14555 raft_consensus.cc:2955] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:18.678725 13873 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:18.683389 13946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:18.684108 13946 raft_consensus.cc:606] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:18.699254 13772 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 5 to 6, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New cstate: current_term: 4 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:18.749891 15273 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:18.752588 14012 tablet_service.cc:1558] Processing DeleteTablet for tablet 435cf49d11154354987bf559d46d3c3a with delete_type TABLET_DATA_TOMBSTONED (TS d1b4fd04200247d99ba9ad1fbeb42b42 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49624
I20260709 13:04:18.753935 15270 raft_consensus.cc:993] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
W20260709 13:04:18.755048 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 43712.2us
I20260709 13:04:18.778949 14024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:18.779698 14024 raft_consensus.cc:606] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:04:18.784615 13872 raft_consensus.cc:2955] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:18.771203 15270 raft_consensus.cc:1081] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:18.807175 13873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:18.808004 13873 raft_consensus.cc:606] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 2 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:18.812485 13757 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 435cf49d11154354987bf559d46d3c3a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:18.829061 14024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:18.830040 14024 raft_consensus.cc:606] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:04:18.852056 15282 tablet_replica.cc:333] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
W20260709 13:04:18.880043 15280 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.085s	user 0.000s	sys 0.003s
I20260709 13:04:18.874694 15282 raft_consensus.cc:2243] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:18.884269 15282 raft_consensus.cc:2272] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:18.896281 15282 ts_tablet_manager.cc:1916] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:04:18.911816 15279 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.099s	user 0.000s	sys 0.003s
W20260709 13:04:18.918469 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 142340us
I20260709 13:04:18.932358 13870 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:18.933063 13870 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:18.933458 13870 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:18.945235 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 152297us
I20260709 13:04:18.958662 13870 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:18.961946 13870 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:18.964115 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:18.964749 13946 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:18.965082 13946 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:18.965843 13946 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:18.967016 13946 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:18.969682 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:18.970427 14555 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:18.972450 14837 raft_consensus.cc:993] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:18.974144 13950 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:18.974824 13950 raft_consensus.cc:606] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:18.979931 13946 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:18.981925 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:18.983145 15268 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:18.983723 15268 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:18.984702 15268 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:18.990674 14837 raft_consensus.cc:1081] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:18.993222 13950 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:35894
I20260709 13:04:18.993809 13950 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:18.997043 13950 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.002187 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:19.004421 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:04:19.032968 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 51457.4us
I20260709 13:04:19.042423 14555 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:19.043994 15282 ts_tablet_manager.cc:1929] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260709 13:04:19.045044 15282 log.cc:1199] T 435cf49d11154354987bf559d46d3c3a P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root/wals/435cf49d11154354987bf559d46d3c3a
I20260709 13:04:19.048082 13759 catalog_manager.cc:5028] TS d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): tablet 435cf49d11154354987bf559d46d3c3a (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:04:19.087196 13950 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:19.098379 13950 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:19.098848 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:19.099552 14561 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.090039 13773 catalog_manager.cc:5697] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:19.104548 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:19.106310 14024 raft_consensus.cc:3055] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:19.106669 14024 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:19.107453 14024 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:19.108358 14561 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:19.109746 13914 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:19.110706 15271 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:19.111250 15271 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:19.112246 15271 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:19.125860 15283 raft_consensus.cc:993] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:19.126513 15283 raft_consensus.cc:1081] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:19.128319 13872 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:19.128876 13872 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:19.129230 13872 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.108829 14024 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.139782 13872 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:19.145386 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:19.147177 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:19.155622 14016 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.154438 13950 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:19.161002 13950 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:19.163744 13772 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:19.178794 13950 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:19.181121 13950 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.146960 13872 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:19.188167 14016 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:19.206355 15059 raft_consensus.cc:993] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:19.206939 15059 raft_consensus.cc:1081] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 2 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:19.208280 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:19.208950 14024 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:19.209782 15289 raft_consensus.cc:993] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:19.212568 15289 raft_consensus.cc:1081] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:19.210194 15268 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:19.213991 13950 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:19.214485 15268 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:19.214666 14016 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49628
I20260709 13:04:19.214726 13944 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:35894
I20260709 13:04:19.215386 14016 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:19.215791 13944 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:19.215802 14016 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.215561 15268 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:19.216337 13944 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.222798 14016 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:19.223970 13944 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:19.224973 14016 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:19.225915 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:19.226562 13872 raft_consensus.cc:3055] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:19.226925 13872 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:19.228106 13872 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:19.229204 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:19.228861 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:19.230064 13872 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.230187 14561 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.228691 13944 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:19.230154 13950 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.243513 14561 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:19.244807 13914 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:19.246135 15271 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:19.246806 15271 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:19.247881 15271 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:19.257994 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:19.258836 14016 raft_consensus.cc:3055] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:19.259171 14016 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:19.260020 14016 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:19.268873 13872 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:19.302807 14016 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.287065 13772 catalog_manager.cc:5697] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:19.260798 13950 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:19.304340 13989 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:19.305269 15289 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:19.305753 15289 raft_consensus.cc:697] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:19.306676 15289 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:19.265357 13773 catalog_manager.cc:5697] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:19.322336 14016 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:19.376657 13950 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:19.379745 15263 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:19.373936 13772 catalog_manager.cc:5697] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 3, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:19.405215 14561 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:19.406872 15059 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:19.513584 14561 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:19.518939 15016 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:19.549932 14016 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:19.552258 15271 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:19.721169 14561 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:19.725025 15016 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:19.731266 13950 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:19.734858 15289 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:19.778810 14024 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:19.788328 15016 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:19.834264 13950 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:19.840332 13872 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:19.853295 15286 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:19.858790 15059 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:19.885887 14024 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:19.893035 15263 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:04:19.996157 14561 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:20.008545 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:20.024545 13950 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:20.026371 15110 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:20.038656 14021 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:20.133370 15110 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:20.178128 15110 raft_consensus.cc:2955] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 3 to 6, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:20.206825 13950 raft_consensus.cc:2955] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 3 to 6, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:20.232165 14016 raft_consensus.cc:2955] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 3 to 6, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:20.283393 13758 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 567aed48a57a41e5820f0b937f2de9be with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
W20260709 13:04:20.446974 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 44771.6us
W20260709 13:04:20.565994 13840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.044s	sys 0.082s
W20260709 13:04:20.566524 13840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.044s	sys 0.084s
I20260709 13:04:20.568665 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:20.612605 13769 catalog_manager.cc:5697] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 3 to 6, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New cstate: current_term: 4 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:20.647024 14021 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:20.832247 13988 consensus_peers.cc:597] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 02dcf286c6bd48e3a6e548def7d786cc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:20.869027 13948 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:04:20.926862 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 77256.6us
I20260709 13:04:20.946105 15237 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
W20260709 13:04:20.958163 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 122331us
W20260709 13:04:20.966665 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 109463us
W20260709 13:04:20.973917 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 73546.8us
I20260709 13:04:20.983261 13872 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:20.988070 15098 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:21.034942 14024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:21.035925 14024 raft_consensus.cc:606] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:04:21.052205 13872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:21.049299 15310 raft_consensus.cc:2955] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:21.066632 13860 tablet_service.cc:1558] Processing DeleteTablet for tablet 567aed48a57a41e5820f0b937f2de9be with delete_type TABLET_DATA_TOMBSTONED (TS e1649107c781428692458f10597c5742 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41672
I20260709 13:04:21.060742 13872 raft_consensus.cc:606] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 1 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:04:21.093652 13871 raft_consensus.cc:2955] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:21.102888 13870 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "617d39d9397c4724be821e5efacbde17"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:21.103655 13870 raft_consensus.cc:606] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:21.103888 15059 raft_consensus.cc:993] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:21.104820 13948 raft_consensus.cc:2955] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:21.107003 13944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:21.107636 13944 raft_consensus.cc:606] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:21.109364 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:04:21.104660 15331 raft_consensus.cc:1081] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 1 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:21.124678 13944 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:35884
I20260709 13:04:21.125241 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:21.125640 13944 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:21.126241 13944 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.158810 15016 raft_consensus.cc:993] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:21.159541 15016 raft_consensus.cc:1081] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:21.163475 13870 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:21.164136 13870 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:21.164508 13870 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.174501 13870 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:21.178515 13944 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:21.185539 15337 tablet_replica.cc:333] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:04:21.187842 15337 raft_consensus.cc:2243] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:21.189042 15337 raft_consensus.cc:2272] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:21.194171 15337 ts_tablet_manager.cc:1916] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:21.209645 13870 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:21.209998 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:21.215135 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:21.215736 13950 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:21.215837 14561 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.216099 13950 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:21.211752 13944 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:21.217154 13950 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:21.219415 13950 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.208164 13772 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index -1 to 4, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:21.215252 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:21.227759 13871 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:21.230134 13871 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:21.232455 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:21.233217 14020 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.246126 13871 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:21.257179 13950 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:21.257798 14020 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:21.258734 13838 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:21.260262 15336 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:21.262480 15336 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:21.263608 15336 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:21.268177 14561 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:21.269623 13871 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.270227 15337 ts_tablet_manager.cc:1929] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260709 13:04:21.270927 15337 log.cc:1199] T 567aed48a57a41e5820f0b937f2de9be P e1649107c781428692458f10597c5742: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/wals/567aed48a57a41e5820f0b937f2de9be
I20260709 13:04:21.273576 13759 catalog_manager.cc:5028] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): tablet 567aed48a57a41e5820f0b937f2de9be (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:04:21.283114 15310 raft_consensus.cc:993] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:21.289182 13914 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:21.289551 14020 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:21.290231 15016 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:21.311792 15016 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:21.312776 15016 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:21.290759 15328 raft_consensus.cc:1081] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:21.300539 13772 catalog_manager.cc:5697] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 3, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.318852 13871 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:21.324914 13950 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:35894
I20260709 13:04:21.325624 13950 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:21.326028 13950 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.342257 13950 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:21.346975 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:21.346346 13950 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:21.347673 13871 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.352474 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:21.353277 14023 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:21.353631 14023 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:21.355278 14023 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:21.356938 14023 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.363986 13871 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:21.368072 13772 catalog_manager.cc:5697] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 2, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:21.371984 13804 debug-util.cc:398] Leaking SignalData structure 0x7b0800034700 after lost signal to thread 12728
I20260709 13:04:21.371256 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:21.373937 13804 debug-util.cc:398] Leaking SignalData structure 0x7b080003cda0 after lost signal to thread 13807
W20260709 13:04:21.376818 13804 debug-util.cc:398] Leaking SignalData structure 0x7b0800345480 after lost signal to thread 13899
I20260709 13:04:21.376762 15271 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:21.377516 15271 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:21.378652 15271 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:21.383831 13804 debug-util.cc:398] Leaking SignalData structure 0x7b080014b000 after lost signal to thread 13974
W20260709 13:04:21.387612 13804 debug-util.cc:398] Leaking SignalData structure 0x7b080009afa0 after lost signal to thread 14048
W20260709 13:04:21.389277 13804 debug-util.cc:398] Leaking SignalData structure 0x7b080033e9a0 after lost signal to thread 14585
I20260709 13:04:21.405510 13772 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 3 to 4, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 4 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:21.459448 13988 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 545fc4aeb71d491985e296a0721d905d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:21.463897 13871 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:21.467739 15310 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:04:21.468255 13944 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:21.474676 14023 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:21.478718 15328 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:21.499924 15237 raft_consensus.cc:2955] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:21.503172 13871 raft_consensus.cc:2955] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:21.523779 13772 catalog_manager.cc:5697] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 reported cstate change: config changed from index -1 to 4, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:21.535529 15336 raft_consensus.cc:993] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:21.536134 15336 raft_consensus.cc:1081] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:21.639995 14020 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "617d39d9397c4724be821e5efacbde17"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49628
I20260709 13:04:21.640772 14020 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:21.641204 14020 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.641223 13950 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:21.649271 13944 raft_consensus.cc:2955] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:21.696520 14020 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:21.704131 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:21.704873 13871 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:21.704362 14020 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:21.705204 13871 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:21.706972 13871 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:21.708397 13871 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.705446 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:21.714529 13944 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.722307 14561 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:21.724377 15271 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:21.737237 13871 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:21.745064 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:21.746255 15343 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:21.747067 15343 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:21.747977 15343 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:21.752888 13914 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 14bba4e2cef04a418c6f660869c7e6e5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:21.754622 13944 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:21.754998 14020 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.759007 15016 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:21.770219 13870 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.772445 15320 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:21.787297 15016 raft_consensus.cc:2955] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:21.784734 13771 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 3, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:21.792455 14020 raft_consensus.cc:2955] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:21.787233 14023 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.810431 15312 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:21.840492 14020 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:21.847862 13771 catalog_manager.cc:5697] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 reported cstate change: config changed from index -1 to 3, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 3 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.862948 13950 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.865370 15336 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:21.903122 13870 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:21.908226 15016 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:21.950239 14016 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:21.982348 13944 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:21.986956 15237 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:21.987233 13914 consensus_peers.cc:597] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 02dcf286c6bd48e3a6e548def7d786cc. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:21.990617 13988 consensus_peers.cc:597] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6b098cc2d51e471cb2c9c1b17eebcb57. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:21.992631 13871 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:22.014235 15237 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
W20260709 13:04:22.115661 15356 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:22.073523 15271 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
W20260709 13:04:22.130298 15355 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:04:22.132176 15310 raft_consensus.cc:2955] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:22.145257 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:22.167204 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 96955.4us
I20260709 13:04:22.192054 15372 ts_tablet_manager.cc:933] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:22.199378 13871 raft_consensus.cc:2955] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:22.238952 13871 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:22.242424 13871 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:22.244160 13871 raft_consensus.cc:2955] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:22.247859 15343 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:22.294792 13950 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:35880
I20260709 13:04:22.295460 13950 raft_consensus.cc:606] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Received request to transfer leadership
I20260709 13:04:22.298702 13944 raft_consensus.cc:2955] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
I20260709 13:04:22.295552 13771 catalog_manager.cc:5697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index -1 to 2, NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) added. New cstate: current_term: 1 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.433553 13946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:35880
I20260709 13:04:22.428467 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:22.449672 13946 raft_consensus.cc:606] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Received request to transfer leadership
I20260709 13:04:22.427275 15372 tablet_copy_client.cc:323] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.67:46599
W20260709 13:04:22.488871 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 182100us
W20260709 13:04:22.562327 13757 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 02dcf286c6bd48e3a6e548def7d786cc already in progress [suppressed 1 similar messages]
W20260709 13:04:22.564042 15378 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 13:04:22.582073 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 122114us
W20260709 13:04:22.585973 15369 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:04:22.627871 13757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 147812us
W20260709 13:04:22.626921 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 96272.2us
I20260709 13:04:22.647322 15383 ts_tablet_manager.cc:933] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:22.650790 15391 ts_tablet_manager.cc:933] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:22.663342 14033 tablet_copy_service.cc:140] P d1b4fd04200247d99ba9ad1fbeb42b42: Received BeginTabletCopySession request for tablet 545fc4aeb71d491985e296a0721d905d from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:55076)
I20260709 13:04:22.664016 14033 tablet_copy_service.cc:161] P d1b4fd04200247d99ba9ad1fbeb42b42: Beginning new tablet copy session on tablet 545fc4aeb71d491985e296a0721d905d from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:55076: session id = 4dd265d61fc141a48ca9d3d5c8f52921-545fc4aeb71d491985e296a0721d905d
I20260709 13:04:22.670941 15392 ts_tablet_manager.cc:933] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Initiating tablet copy from peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:22.707396 13943 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:22.711071 14033 tablet_copy_source_session.cc:215] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:22.654795 15353 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:04:22.747699 15372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 545fc4aeb71d491985e296a0721d905d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.756842 15360 raft_consensus.cc:993] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:22.757457 15360 raft_consensus.cc:1081] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:22.777617 13944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:35880
I20260709 13:04:22.796804 15098 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:22.803263 14024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:22.803987 14024 raft_consensus.cc:606] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:04:22.805723 15391 tablet_copy_client.cc:323] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.66:43953
I20260709 13:04:22.808162 13960 tablet_copy_service.cc:140] P 3779649036364744823fc035dac892c7: Received BeginTabletCopySession request for tablet 02dcf286c6bd48e3a6e548def7d786cc from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:58096)
I20260709 13:04:22.809031 13960 tablet_copy_service.cc:161] P 3779649036364744823fc035dac892c7: Beginning new tablet copy session on tablet 02dcf286c6bd48e3a6e548def7d786cc from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:58096: session id = 4dd265d61fc141a48ca9d3d5c8f52921-02dcf286c6bd48e3a6e548def7d786cc
I20260709 13:04:22.819806 13950 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:22.820501 13950 raft_consensus.cc:606] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:22.813437 15383 tablet_copy_client.cc:323] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.66:43953
I20260709 13:04:22.824597 15392 tablet_copy_client.cc:323] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Beginning tablet copy session from remote peer at address 127.12.109.67:46599
W20260709 13:04:22.845445 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 45064.9us
I20260709 13:04:22.854307 13868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:22.855206 13868 raft_consensus.cc:606] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:22.858433 15331 raft_consensus.cc:993] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:22.858961 15331 raft_consensus.cc:1081] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:22.866036 13943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76071b78766a43e0b5097f615de4c569"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:22.866487 13959 tablet_copy_service.cc:140] P 3779649036364744823fc035dac892c7: Received BeginTabletCopySession request for tablet 14bba4e2cef04a418c6f660869c7e6e5 from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:58096)
I20260709 13:04:22.866984 13943 raft_consensus.cc:606] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:22.867103 13959 tablet_copy_service.cc:161] P 3779649036364744823fc035dac892c7: Beginning new tablet copy session on tablet 14bba4e2cef04a418c6f660869c7e6e5 from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:58096: session id = 4dd265d61fc141a48ca9d3d5c8f52921-14bba4e2cef04a418c6f660869c7e6e5
I20260709 13:04:22.868470 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:04:22.876112 14024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49644
I20260709 13:04:22.876758 14024 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.877147 14024 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:22.905819 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:22.905898 14024 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:22.913941 14022 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49628
I20260709 13:04:22.914695 14022 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.915099 14022 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.918325 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:22.918959 13871 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:22.922732 15310 raft_consensus.cc:993] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:22.923329 15310 raft_consensus.cc:1081] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:22.914016 13959 tablet_copy_source_session.cc:215] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:22.931567 15383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14bba4e2cef04a418c6f660869c7e6e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.932616 14024 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:22.934772 13943 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:35894
I20260709 13:04:22.935352 13943 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.935739 13943 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.937080 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:22.938500 14033 tablet_copy_service.cc:140] P d1b4fd04200247d99ba9ad1fbeb42b42: Received BeginTabletCopySession request for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 from peer 4dd265d61fc141a48ca9d3d5c8f52921 ({username='slave'} at 127.0.0.1:55076)
I20260709 13:04:22.939040 14033 tablet_copy_service.cc:161] P d1b4fd04200247d99ba9ad1fbeb42b42: Beginning new tablet copy session on tablet 6b098cc2d51e471cb2c9c1b17eebcb57 from peer 4dd265d61fc141a48ca9d3d5c8f52921 at {username='slave'} at 127.0.0.1:55076: session id = 4dd265d61fc141a48ca9d3d5c8f52921-6b098cc2d51e471cb2c9c1b17eebcb57
I20260709 13:04:22.940196 13871 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:22.954595 13943 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:22.957001 13943 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:22.960464 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:22.961627 14024 raft_consensus.cc:3055] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:22.962106 14024 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:22.962159 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:22.962860 13870 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.963187 14033 tablet_copy_source_session.cc:215] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:22.963546 13989 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:22.964368 15343 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:22.964862 15343 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:22.963059 14024 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:22.965819 15343 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:22.975924 15392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b098cc2d51e471cb2c9c1b17eebcb57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.981454 13944 raft_consensus.cc:606] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Received request to transfer leadership
I20260709 13:04:22.980774 14024 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.987535 13960 tablet_copy_source_session.cc:215] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:22.991684 13870 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:22.993243 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:22.988709 14022 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:22.996028 14024 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:22.997463 15391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02dcf286c6bd48e3a6e548def7d786cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.998268 15016 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.001719 15016 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:23.002328 13942 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:23.002736 13942 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:23.002592 15016 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:22.997567 15359 raft_consensus.cc:993] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:23.006769 15359 raft_consensus.cc:1081] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:23.013492 13868 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
I20260709 13:04:23.014143 13868 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.014472 13868 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:23.017643 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:23.018308 13866 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.018563 13866 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:23.019389 13866 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:23.021960 13866 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.022549 15372 tablet_copy_client.cc:806] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:23.023253 15372 tablet_copy_client.cc:670] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:23.028410 13868 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:23.010195 13942 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:23.031630 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:23.032202 13866 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:23.032312 13950 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:23.032850 13950 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:23.033881 13950 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:23.034983 13942 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:23.035207 13950 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:23.036479 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:23.045538 15399 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.046392 15399 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:23.047914 15399 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:23.031533 14022 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:23.063514 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:23.064232 13943 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.081967 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:23.082850 14558 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:23.091646 13868 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:23.108466 14558 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:23.110014 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:23.109826 13769 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 4 to 5, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 5 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.111588 15385 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:23.116215 15385 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:23.117131 15385 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:23.121531 13942 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:23.127600 15387 raft_consensus.cc:993] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:23.138412 13943 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:23.154158 13950 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:23.154851 15372 tablet_copy_client.cc:538] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:23.176970 15320 raft_consensus.cc:1081] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:23.185889 13868 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76071b78766a43e0b5097f615de4c569"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41676
W20260709 13:04:23.221969 13821 auto_rebalancer.cc:893] Removing replace marker failed: Replica 3779649036364744823fc035dac892c7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } } }
W20260709 13:04:23.314507 13821 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 02dcf286c6bd48e3a6e548def7d786cc already in progress
W20260709 13:04:23.314999 13821 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 02dcf286c6bd48e3a6e548def7d786cc already in progress
I20260709 13:04:23.186767 13868 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.321373 13868 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:23.310324 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 127067us
I20260709 13:04:23.324867 15383 tablet_copy_client.cc:806] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:23.326221 15383 tablet_copy_client.cc:670] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
W20260709 13:04:23.326362 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 135623us
I20260709 13:04:23.337010 15391 tablet_copy_client.cc:806] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
W20260709 13:04:23.313848 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.363325 13772 catalog_manager.cc:5697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.397493 15372 tablet_bootstrap.cc:492] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:04:23.397188 13868 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:23.401432 13868 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:23.383665 15391 tablet_copy_client.cc:670] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
W20260709 13:04:23.412119 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 185994us
I20260709 13:04:23.412801 13769 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 4 to 5, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 5 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.418712 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:23.419514 13942 raft_consensus.cc:3055] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.419858 13942 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:23.420523 15392 tablet_copy_client.cc:806] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:23.421236 15392 tablet_copy_client.cc:670] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:23.457643 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:23.463657 14024 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.420779 13942 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:23.494663 14024 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:23.495571 15383 tablet_copy_client.cc:538] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:23.498926 13871 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.520546 15386 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:23.552124 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:23.554023 15385 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.554620 15385 raft_consensus.cc:697] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:23.555691 15385 consensus_queue.cc:237] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:23.570179 13942 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.572199 13771 catalog_manager.cc:5697] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 1 to 2, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:23.582433 13870 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:23.585347 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 56200us
I20260709 13:04:23.586982 15391 tablet_copy_client.cc:538] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:04:23.588161 13988 consensus_peers.cc:597] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Couldn't send request to peer 4dd265d61fc141a48ca9d3d5c8f52921. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:23.592629 14023 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.592736 13870 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:23.593684 15413 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.614039 15392 tablet_copy_client.cc:538] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:23.608915 15413 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:04:23.614100 15357 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:04:23.620339 13942 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:23.627573 13941 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:23.632997 15362 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:23.662295 15392 tablet_bootstrap.cc:492] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:04:23.662258 15391 tablet_bootstrap.cc:492] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
I20260709 13:04:23.690929 13943 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:04:23.734432 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 43598.6us
I20260709 13:04:23.731264 13769 catalog_manager.cc:5697] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.754869 13942 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:23.763033 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 70547.1us
W20260709 13:04:23.821007 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46436.8us
I20260709 13:04:23.834421 15373 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:04:23.820894 15383 tablet_bootstrap.cc:492] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap starting.
W20260709 13:04:23.848433 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 78338.8us
I20260709 13:04:23.855536 15373 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:04:23.907377 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:23.909248 14555 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:23.941898 15427 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
W20260709 13:04:23.971297 15428 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:04:23.979245 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 150761us
I20260709 13:04:24.123281 13948 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:24.107012 15415 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:04:24.222415 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 174300us
W20260709 13:04:24.223167 15403 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:04:24.220957 15372 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:04:24.250432 15402 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:04:24.265944 15384 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:24.435326 15432 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:04:24.528021 13868 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.006s	sys 0.022s
W20260709 13:04:24.529958 13868 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.007s	sys 0.022s
W20260709 13:04:24.530606 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 299314us
I20260709 13:04:24.603925 15372 tablet_bootstrap.cc:492] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:04:24.685721 13950 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.012s	sys 0.024s
W20260709 13:04:24.693516 13950 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.012s	sys 0.024s
I20260709 13:04:24.707880 15372 tablet_bootstrap.cc:492] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
W20260709 13:04:24.712121 13839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.008s	sys 0.024s
W20260709 13:04:24.719884 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 509665us
W20260709 13:04:24.770668 13987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.022s	sys 0.050s
W20260709 13:04:24.771972 13987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.023s	sys 0.050s
I20260709 13:04:24.769784 15372 ts_tablet_manager.cc:1403] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 1.373s	user 0.041s	sys 0.012s
I20260709 13:04:24.795557 15391 tablet_bootstrap.cc:492] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:24.810627 15392 tablet_bootstrap.cc:492] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:24.836928 14018 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.855825 15372 raft_consensus.cc:359] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:24.856909 15372 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:24.857735 15451 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:24.857671 15372 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:24.847347 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:24.860956 15372 ts_tablet_manager.cc:1434] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.080s	user 0.002s	sys 0.005s
I20260709 13:04:24.829859 15392 tablet_bootstrap.cc:492] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:04:24.826440 15391 tablet_bootstrap.cc:492] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
W20260709 13:04:24.937013 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52489.2us
I20260709 13:04:24.937214 14033 tablet_copy_service.cc:342] P d1b4fd04200247d99ba9ad1fbeb42b42: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-545fc4aeb71d491985e296a0721d905d received from {username='slave'} at 127.0.0.1:55076
I20260709 13:04:24.937762 14033 tablet_copy_service.cc:434] P d1b4fd04200247d99ba9ad1fbeb42b42: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-545fc4aeb71d491985e296a0721d905d on tablet 545fc4aeb71d491985e296a0721d905d with peer 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:04:24.952083 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 66035.7us
I20260709 13:04:24.968533 15392 ts_tablet_manager.cc:1403] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 1.306s	user 0.036s	sys 0.012s
I20260709 13:04:24.980927 15391 ts_tablet_manager.cc:1403] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 1.335s	user 0.040s	sys 0.025s
I20260709 13:04:25.020565 14555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 204) took 1037 ms. Trace:
I20260709 13:04:25.021154 14555 rpcz_store.cc:276] 0709 13:04:23.982886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.005215 (+ 22329us) service_pool.cc:224] Handling call
0709 13:04:24.052423 (+ 47208us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:24.715143 (+662720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:24.715263 (+   120us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:24.716747 (+  1484us) log.cc:844] Serialized 26 byte log entry
0709 13:04:24.789644 (+ 72897us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:24.999187 (+209543us) log.cc:844] Serialized 14 byte log entry
0709 13:04:25.008578 (+  9391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.020247 (+ 11669us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:25.020301 (+    54us) raft_consensus.cc:1685] finished
0709 13:04:25.020312 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.020474 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45275,"raft.queue_time_us":108489,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":18148,"thread_start_us":680094,"threads_started":3,"wal-append.queue_time_us":145476}
W20260709 13:04:25.031360 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 92190.6us
I20260709 13:04:25.120036 15392 raft_consensus.cc:359] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:25.121054 15392 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:25.123481 15391 raft_consensus.cc:359] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:25.124497 15391 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:25.125278 15391 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:25.127501 15392 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:25.194804 15392 ts_tablet_manager.cc:1434] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.225s	user 0.009s	sys 0.000s
I20260709 13:04:25.202275 13869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:25.203003 13869 raft_consensus.cc:606] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 1 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:25.230497 15391 ts_tablet_manager.cc:1434] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.230s	user 0.009s	sys 0.000s
I20260709 13:04:25.240780 13960 tablet_copy_service.cc:342] P 3779649036364744823fc035dac892c7: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-02dcf286c6bd48e3a6e548def7d786cc received from {username='slave'} at 127.0.0.1:58096
I20260709 13:04:25.241369 13960 tablet_copy_service.cc:434] P 3779649036364744823fc035dac892c7: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-02dcf286c6bd48e3a6e548def7d786cc on tablet 02dcf286c6bd48e3a6e548def7d786cc with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:25.241794 14016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:25.242672 14016 raft_consensus.cc:606] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:25.280821 14555 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEARNER]: Advancing to term 5
I20260709 13:04:25.303370 13869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:25.304140 13869 raft_consensus.cc:606] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:25.321549 14033 tablet_copy_service.cc:342] P d1b4fd04200247d99ba9ad1fbeb42b42: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-6b098cc2d51e471cb2c9c1b17eebcb57 received from {username='slave'} at 127.0.0.1:55076
I20260709 13:04:25.322162 14033 tablet_copy_service.cc:434] P d1b4fd04200247d99ba9ad1fbeb42b42: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-6b098cc2d51e471cb2c9c1b17eebcb57 on tablet 6b098cc2d51e471cb2c9c1b17eebcb57 with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:25.382951 14024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:25.383680 14024 raft_consensus.cc:606] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
W20260709 13:04:25.390460 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 104671us
I20260709 13:04:25.393133 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:04:25.467036 14560 raft_consensus.cc:1217] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
W20260709 13:04:25.526736 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 60004.8us
W20260709 13:04:25.530426 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 267992us
I20260709 13:04:25.563796 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:25.591847 14560 raft_consensus.cc:1217] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:25.597810 15383 tablet_bootstrap.cc:492] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:25.605226 15383 tablet_bootstrap.cc:492] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Bootstrap complete.
I20260709 13:04:25.632239 15383 ts_tablet_manager.cc:1403] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent bootstrapping tablet: real 1.839s	user 0.047s	sys 0.024s
I20260709 13:04:25.650435 15441 raft_consensus.cc:993] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:25.651057 15441 raft_consensus.cc:1081] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:25.715677 15424 raft_consensus.cc:993] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:25.715893 15384 raft_consensus.cc:993] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:25.716414 15384 raft_consensus.cc:1081] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 1 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:25.716801 15422 raft_consensus.cc:1081] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Signalling peer e1649107c781428692458f10597c5742 to start an election
I20260709 13:04:25.715859 15411 raft_consensus.cc:993] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:25.717890 15411 raft_consensus.cc:1081] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:04:25.720362 13869 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7"
dest_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41684
I20260709 13:04:25.720963 13869 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:25.721299 13869 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:25.735268 14024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49628
I20260709 13:04:25.737562 13942 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:35894
I20260709 13:04:25.738268 13942 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:25.738698 13942 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.737502 14016 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49628
I20260709 13:04:25.740479 14016 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:25.741014 14016 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:25.736954 13869 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:25.746498 13869 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:25.735976 14024 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:25.748270 14024 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:25.764436 13942 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:25.704892 15383 raft_consensus.cc:359] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:25.770601 15383 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Initialized, Role: LEARNER
I20260709 13:04:25.771610 15383 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:25.776119 13942 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:25.794425 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 59799.9us
I20260709 13:04:25.796530 14024 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:25.797753 14016 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:25.804911 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:25.805564 13870 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:25.805898 13870 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:25.806087 14016 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:25.809091 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:25.809341 14024 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:25.809723 13869 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:25.810542 13869 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:25.811352 13869 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:25.812826 13869 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:25.813908 15425 raft_consensus.cc:1064] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:25.811328 13870 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:25.820626 13870 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:25.816792 15425 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:25.819523 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:25.832551 13946 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:25.836861 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:25.837585 13948 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:25.838207 13869 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:25.840938 13989 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:25.843645 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:25.844266 14024 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:25.844597 14024 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:25.849108 14024 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:25.809410 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:25.852649 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:25.853307 13944 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:25.850535 14024 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.858336 13870 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:25.860509 15411 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:25.873379 13948 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:25.863809 15411 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:25.859781 14560 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.874892 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:25.883447 14024 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:25.885020 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:25.900121 14015 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:25.900502 14015 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:25.893493 13944 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:25.902083 14015 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:25.906075 14015 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:25.910872 15411 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:25.920598 14015 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:25.907624 15421 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:25.929699 15421 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:25.930706 15421 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:25.945727 15383 ts_tablet_manager.cc:1434] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent starting tablet: real 0.312s	user 0.010s	sys 0.000s
W20260709 13:04:25.977973 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 74914.7us
I20260709 13:04:25.991721 15444 raft_consensus.cc:1064] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:25.996124 14556 raft_consensus.cc:1217] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 13:04:25.996361 15444 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:26.003710 13960 tablet_copy_service.cc:342] P 3779649036364744823fc035dac892c7: Request end of tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-14bba4e2cef04a418c6f660869c7e6e5 received from {username='slave'} at 127.0.0.1:58096
I20260709 13:04:26.004231 13960 tablet_copy_service.cc:434] P 3779649036364744823fc035dac892c7: ending tablet copy session 4dd265d61fc141a48ca9d3d5c8f52921-14bba4e2cef04a418c6f660869c7e6e5 on tablet 14bba4e2cef04a418c6f660869c7e6e5 with peer 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:26.020905 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:26.026089 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:26.037529 15430 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:26.038273 15430 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:26.039372 15430 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:26.042560 15359 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:26.043095 15359 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:26.043954 15359 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:26.085666 14560 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:26.041706 13946 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:26.102461 13870 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:26.103981 15410 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:26.105165 13944 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:26.114899 14560 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:26.132388 13771 catalog_manager.cc:5697] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 3, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:26.131474 15399 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:04:26.167747 13873 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:26.168583 13946 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:26.173549 15399 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:04:26.277235 14560 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:26.176398 15420 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.114s
W20260709 13:04:26.336290 15478 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.130s	user 0.008s	sys 0.000s
I20260709 13:04:26.347712 13769 catalog_manager.cc:5697] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 1 to 2, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:26.365176 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 230505us
W20260709 13:04:26.374869 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 50986.8us
I20260709 13:04:26.289672 15445 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:04:26.381850 15310 raft_consensus.cc:1064] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
I20260709 13:04:26.428830 13948 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:26.436486 13772 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 3 to 4, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 4 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:26.448190 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 89100us
W20260709 13:04:26.445747 15477 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
W20260709 13:04:26.461341 15480 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 13:04:26.462677 15483 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
W20260709 13:04:26.466292 15474 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.150s	user 0.002s	sys 0.000s
I20260709 13:04:26.429545 15425 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:04:26.450125 15485 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:04:26.488018 15310 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:26.500013 15491 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:04:26.515482 15421 raft_consensus.cc:2955] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Committing config change with OpId 5.7: config changed from index 4 to 7, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:26.551484 13943 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:26.553625 14560 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:04:26.554836 15410 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:26.555279 15359 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:26.561412 13872 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:26.554951 13870 raft_consensus.cc:2955] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 4 to 7, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:26.570428 15490 raft_consensus.cc:1064] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: attempting to promote NON_VOTER 4dd265d61fc141a48ca9d3d5c8f52921 to VOTER
W20260709 13:04:26.585127 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 156786us
I20260709 13:04:26.600859 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:26.621857 15412 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:04:26.647627 15490 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:26.712917 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 56173.4us
W20260709 13:04:26.802973 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 219284us
W20260709 13:04:26.925026 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 132233us
I20260709 13:04:26.930300 13871 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:26.934396 14020 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:04:26.936920 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 218059us
W20260709 13:04:27.110755 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 172793us
W20260709 13:04:27.121971 15498 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.450s	user 0.000s	sys 0.001s
W20260709 13:04:27.192202 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 258584us
W20260709 13:04:27.269330 15421 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.751s	user 0.005s	sys 0.009s
I20260709 13:04:27.244609 15444 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:04:27.264055 15328 raft_consensus.cc:2955] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:27.295899 15506 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:04:27.300915 15489 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.096s
W20260709 13:04:27.347659 15496 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.547s	user 0.015s	sys 0.000s
W20260709 13:04:27.379827 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 140570us
I20260709 13:04:27.407902 14556 raft_consensus.cc:2955] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 4 to 7, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:27.467221 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 84246us
I20260709 13:04:27.454133 14555 raft_consensus.cc:2955] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:27.488821 13944 raft_consensus.cc:2955] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 4 to 7, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:27.561730 14016 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:04:27.542306 15411 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:27.588702 13868 raft_consensus.cc:2955] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:27.570205 13942 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:27.569926 15452 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:27.614629 13948 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:27.615020 14561 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 LEARNER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:27.621053 15420 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:27.639750 15504 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:27.640504 15504 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:27.608422 15411 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:27.619431 15399 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:27.645638 15411 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:27.646256 15511 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:27.617622 15452 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:27.648655 15505 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:27.649857 15511 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:27.689908 15511 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:27.700528 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:27.662393 15399 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:27.841687 15399 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:27.692713 15505 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:27.851152 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:27.881628 15505 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:27.849373 15504 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:27.856974 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 136579us
W20260709 13:04:27.894845 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 260276us
I20260709 13:04:27.898943 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:27.911440 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 289017us
I20260709 13:04:27.864935 15452 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:27.915151 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 303338us
I20260709 13:04:27.979146 13872 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:27.964905 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:28.012975 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:28.033514 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 102457us
W20260709 13:04:28.034245 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 108288us
I20260709 13:04:28.036922 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:28.039369 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 1807) took 1105 ms. Trace:
I20260709 13:04:28.040007 13866 rpcz_store.cc:276] 0709 13:04:26.929771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.930222 (+   451us) service_pool.cc:224] Handling call
0709 13:04:26.930521 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.402960 (+472439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.402998 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.467847 (+ 64849us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.654461 (+186614us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:27.654518 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.654619 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.034901 (+380282us) raft_consensus.cc:1685] finished
0709 13:04:28.034932 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.035116 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":218127,"raft.queue_time_us":260373,"raft.run_cpu_time_us":600,"raft.run_wall_time_us":5474,"thread_start_us":365482,"threads_started":2,"wal-append.queue_time_us":347227}
I20260709 13:04:27.898558 15488 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.138s
W20260709 13:04:28.033670 15498 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:04:28.077843 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:28.094734 15460 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:28.094854 13949 raft_consensus.cc:2955] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:28.107402 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 209201us
W20260709 13:04:28.093591 15496 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:04:28.027293 15515 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 13:04:28.091295 15504 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.091653 15505 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:28.113603 15424 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:28.115952 15425 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.095510 15460 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.135043 15529 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
W20260709 13:04:28.170888 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 270754us
I20260709 13:04:28.204829 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:28.182683 14016 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 1.
W20260709 13:04:28.205670 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 78984.6us
I20260709 13:04:28.155534 15422 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.228992 15373 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.229867 15373 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:28.232908 15373 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:28.228572 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 194011us
I20260709 13:04:28.247733 15449 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:28.137565 15504 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:28.237766 15496 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:04:28.127398 15505 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:28.293161 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46722.7us
W20260709 13:04:28.298223 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 371085us
I20260709 13:04:28.219381 15425 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.215432 15424 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.289290 15481 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:28.352938 15510 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:28.353569 15547 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:28.353994 15535 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:04:28.336464 15552 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.187s	user 0.000s	sys 0.009s
I20260709 13:04:28.353747 15460 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:28.246930 15422 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.371524 15422 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:28.260424 15449 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:28.372354 15407 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.373035 15407 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:28.386525 15512 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.392302 15524 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.388737 15373 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.389073 15399 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.392328 15425 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:28.387207 15512 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.399199 15524 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.436417 15515 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.438661 15564 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:28.417954 15537 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.315s	user 0.002s	sys 0.000s
I20260709 13:04:28.384462 15547 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:28.439489 15564 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:28.378902 15481 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.456101 15550 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:28.384905 15510 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.418926 15504 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:28.388239 15535 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.473296 15510 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:28.430167 15373 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:28.430265 15399 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.489003 15399 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:28.459664 15449 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:28.501339 15505 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:28.418229 15546 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.437994 15515 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.491436 15550 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:28.635634 15564 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:28.643234 13771 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57626 (request call id 240) took 2531 ms. Trace:
I20260709 13:04:28.643916 15535 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:28.644811 15505 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:28.663570 15524 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:28.588419 15512 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:28.672875 13914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.067s	sys 0.128s
W20260709 13:04:28.675339 13914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.067s	sys 0.128s
W20260709 13:04:28.677280 13915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.003s	sys 0.026s
W20260709 13:04:28.677654 13915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.003s	sys 0.026s
W20260709 13:04:28.677976 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 608616us
I20260709 13:04:28.603488 15546 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.667889 15505 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:28.747280 15547 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:28.651432 13771 rpcz_store.cc:276] 0709 13:04:26.109201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.110187 (+   986us) service_pool.cc:224] Handling call
0709 13:04:28.640926 (+2530739us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:26.437179 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:26.437798 (+   619us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:26.437834 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:04:26.437852 (+    18us) tablet.cc:662] Decoding operations
0709 13:04:26.438230 (+   378us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:26.438307 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:26.438342 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:26.438533 (+   191us) tablet.cc:701] Row locks acquired
0709 13:04:26.438549 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:04:26.438692 (+   143us) write_op.cc:270] Start()
0709 13:04:26.438817 (+   125us) write_op.cc:276] Timestamp: P: 1783602266438645 usec, L: 0
0709 13:04:26.438839 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:04:26.439614 (+   775us) log.cc:844] Serialized 541 byte log entry
0709 13:04:26.519922 (+ 80308us) spinlock_profiling.cc:243] Waited 5.78 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c03290ac 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b
0709 13:04:27.309157 (+789235us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:04:27.872406 (+563249us) op_driver.cc:464] REPLICATION: finished
0709 13:04:28.378228 (+505822us) write_op.cc:301] APPLY: starting
0709 13:04:28.378338 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:28.378456 (+   118us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:28.378481 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:28.386557 (+  8076us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:28.386830 (+   273us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:04:28.386907 (+    77us) write_op.cc:312] APPLY: finished
0709 13:04:28.387229 (+   322us) log.cc:844] Serialized 23 byte log entry
0709 13:04:28.387487 (+   258us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:28.387616 (+   129us) write_op.cc:454] Released schema lock
0709 13:04:28.387686 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:28.401140 (+ 13454us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:28.409634 (+  8494us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":505431,"apply.run_cpu_time_us":2482,"apply.run_wall_time_us":15773,"num_ops":1,"prepare.queue_time_us":292179,"prepare.run_cpu_time_us":12235,"prepare.run_wall_time_us":167655,"raft.queue_time_us":490488,"raft.run_cpu_time_us":62786,"raft.run_wall_time_us":172794,"replication_time_us":1433337,"spinlock_wait_cycles":46933760,"thread_start_us":1079209,"threads_started":4,"wal-append.queue_time_us":218445}]]}
I20260709 13:04:28.759972 14556 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEARNER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:28.706363 15407 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:28.705726 15467 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.743170 15512 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.781338 15542 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.788906 15560 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.785550 15547 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:28.768122 15449 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:04:28.816973 15481 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:28.801874 15566 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:28.848397 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:28.853094 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:28.846896 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:28.847707 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:28.850186 15559 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.850762 15565 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:28.789297 15467 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.808490 15512 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.855957 13873 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:04:28.812647 15542 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:28.828578 15515 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:28.819582 15547 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.864094 13866 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
W20260709 13:04:28.870216 13913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.007s	sys 0.019s
W20260709 13:04:28.870766 13913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.007s	sys 0.020s
I20260709 13:04:28.843528 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 1810) took 1344 ms. Trace:
I20260709 13:04:28.879130 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:28.879244 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:28.880817 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:28.827899 15560 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.893584 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:28.908670 15519 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:28.909081 15556 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:28.908710 15574 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.909718 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:28.908947 15543 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:28.909818 15556 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:28.910674 14554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 396) took 1220 ms (client timeout 1618 ms). Trace:
W20260709 13:04:28.911207 14554 rpcz_store.cc:269] 0709 13:04:27.690649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.690919 (+   270us) service_pool.cc:224] Handling call
0709 13:04:28.909405 (+1218486us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b5800004898. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:04:28.910588 (+  1183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4145920}
I20260709 13:04:28.915057 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:28.915796 13944 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:28.870296 15508 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:04:28.909461 15519 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:28.888537 15505 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:28.923452 15560 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:28.879469 13943 rpcz_store.cc:276] 0709 13:04:27.485254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.494754 (+  9500us) service_pool.cc:224] Handling call
0709 13:04:28.767481 (+1272727us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b5800202e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:04:28.785237 (+ 17756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.795465 (+ 10228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.798882 (+  3417us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:28.808137 (+  9255us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:28.810894 (+  2757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.820684 (+  9790us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.829407 (+  8723us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3333376}
W20260709 13:04:28.937724 15558 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:04:28.882692 15550 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:28.931329 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:28.946538 15542 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:28.986287 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55082.1us
I20260709 13:04:28.947489 15551 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:28.881698 15568 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:28.949971 13914 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:28.880182 15565 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.000628 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:29.000675 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 50905.8us
I20260709 13:04:28.909857 15574 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.011638 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.012490 14015 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:29.013484 15574 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.013836 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:28.995393 15481 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:29.018124 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.017417 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:28.992591 15550 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:04:29.020578 15544 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:04:28.863466 15566 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.020295 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:28.880295 15559 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.026203 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.026732 15565 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.027122 15559 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:29.000218 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.027980 13943 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:29.028941 15566 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:29.003571 15460 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:29.020939 15448 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:29.003129 15560 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:29.030267 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.039709 15500 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.039750 15577 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:28.910955 15543 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.040341 15500 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.040541 15577 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.034983 14020 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:29.004693 15542 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:29.043156 15500 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.045178 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:29.048753 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:29.020812 15563 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:29.022521 15556 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.043193 15543 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.021733 15481 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.052024 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:29.043342 15542 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.052937 15475 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.053128 15569 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:29.037775 15574 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.062486 15573 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:29.031841 15550 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.059315 15481 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:29.063376 15547 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.067595 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.044106 15535 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.073868 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.074982 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.075182 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.050498 15565 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.076769 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.075855 13945 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:29.077730 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.068193 15543 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.047212 15560 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.008831 15551 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.065879 15563 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.090559 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.074664 15422 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:29.075858 15500 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.022209 15568 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.090860 15512 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:29.090585 15514 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:29.111937 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.112582 13872 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:29.079773 15535 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.075352 15574 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.114607 15577 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:29.099757 15563 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:29.114778 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.107760 15422 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.097179 15543 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.120015 15551 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:29.120632 15551 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:29.121110 15422 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:29.094254 15467 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.092939 15565 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.124257 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.124361 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.124364 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.128855 13867 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:29.125269 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.058935 15448 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.121114 15551 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.074535 15475 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.121675 15543 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.133735 15448 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:29.140797 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.131294 15574 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.127007 15487 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.121234 15562 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:04:29.150959 15583 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.151821 15582 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:29.152547 15580 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.138888 15542 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:29.115127 15500 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.160252 13838 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:29.163669 15525 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:29.167551 15576 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:04:29.174906 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 103248us
W20260709 13:04:29.164376 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 102617us
I20260709 13:04:29.177385 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.181260 13838 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:29.185832 15575 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:29.186280 15575 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.186623 15575 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:29.178350 15567 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:04:29.174031 15576 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.201462 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.208132 13838 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:29.145262 15514 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.212900 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.206365 13875 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:04:29.184870 15487 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.197296 15547 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:29.218655 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.219219 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.219419 13945 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:29.222405 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.223062 14024 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:29.223057 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.223908 13870 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:29.202310 15560 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:29.224067 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.226007 14015 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:29.227284 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.227900 13949 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:04:29.207007 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.178249 15580 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.176738 15582 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.179005 15583 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:29.488283 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 259490us
I20260709 13:04:29.491909 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.492583 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.493295 13949 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:29.232069 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.253051 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.511704 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.527274 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:29.613335 15584 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:29.218677 15556 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:29.218938 15547 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.624677 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.644029 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.664387 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.676721 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.418743 15565 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.677436 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.248054 15543 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:29.458297 15399 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:29.431232 15551 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.491673 15535 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.493155 15560 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:29.688723 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.621505 15500 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:29.255916 15585 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:29.695726 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.623060 15556 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.628265 15547 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:29.699810 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.694370 15593 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.669986 15515 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:29.704586 14020 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:29.676074 15492 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.677700 15559 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:29.713034 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:29.713616 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 173320us
I20260709 13:04:29.714097 13874 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:04:29.308310 15481 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:29.715931 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.314010 15563 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.719424 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.614375 15584 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.708786 15436 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.710593 15556 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:29.699244 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 1814) took 1759 ms. Trace:
I20260709 13:04:29.722380 13769 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57648 (request call id 272) took 3393 ms. Trace:
I20260709 13:04:29.729362 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.729697 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:29.700594 15487 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.689694 15399 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.746538 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:29.730784 13867 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:29.759209 13941 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 2: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:29.760980 13840 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:29.701581 15543 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.716040 15550 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.762534 13840 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:29.779899 15584 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:29.729244 13769 rpcz_store.cc:276] 0709 13:04:26.329185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.331806 (+  2621us) service_pool.cc:224] Handling call
0709 13:04:29.722285 (+3390479us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:26.638545 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:26.937602 (+299057us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:26.941375 (+  3773us) write_op.cc:435] Acquired schema lock
0709 13:04:26.945315 (+  3940us) tablet.cc:662] Decoding operations
0709 13:04:27.109787 (+164472us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:27.113197 (+  3410us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:27.117446 (+  4249us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:27.123131 (+  5685us) tablet.cc:701] Row locks acquired
0709 13:04:27.123152 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:04:27.123305 (+   153us) write_op.cc:270] Start()
0709 13:04:27.123434 (+   129us) write_op.cc:276] Timestamp: P: 1783602267123266 usec, L: 0
0709 13:04:27.123472 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:04:27.124468 (+   996us) log.cc:844] Serialized 541 byte log entry
0709 13:04:28.381461 (+1256993us) op_driver.cc:464] REPLICATION: finished
0709 13:04:28.431818 (+ 50357us) write_op.cc:301] APPLY: starting
0709 13:04:28.435595 (+  3777us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:28.451584 (+ 15989us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:28.464760 (+ 13176us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:28.503765 (+ 39005us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:28.588186 (+ 84421us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:04:28.605143 (+ 16957us) write_op.cc:312] APPLY: finished
0709 13:04:28.651752 (+ 46609us) log.cc:844] Serialized 23 byte log entry
0709 13:04:28.651989 (+   237us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:28.652116 (+   127us) write_op.cc:454] Released schema lock
0709 13:04:28.652162 (+    46us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":29632,"apply.run_cpu_time_us":3618,"apply.run_wall_time_us":220784,"num_ops":1,"prepare.queue_time_us":275478,"prepare.run_cpu_time_us":4478,"prepare.run_wall_time_us":488262,"replication_time_us":1255011,"thread_start_us":375697,"threads_started":1}]]}
I20260709 13:04:29.784653 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.787173 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:29.714205 15560 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.715919 15515 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.719754 15559 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.787151 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.764317 15487 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:29.799537 13947 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:29.783452 15565 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:29.719750 15593 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.765158 15475 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.809363 15591 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:29.811101 15588 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:29.719385 15585 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.789310 15566 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:29.795401 15514 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:29.829090 14024 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:29.717317 15492 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.811905 15588 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.736832 13942 rpcz_store.cc:276] 0709 13:04:27.936217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.936357 (+   140us) service_pool.cc:224] Handling call
0709 13:04:28.049927 (+113570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.097563 (+ 47636us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:28.153277 (+ 55714us) log.cc:844] Serialized 14 byte log entry
0709 13:04:29.240269 (+1086992us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b1800360ca0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287 00007fb2c344e208
0709 13:04:29.622161 (+381892us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.626404 (+  4243us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:29.663148 (+ 36744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.680402 (+ 17254us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.695936 (+ 15534us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22733,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":157080,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":5748,"spinlock_wait_cycles":23935232,"thread_start_us":243607,"threads_started":1}
I20260709 13:04:29.837572 15373 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:29.842939 15543 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.846586 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1059) took 2224 ms. Trace:
I20260709 13:04:29.807188 15565 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:29.833074 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:29.821611 15448 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:29.787707 15513 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:29.773137 15436 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.810174 15591 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:29.970134 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 1865) took 1810 ms. Trace:
I20260709 13:04:29.970636 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 1870) took 1744 ms. Trace:
I20260709 13:04:29.925550 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 1864) took 1765 ms. Trace:
I20260709 13:04:29.973992 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.850682 13946 rpcz_store.cc:276] 0709 13:04:27.616996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.617549 (+   553us) service_pool.cc:224] Handling call
0709 13:04:27.617848 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.688684 (+ 70836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.688716 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:27.688761 (+    45us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:27.688796 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.841335 (+2152539us) spinlock_profiling.cc:243] Waited 70.6 ms on lock 0x7b54001469d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:29.841404 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.841578 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162387712}
I20260709 13:04:29.923451 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.136911 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:30.144403 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 1962) took 1295 ms. Trace:
I20260709 13:04:29.949373 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 1867) took 1768 ms. Trace:
I20260709 13:04:29.807181 15527 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:29.827795 15566 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:29.985663 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.090736 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.162200 14023 rpcz_store.cc:276] 0709 13:04:28.159776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.159946 (+   170us) service_pool.cc:224] Handling call
0709 13:04:29.924720 (+1764774us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030eeee 0000000000375939 00007fb2c83821e0 00007fb2c844b0c4 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:04:29.924945 (+   225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.925186 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.925214 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.925262 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:29.925293 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.925350 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.925459 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25063168}
I20260709 13:04:30.068866 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.961707 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.169946 13869 rpcz_store.cc:276] 0709 13:04:28.848816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.848955 (+   139us) service_pool.cc:224] Handling call
0709 13:04:28.849269 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.887458 (+ 38189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.887495 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:28.887556 (+    61us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:28.887594 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.144166 (+1256572us) spinlock_profiling.cc:243] Waited 38 ms on lock 0x7b54003ac5d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:30.144215 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.144343 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87366528}
I20260709 13:04:30.177521 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.951709 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 1868) took 1770 ms. Trace:
I20260709 13:04:30.161375 14016 rpcz_store.cc:276] 0709 13:04:28.227901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.228456 (+   555us) service_pool.cc:224] Handling call
0709 13:04:28.822892 (+594436us) spinlock_profiling.cc:243] Waited 67 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:04:28.823221 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.886438 (+ 63217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.886477 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:28.886525 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:28.886558 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.970365 (+1083807us) spinlock_profiling.cc:243] Waited 63 ms on lock 0x7b5400411754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:29.970426 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.970561 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299050624}
I20260709 13:04:30.133903 13772 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57642 (request call id 265) took 3841 ms. Trace:
I20260709 13:04:30.143886 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 1871) took 1915 ms. Trace:
W20260709 13:04:30.097139 15575 consensus_meta.cc:287] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.910s	user 0.010s	sys 0.005s
I20260709 13:04:29.884457 15583 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:30.240114 13772 rpcz_store.cc:276] 0709 13:04:26.318195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.325418 (+  7223us) service_pool.cc:224] Handling call
0709 13:04:30.133829 (+3808411us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:27.125106 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:27.125898 (+   792us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:27.125949 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:04:27.125977 (+    28us) tablet.cc:662] Decoding operations
0709 13:04:27.195317 (+ 69340us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:27.201008 (+  5691us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:27.202712 (+  1704us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:27.232797 (+ 30085us) tablet.cc:701] Row locks acquired
0709 13:04:27.235519 (+  2722us) write_op.cc:260] PREPARE: finished
0709 13:04:27.250558 (+ 15039us) write_op.cc:270] Start()
0709 13:04:27.259343 (+  8785us) write_op.cc:276] Timestamp: P: 1783602267245001 usec, L: 0
0709 13:04:27.264096 (+  4753us) op_driver.cc:348] REPLICATION: starting
0709 13:04:27.380016 (+115920us) log.cc:844] Serialized 536 byte log entry
0709 13:04:28.820025 (+1440009us) op_driver.cc:464] REPLICATION: finished
0709 13:04:28.843903 (+ 23878us) write_op.cc:301] APPLY: starting
0709 13:04:28.855472 (+ 11569us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:28.867120 (+ 11648us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:28.888344 (+ 21224us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:28.895128 (+  6784us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:28.915487 (+ 20359us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:04:28.926775 (+ 11288us) write_op.cc:312] APPLY: finished
0709 13:04:28.959044 (+ 32269us) log.cc:844] Serialized 23 byte log entry
0709 13:04:29.001718 (+ 42674us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:29.003680 (+  1962us) write_op.cc:454] Released schema lock
0709 13:04:29.006270 (+  2590us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:29.032480 (+ 26210us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:29.037493 (+  5013us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":704390,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":6941,"apply.run_cpu_time_us":3070,"apply.run_wall_time_us":191867,"num_ops":1,"prepare.queue_time_us":671311,"prepare.run_cpu_time_us":12210,"prepare.run_wall_time_us":334035,"replication_time_us":1547625}]]}
I20260709 13:04:29.919128 15399 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:29.890450 15539 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:30.038101 15559 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:30.194219 13945 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.003s	sys 0.004s
I20260709 13:04:30.178720 13871 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:04:30.269918 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 1869) took 1984 ms. Trace:
I20260709 13:04:29.981220 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.270448 14017 rpcz_store.cc:276] 0709 13:04:28.181958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.237536 (+ 55578us) service_pool.cc:224] Handling call
0709 13:04:30.151555 (+1914019us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:04:30.165465 (+ 13910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.165696 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.165722 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:30.165765 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:30.165795 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.165852 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.165964 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24456576}
I20260709 13:04:30.170337 14022 rpcz_store.cc:276] 0709 13:04:28.181218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.181435 (+   217us) service_pool.cc:224] Handling call
0709 13:04:28.823824 (+642389us) spinlock_profiling.cc:243] Waited 79.8 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:04:28.824070 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.029045 (+204975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.029080 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.029128 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:29.029160 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.949021 (+919861us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b540020d0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:29.949104 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.949299 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654354560}
I20260709 13:04:30.040529 15560 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:30.241447 13914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.009s	sys 0.009s
W20260709 13:04:30.272157 13914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.010s	sys 0.009s
I20260709 13:04:29.952555 15568 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:30.167232 15565 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:30.220017 15353 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:30.213413 13838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.005s	sys 0.003s
I20260709 13:04:29.984916 15387 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:30.026583 15563 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.969084 15584 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:30.288319 15584 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:30.289215 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:29.933697 15546 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:30.039283 15515 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:30.069304 15588 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.852842 15580 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:30.213802 13989 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.006s	sys 0.003s
W20260709 13:04:30.292135 13989 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.008s	sys 0.004s
W20260709 13:04:30.292480 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 620003us
I20260709 13:04:30.293782 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:30.195116 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 408019us
I20260709 13:04:30.240475 14019 rpcz_store.cc:276] 0709 13:04:28.244449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.247977 (+  3528us) service_pool.cc:224] Handling call
0709 13:04:30.142963 (+1894986us) spinlock_profiling.cc:243] Waited 6.74 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:04:30.143192 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.143389 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.143417 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:30.143463 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:30.143496 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.143578 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.143693 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15491456}
I20260709 13:04:30.295687 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:30.246815 15547 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:29.958354 15481 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.955394 15576 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:30.090039 15492 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:30.268738 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.020152 15593 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:29.911125 15582 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:29.949267 15523 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:30.309244 13989 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:30.269975 13945 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.008s	sys 0.009s
W20260709 13:04:30.279366 13838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.005s	sys 0.005s
W20260709 13:04:30.280453 14524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.019s	sys 0.011s
I20260709 13:04:30.245733 15527 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:30.310356 14524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.019s	sys 0.011s
I20260709 13:04:30.312565 13868 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:30.313339 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:30.279011 14021 rpcz_store.cc:276] 0709 13:04:28.181630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.181752 (+   122us) service_pool.cc:224] Handling call
0709 13:04:29.950843 (+1769091us) spinlock_profiling.cc:243] Waited 80.1 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:04:29.951095 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.951324 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.951362 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.951418 (+    56us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:29.951448 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.951509 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.951641 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184275328}
I20260709 13:04:30.220361 15566 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.286566 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:29.980616 15435 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:30.254518 15513 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:30.133663 15556 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:30.253588 15539 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:30.303344 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:30.161125 14018 rpcz_store.cc:276] 0709 13:04:28.160149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.160296 (+   147us) service_pool.cc:224] Handling call
0709 13:04:29.969252 (+1808956us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:04:29.969506 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.969725 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.969755 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.969804 (+    49us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:29.969839 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.969898 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.970037 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65488256}
I20260709 13:04:30.245118 15575 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:30.310755 13987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.004s	sys 0.013s
W20260709 13:04:30.341332 13987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.004s	sys 0.013s
I20260709 13:04:30.294772 15387 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:30.316154 13838 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:30.325310 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 1965) took 1440 ms. Trace:
I20260709 13:04:30.346460 13866 rpcz_store.cc:276] 0709 13:04:28.884757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.884898 (+   141us) service_pool.cc:224] Handling call
0709 13:04:28.885195 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.116528 (+231333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.116561 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.116611 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:29.116645 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.324936 (+1208291us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54003ac0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:30.325007 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.325219 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":531468672}
I20260709 13:04:29.957798 15595 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.454s
I20260709 13:04:30.302668 15580 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:30.350684 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.331306 13989 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:30.361766 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:30.362736 14561 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:30.302776 15560 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:30.337925 13913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.025s	sys 0.054s
I20260709 13:04:30.294167 15353 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:30.314687 15527 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.352154 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:30.366086 13913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.026s	sys 0.054s
I20260709 13:04:30.366780 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.358974 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.335836 15523 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:30.368216 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.448948 14022 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:30.449302 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.450605 15508 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:30.451234 15508 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:30.453408 15508 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:30.401228 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.387897 13872 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 2: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:30.415756 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.455188 13870 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 1.
W20260709 13:04:30.368247 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52166.3us
I20260709 13:04:30.401904 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.394831 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.323160 15601 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:30.460268 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:30.386474 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55310.4us
I20260709 13:04:30.470497 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.468067 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:30.473276 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.476271 13838 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:30.474195 13945 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:30.476850 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.477979 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.478753 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.479571 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.480834 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.486874 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.488976 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.417786 15603 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:30.394774 15546 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:30.457381 15591 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:30.493279 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.352002 15435 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:30.354123 15605 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:30.388590 15430 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:30.491374 15399 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:30.483917 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:30.497934 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.356642 15436 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:30.464277 15601 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:30.465562 15528 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:30.502890 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.507292 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.549041 13838 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:30.588551 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.509654 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.477620 15602 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:30.471674 15493 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:30.505291 13988 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.636s (SENT)
I20260709 13:04:30.572966 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.495298 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.589913 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.496767 15603 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:30.520373 15488 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:30.602162 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:30.599810 13866 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:04:30.515527 15430 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:30.506006 15605 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:30.575587 15460 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:30.601133 15493 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:30.518567 15424 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:30.619995 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:30.557732 15399 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:30.610270 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.621184 15399 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.621289 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:30.622831 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 1839) took 1402 ms. Trace:
I20260709 13:04:30.611199 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.602468 15593 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:30.608901 15430 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:30.631142 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.507911 15546 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:30.595463 15582 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:30.626852 13945 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:30.580427 15601 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.605139 15575 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:30.601691 15603 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:30.602156 15547 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:30.603053 15488 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:30.599050 15602 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:30.601205 15528 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:30.871471 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 365974us
I20260709 13:04:30.653014 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.877840 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.879227 13950 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:30.868176 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:30.881177 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:30.623240 13868 rpcz_store.cc:276] 0709 13:04:29.220303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.220486 (+   183us) service_pool.cc:224] Handling call
0709 13:04:29.220727 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.324257 (+1103530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.622608 (+298351us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002e86d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:30.622774 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2510359936}
I20260709 13:04:30.882694 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:30.885732 13838 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:30.889590 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:30.891217 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:30.896601 13942 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.892624 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.893419 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.885746 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:30.899946 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.895320 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.903348 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.901732 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.905404 13946 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:30.889839 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:30.664891 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.665266 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.664724 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.930886 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:30.932842 13941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 182) took 1755 ms (client timeout 2222 ms). Trace:
W20260709 13:04:30.933534 13941 rpcz_store.cc:269] 0709 13:04:29.176823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.177077 (+   254us) service_pool.cc:224] Handling call
0709 13:04:30.932321 (+1755244us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b5400087d54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:30.932597 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1289593088}
I20260709 13:04:30.937590 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 169) took 1260 ms. Trace:
I20260709 13:04:30.935386 13944 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:04:30.939584 13872 rpcz_store.cc:276] 0709 13:04:29.676942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.677128 (+   186us) service_pool.cc:224] Handling call
0709 13:04:30.937007 (+1259879us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b54000daa54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:30.937318 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1624116608}
I20260709 13:04:30.668471 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.938500 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1088) took 1452 ms. Trace:
I20260709 13:04:30.669273 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.914001 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.941294 14015 rpcz_store.cc:276] 0709 13:04:29.485879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.489548 (+  3669us) service_pool.cc:224] Handling call
0709 13:04:29.490024 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.365959 (+875935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.365989 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:30.366035 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:30.366091 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.937914 (+571823us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b54002ebb54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:30.937983 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.938269 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2013845376}
I20260709 13:04:30.945122 13773 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54072 (request call id 130) took 4003 ms. Trace:
I20260709 13:04:30.946960 13773 rpcz_store.cc:276] 0709 13:04:26.967738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.114863 (+147125us) service_pool.cc:224] Handling call
0709 13:04:30.944550 (+3829687us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:04:30.916278 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:30.922569 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:30.929276 13948 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:30.924072 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:30.959504 13946 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
W20260709 13:04:31.039520 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 154270us
I20260709 13:04:30.627481 15605 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:30.799667 15565 consensus_meta.cc:287] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.000s
I20260709 13:04:30.631711 15449 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:04:30.712380 15610 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:04:30.685241 15494 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:30.631132 15481 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:04:31.058478 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 141082us
I20260709 13:04:30.683729 15424 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.066226 15508 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.066601 15449 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:30.685369 15556 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.068178 15504 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:31.069464 15488 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.069999 15560 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.070844 15618 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:30.736433 15513 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:31.071729 15591 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:30.786664 15539 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:30.787902 15546 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:31.072216 15529 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.078544 15436 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.079344 15588 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:31.080263 15576 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:30.651655 15582 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:30.812366 15566 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:31.137010 15566 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.138571 13867 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:04:31.141606 15528 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.086623 15580 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:30.710172 15387 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:30.728757 15523 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.129702 15529 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.130661 15436 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.131644 15588 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.116875 15591 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:31.133533 15576 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.134452 15582 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.152380 15581 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:31.083259 15521 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.107206 15504 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:31.199156 15481 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.199666 15387 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:31.200727 15387 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.202754 13950 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:31.204200 15521 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.208807 15492 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.209585 15492 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.213379 15519 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.215202 13945 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:31.219764 15387 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.207978 15521 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:31.198135 14524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.019s	sys 0.034s
I20260709 13:04:31.184512 15591 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.794783 15527 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.193334 15504 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:30.631479 15460 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.248759 13947 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:31.245965 15494 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.247334 15565 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.267910 15565 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.268013 13873 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:31.255748 15588 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:31.256455 15435 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.257310 15529 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.259121 15576 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.253615 15436 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.264004 15493 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.201933 15481 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.276155 15624 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.279894 15581 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:31.284716 15601 consensus_meta.cc:287] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.597s	user 0.016s	sys 0.000s
I20260709 13:04:31.188524 15508 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.297786 13874 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:31.302918 15585 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:31.306205 15353 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.250276 15460 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.316291 15424 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.317842 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:31.326427 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 58317us
I20260709 13:04:31.287084 15492 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.349035 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 1981) took 1068 ms. Trace:
I20260709 13:04:31.356570 13867 rpcz_store.cc:276] 0709 13:04:30.280262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.314551 (+ 34289us) service_pool.cc:224] Handling call
0709 13:04:30.340050 (+ 25499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.139215 (+799165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.348496 (+209281us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54002e7cd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:31.348894 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1833813376}
I20260709 13:04:31.359499 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 1974) took 1222 ms. Trace:
I20260709 13:04:31.359946 13874 rpcz_store.cc:276] 0709 13:04:30.136794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.136911 (+   117us) service_pool.cc:224] Handling call
0709 13:04:30.137228 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.345947 (+1208719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.359170 (+ 13223us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540026b6d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:31.359415 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2655634304}
I20260709 13:04:31.349203 15584 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.366283 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 1970) took 1393 ms. Trace:
I20260709 13:04:31.367038 13873 rpcz_store.cc:276] 0709 13:04:29.973306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.973745 (+   439us) service_pool.cc:224] Handling call
0709 13:04:29.974038 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.268609 (+1294571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.365758 (+ 97149us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54004619d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:31.366142 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2974851840}
I20260709 13:04:31.382270 15509 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.383285 15509 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.383833 15511 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.384790 15511 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.383867 15359 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.386737 15359 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.387116 15449 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.354704 15527 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.304195 14554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 401) took 2048 ms. Trace:
W20260709 13:04:31.339262 13839 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.001s	sys 0.014s
W20260709 13:04:31.394330 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 76607.3us
I20260709 13:04:31.406663 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:31.403929 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:31.408030 14561 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:04:31.402983 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:31.392213 15509 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.409411 14560 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:31.403592 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:31.402037 14554 rpcz_store.cc:276] 0709 13:04:29.256010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.262175 (+  6165us) service_pool.cc:224] Handling call
0709 13:04:29.262590 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.797876 (+535286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.800209 (+  2333us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.800276 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:29.800324 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.303635 (+1503311us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54003372d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:31.303730 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.303946 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1206404352}
I20260709 13:04:31.395478 15424 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:31.402436 15489 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:31.398202 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:31.392623 15359 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.412693 15489 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:31.395433 15511 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.414464 15489 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.416383 15504 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:31.391178 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 505735us
I20260709 13:04:31.350263 15601 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:31.424753 14022 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:04:31.425580 15601 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:31.428367 13839 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.001s	sys 0.015s
I20260709 13:04:31.403429 15602 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.431902 15628 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.433216 15628 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.432128 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:31.430035 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 946127us
I20260709 13:04:31.437753 14555 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:04:31.442020 15628 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.443790 15494 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:31.449054 15430 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.796s	user 0.002s	sys 0.015s
I20260709 13:04:31.476368 15430 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.485374 15430 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.450183 15624 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:31.451576 13942 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.554s	user 0.012s	sys 0.005s
I20260709 13:04:31.454670 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:31.492507 15624 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.493225 15527 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:31.493746 15527 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:31.494161 15527 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.455229 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:31.503738 13914 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:31.504488 15556 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:31.504756 15602 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:31.505033 15556 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:31.505203 15602 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:31.505470 15556 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.505602 15602 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:31.506242 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 51175.3us
I20260709 13:04:31.507040 15628 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:31.507706 15628 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.506410 13914 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:31.456636 15603 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.458588 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:31.464382 15605 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.466392 13987 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:31.466976 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 898802us
I20260709 13:04:31.398752 15508 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.468380 15581 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.468588 15489 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:31.552318 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 129037us
W20260709 13:04:31.470149 15399 consensus_meta.cc:287] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.848s	user 0.007s	sys 0.007s
I20260709 13:04:31.512151 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:31.512538 15585 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:31.554486 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:31.555263 15624 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:31.555645 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 43704.1us
I20260709 13:04:31.556160 15584 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:31.556793 15584 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.559746 15584 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.514772 13942 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:31.567159 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:31.567734 15582 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.568900 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:31.571233 13949 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:31.572716 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 1834) took 1472 ms. Trace:
I20260709 13:04:31.573135 13949 rpcz_store.cc:276] 0709 13:04:30.100149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.934869 (+834720us) service_pool.cc:224] Handling call
0709 13:04:31.570800 (+635931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.571770 (+   970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.572497 (+   727us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b5400092750. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:31.572655 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1461453056}
I20260709 13:04:31.556834 15547 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:31.577560 15508 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:31.577901 15489 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.578128 15508 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.578624 15508 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:31.519837 15603 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:31.520885 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:31.522432 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:31.524245 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:31.525022 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:31.526782 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:31.527491 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:31.534497 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:31.534248 15591 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:31.512189 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 53674.7us
I20260709 13:04:31.540395 15556 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.490432 15488 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.469391 15504 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.533360 15527 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.553758 15399 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.513091 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:31.509994 15628 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:31.579396 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 73054.9us
I20260709 13:04:31.582463 13941 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 1.
I20260709 13:04:31.583360 13943 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:31.593762 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:31.596320 13914 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:31.597303 15527 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.597479 15504 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:31.598110 15504 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.599396 15399 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:31.600358 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:31.600955 15504 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.600847 13773 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 reported cstate change: config changed from index 4 to 7, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:31.602537 15606 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:31.602824 13838 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:31.603109 15606 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.603664 15606 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.602968 13773 catalog_manager.cc:5697] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 reported cstate change: config changed from index 4 to 5, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:31.586370 15624 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:31.612923 15504 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:31.614090 15591 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.585225 15581 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:31.615095 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:31.621935 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:31.622850 15585 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:31.622934 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:31.623176 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 68783us
I20260709 13:04:31.625084 15581 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.627228 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:31.628029 15603 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:31.628422 15581 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.629595 13987 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:31.630194 15504 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:31.630450 15617 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:31.633175 13838 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:31.635623 15601 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:31.636797 15601 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:31.638950 15601 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.638800 15624 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:31.624230 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:31.637001 15588 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:31.646297 15455 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:31.646420 15588 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.646838 15455 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:31.647025 15588 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.647274 15455 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.625375 15585 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:31.640573 13773 catalog_manager.cc:5697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:04:31.624400 15582 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.631108 15617 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.657292 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.639984 15603 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:31.648370 15494 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:31.648059 15489 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:31.669795 15489 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:31.670295 15489 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.662485 13874 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:31.674450 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.665977 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:31.665072 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:31.631408 15504 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.654731 15585 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:31.675539 13945 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 2: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:31.665072 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.670346 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:31.680744 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:31.674573 15617 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:31.693240 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51757.9us
W20260709 13:04:31.694736 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 65263.6us
I20260709 13:04:31.693758 13950 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
W20260709 13:04:31.696134 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 55609.5us
I20260709 13:04:31.668398 15424 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:31.697623 13873 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:04:31.702180 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.662989 15602 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.708467 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.696099 15603 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.712682 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:31.713706 14555 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.712903 15603 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:31.721685 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.700011 15424 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:31.707864 15488 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.708149 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:31.706845 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 65707.3us
I20260709 13:04:31.733129 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:31.736331 13945 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.736624 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.725965 13987 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:31.717674 15605 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.724799 15556 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.772538 14555 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:31.775218 15424 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.782106 15603 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:31.785843 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 66400us
I20260709 13:04:31.791148 13914 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:04:31.793579 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 92667.3us
I20260709 13:04:31.726333 15602 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:31.810240 13871 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:31.813611 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.789848 15424 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.819113 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:31.836923 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:31.827230 15492 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:31.816671 15588 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.820086 15489 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.847870 15492 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:31.855257 13869 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:31.852353 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.860041 15492 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:31.861443 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46419us
I20260709 13:04:31.862900 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:31.860674 13868 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:04:31.866509 15608 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:31.866936 15608 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:31.867307 15608 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.832479 15601 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.880685 15606 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.847915 15603 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:31.886729 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:31.888794 13872 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:31.891459 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.883430 15605 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:31.902295 13987 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:31.914821 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.918692 13868 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:31.920694 15628 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:31.921653 15584 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:31.921948 15628 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:31.922394 15628 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:31.922667 15511 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:31.923063 15511 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:31.923220 15628 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.924221 15511 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:31.903436 15605 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:31.933530 15603 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:31.931145 13945 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:31.937211 13868 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:31.932417 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45100.6us
I20260709 13:04:31.941154 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:31.924934 15628 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.945672 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.942680 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:31.948714 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.940047 15605 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.917892 15508 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:31.953104 15602 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:04:31.956133 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:31.943189 15511 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.959327 13838 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:31.959826 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:31.961612 13945 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 2: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:31.959517 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.957760 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:31.963873 13871 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:31.954787 15588 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:31.968849 13872 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:31.965453 15631 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.953230 15603 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:31.971856 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.972504 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.975502 13987 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:31.967438 15617 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:04:31.980609 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.987277 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.992107 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:31.997339 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:31.998245 13872 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 2: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:31.985198 15489 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:31.985776 15617 raft_consensus.cc:697] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:31.990458 15606 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:32.010133 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.011333 13873 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:31.988454 15601 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:32.001446 15556 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:32.005928 15591 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:32.014565 15591 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.016767 15591 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:32.012117 15603 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:32.023605 15556 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.026069 15556 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:32.026381 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.027653 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.028046 14555 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:32.031888 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 65964.3us
I20260709 13:04:32.033990 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.040751 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.050904 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.052600 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.055280 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.071487 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.067282 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.084611 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:32.090287 13867 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:32.089861 14556 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 2: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:32.092429 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.093438 13875 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 2: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:32.094458 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:32.097975 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.104662 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.032455 15455 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:32.122272 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.123430 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:32.124396 14555 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:32.129918 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:32.130527 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 176205us
I20260709 13:04:32.134028 14555 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:04:32.136860 13914 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:32.112103 15508 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:32.125567 15605 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:32.146669 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 73928.3us
I20260709 13:04:32.153381 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.144680 15606 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:32.150136 15628 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:32.156059 15606 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:32.145298 15608 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.148756 15399 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:04:32.169530 15494 consensus_queue.cc:1059] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:04:32.137620 15617 consensus_queue.cc:237] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.178823 13987 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:32.182899 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.183627 14555 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 2: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:04:32.193715 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45778.1us
I20260709 13:04:32.165558 15628 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.203063 15605 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:32.204399 15455 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:32.208691 15631 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:32.211774 15628 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:32.213286 15608 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:32.216367 15640 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:32.215878 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:04:32.219467 15585 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.008s
I20260709 13:04:32.220233 15399 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:32.220721 15399 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:32.220115 15585 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.221113 15399 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:32.222172 14561 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:32.198386 14555 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:32.235380 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.166159 15606 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:32.238713 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 74325.5us
I20260709 13:04:32.239960 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:32.236088 15494 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:04:32.243454 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.237551 15608 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:04:32.242540 15640 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:04:32.250741 15640 raft_consensus.cc:3055] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:32.251152 15640 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:32.252035 15640 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.253294 15640 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.248704 15603 raft_consensus.cc:3055] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:32.247009 15608 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:32.244095 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:32.254936 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 86399.7us
I20260709 13:04:32.268600 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.269483 14555 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:32.268896 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:32.256481 15603 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:32.272444 15603 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.273836 15603 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.259759 15608 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:32.283241 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.276252 15602 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:32.267256 15591 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:32.284241 15359 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:32.291316 15359 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:32.291762 15359 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.291965 15585 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:32.286418 15602 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:32.292789 15602 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:32.294085 15591 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.296749 15591 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:32.299855 15608 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.301492 15608 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:32.303740 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 49306.6us
I20260709 13:04:32.270690 15399 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.305920 15399 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:32.306718 15399 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:32.309504 15399 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:32.311719 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:32.312371 13946 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:32.312811 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:32.322875 15436 raft_consensus.cc:3055] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:32.323357 15436 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:32.324188 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:32.324312 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.321172 13946 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:04:32.326649 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.327538 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:32.328150 13945 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:32.324081 13987 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:32.317940 15601 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:32.309887 15399 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:32.357347 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.361270 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:32.366565 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:32.376730 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.381994 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:32.378952 14555 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:32.386531 13941 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:32.386132 15602 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.389322 15602 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:32.389529 13942 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 4: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:32.357720 15601 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:32.398451 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:32.400825 13949 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:32.399150 13945 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:32.402148 14555 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:32.403132 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:32.403712 15359 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.404506 15648 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:04:32.403162 14561 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:32.376525 15399 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.413822 15399 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.419533 15359 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:32.419756 15645 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:32.422040 15645 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:32.422891 15645 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.429576 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.324292 15436 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:32.429908 15646 log.cc:927] Time spent T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:32.431041 15436 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.363795 15424 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:04:32.436365 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:32.438850 15648 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:32.439195 15624 raft_consensus.cc:3055] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:32.439337 15648 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:32.439749 15624 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:32.440455 15648 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.440907 15624 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.442412 15648 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.449822 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:32.450459 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.450148 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:32.451289 14556 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:32.450825 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.451104 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.452054 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:32.452342 14558 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:32.453688 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:32.454478 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.455735 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.456578 14560 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 2: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:32.458738 14558 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:32.460896 15624 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.455101 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.395655 15492 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:32.466480 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 57655.6us
I20260709 13:04:32.464975 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.401661 15602 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:32.462646 15649 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:04:32.478252 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:32.478646 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 153954us
I20260709 13:04:32.478785 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.479413 14555 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:32.480738 13838 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:32.482292 15504 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:32.484916 13869 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:04:32.478243 15644 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:04:32.486387 13871 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:32.475054 15637 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.482474 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:32.491925 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.492611 15584 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:04:32.493777 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.493252 15584 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.495569 15639 raft_consensus.cc:3055] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:32.495883 14553 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:32.495965 15639 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:32.497133 15584 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:32.497071 15639 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.499854 15639 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.503585 15617 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:32.504225 15617 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:32.505229 15617 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.507956 15601 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.509492 15601 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:32.521586 15488 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:32.522668 15488 consensus_queue.cc:1243] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:32.523391 15504 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:32.523777 15504 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:32.524721 15504 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:32.529341 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 61363.3us
I20260709 13:04:32.530889 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:32.559685 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.553124 15637 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:32.547835 15584 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:04:32.563390 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 72967.1us
I20260709 13:04:32.565066 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.567291 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.568593 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.564280 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:32.574214 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.576205 14555 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:32.577811 13949 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:32.578465 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:32.579824 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:32.581146 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.581828 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:32.582695 15504 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:32.583465 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.583374 15651 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:32.583796 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.584151 15651 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:32.584537 15651 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:32.585529 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.591297 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.593762 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.603843 13941 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:32.614701 15645 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.618870 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.621376 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.622442 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.623751 14555 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 2: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:32.625437 15492 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:32.629802 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.629913 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.631881 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:32.625711 13773 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57626 (request call id 241) took 1059 ms. Trace:
I20260709 13:04:32.629678 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.639178 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.640061 13869 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 2: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:32.640241 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:32.641167 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.641798 13871 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.644165 15488 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:32.649169 13870 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:32.637375 13773 rpcz_store.cc:276] 0709 13:04:31.565983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.566232 (+   249us) service_pool.cc:224] Handling call
0709 13:04:32.625621 (+1059389us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:31.924842 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:31.960911 (+ 36069us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:31.960962 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:04:31.960989 (+    27us) tablet.cc:662] Decoding operations
0709 13:04:32.001377 (+ 40388us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:32.003438 (+  2061us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:32.003476 (+    38us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:04:32.003998 (+   522us) tablet.cc:701] Row locks acquired
0709 13:04:32.004050 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:04:32.004249 (+   199us) write_op.cc:270] Start()
0709 13:04:32.004417 (+   168us) write_op.cc:276] Timestamp: P: 1783602272004184 usec, L: 0
0709 13:04:32.004455 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:04:32.005275 (+   820us) log.cc:844] Serialized 1433 byte log entry
0709 13:04:32.326195 (+320920us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b540012b220. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33a8dbe 00007fb2c33b3930 00007fb2c33b38d8 00007fb2c33b3858 00007fb2c33b3813 00007fb2c33b2a9c 00007fb2c929e626 00007fb2c9298dd8 00007fb2c33ef7cb 00007fb2c33f22c0
0709 13:04:32.353201 (+ 27006us) op_driver.cc:464] REPLICATION: finished
0709 13:04:32.497452 (+144251us) write_op.cc:301] APPLY: starting
0709 13:04:32.497648 (+   196us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:32.526192 (+ 28544us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:32.526233 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:32.618611 (+ 92378us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:32.618991 (+   380us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:04:32.619092 (+   101us) write_op.cc:312] APPLY: finished
0709 13:04:32.619534 (+   442us) log.cc:844] Serialized 35 byte log entry
0709 13:04:32.619808 (+   274us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.619955 (+   147us) write_op.cc:454] Released schema lock
0709 13:04:32.620041 (+    86us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:32.620541 (+   500us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.620578 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"spinlock_wait_cycles":526592,"child_traces":[["op",{"apply.queue_time_us":143950,"apply.run_cpu_time_us":3939,"apply.run_wall_time_us":123075,"num_ops":3,"prepare.queue_time_us":141472,"prepare.run_cpu_time_us":30089,"prepare.run_wall_time_us":275954,"raft.queue_time_us":25848,"raft.run_cpu_time_us":14781,"raft.run_wall_time_us":57697,"replication_time_us":348515,"spinlock_wait_cycles":2914560,"thread_start_us":308417,"threads_started":4,"wal-append.queue_time_us":185160}]]}
I20260709 13:04:32.654166 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.654512 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.655025 13869 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:32.657632 15602 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:32.671705 13869 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:32.674396 13946 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:32.676952 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:32.681506 13941 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:32.685487 13772 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57642 (request call id 266) took 1044 ms. Trace:
I20260709 13:04:32.686321 13869 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:32.662153 15645 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:32.687983 15582 consensus_queue.cc:1059] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:32.692615 13839 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:32.693358 15633 consensus_queue.cc:1059] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:32.685956 13772 rpcz_store.cc:276] 0709 13:04:31.626674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.626941 (+   267us) service_pool.cc:224] Handling call
0709 13:04:32.671077 (+1044136us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":7}
I20260709 13:04:32.684636 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.694016 13839 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:32.695683 13988 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:32.696610 15399 consensus_queue.cc:1059] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:04:32.696861 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.697191 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.697510 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:32.707327 13988 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:32.708312 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:32.708612 13946 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:32.712080 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:32.712867 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:32.720608 13941 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:32.720463 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:32.721161 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:32.726889 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:32.727087 13950 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:32.696249 15591 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:32.694504 15633 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:32.719947 15624 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:32.737766 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:32.738241 15399 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:32.739398 13988 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:32.746155 13948 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:32.742880 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:32.745584 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:32.736138 15633 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:32.696455 15603 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:04:32.748473 15647 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:32.705046 15601 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:04:32.727759 15556 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:32.734587 15617 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:04:32.750329 15424 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.753750 13946 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:32.755362 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:32.756472 15651 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.808748 15651 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:32.809303 15455 consensus_queue.cc:1059] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:32.813843 13867 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:32.816421 15631 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:32.816892 15631 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:32.817850 15631 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.819146 15631 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:32.823983 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 104857us
I20260709 13:04:32.830708 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:32.746996 13871 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:32.770411 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:04:32.843228 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 72911.1us
I20260709 13:04:32.770467 13949 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:32.771713 13870 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:32.769591 15635 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:32.725901 15645 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:32.773070 15624 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:32.847956 14554 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:32.848441 15602 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:32.848080 15624 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.774013 15653 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:32.774158 15608 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:32.849617 15651 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:04:32.774892 13772 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:32.850070 15653 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:32.851155 15603 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:32.851590 15603 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:32.851332 15653 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.852619 15603 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.853221 15653 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.853773 15603 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.850229 15608 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:32.775189 15633 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.861114 15633 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:32.866281 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 63647.2us
I20260709 13:04:32.777704 15488 raft_consensus.cc:3055] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:32.778759 15648 raft_consensus.cc:3055] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:32.874094 15488 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:32.788179 15640 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:32.874389 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:32.875247 14023 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:32.874895 15640 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.875276 15488 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.877363 15640 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:32.788471 15601 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:32.759006 15399 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.878180 15601 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.879988 15399 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:32.789573 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:32.790350 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:32.881718 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.790437 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:32.882676 15624 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:32.790563 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:32.880290 15601 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:32.846704 13872 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:32.874190 15648 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:32.884545 15488 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.884800 15648 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.885896 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.886541 15648 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:32.892805 15601 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:32.894202 15601 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:32.894568 15601 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:32.895949 15601 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:32.897344 15601 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.880280 15608 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:32.789605 13869 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:32.900110 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 189343us
W20260709 13:04:32.901172 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.906559 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.924778 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:32.927628 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:32.938759 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:32.927029 15424 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:32.939793 14017 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:32.949218 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:32.950162 15488 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:32.950335 14016 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:32.951254 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:32.953137 14019 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
W20260709 13:04:32.963706 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 58693.4us
I20260709 13:04:32.964665 13839 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:32.972406 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:32.973040 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:32.974774 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:32.975539 14017 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:32.980209 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:32.980988 14019 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:32.968519 15608 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:32.992928 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:32.993268 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.983793 13759 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 545fc4aeb71d491985e296a0721d905d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d1b4fd04200247d99ba9ad1fbeb42b42 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:32.997068 14015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:32.997942 14015 raft_consensus.cc:606] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:32.998369 14015 raft_consensus.cc:614] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:32.988893 15591 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:32.999933 14018 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:32.979260 15488 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:33.009310 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.010386 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.010614 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.011088 14554 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:33.012405 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.011137 14021 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:04:33.013192 14022 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:33.011265 14017 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:33.015380 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.016145 14023 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:33.019218 13949 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:33.019750 14016 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 2: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:33.021783 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.022253 15492 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:33.022668 14019 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:04:33.028573 14017 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:33.036864 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 64013us
I20260709 13:04:33.032425 15489 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:04:33.038681 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.039569 14022 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:33.041716 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:33.044535 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 50986.1us
I20260709 13:04:33.046918 13869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:33.048810 13869 raft_consensus.cc:606] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:33.048736 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.049307 13869 raft_consensus.cc:614] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:33.049536 14016 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:33.050733 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.052591 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:33.051578 14019 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:33.065693 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:33.070915 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:33.071044 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.071204 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.084237 15492 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.084898 15492 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.088155 15492 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:33.088029 13869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:33.088883 13869 raft_consensus.cc:606] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:33.066828 15504 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:33.070286 15635 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:33.120906 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:33.116291 13771 catalog_manager.cc:5697] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 3, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:33.133607 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.118873 15633 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:33.118166 15504 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:33.136546 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.156345 14019 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:33.146592 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.157387 14024 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:04:33.147205 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.134887 13771 catalog_manager.cc:5697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 3, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.147953 14023 raft_consensus.cc:3055] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:33.148514 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.159487 14021 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:33.148964 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.149652 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.160398 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.147967 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.150671 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.154968 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.161911 14015 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:33.158776 14023 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:33.162889 13771 catalog_manager.cc:5697] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.167626 14016 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:04:33.167721 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:33.172976 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.126770 15635 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:33.161079 14019 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
W20260709 13:04:33.243774 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 172883us
I20260709 13:04:33.245368 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:33.247017 13946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:33.247946 13946 raft_consensus.cc:606] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:33.158321 15504 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.249899 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.253553 14019 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:33.254681 13874 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:33.255990 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.256686 14019 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:04:33.264026 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 150363us
I20260709 13:04:33.270941 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:33.272135 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.283317 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:33.284865 13871 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:33.286589 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.287257 14024 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:33.262468 15662 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.281857 15635 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:33.302498 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.303478 14016 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:04:33.291283 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.295235 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.296135 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.304906 14024 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:04:33.305023 14019 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
W20260709 13:04:33.306445 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.305938 14022 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:33.312139 14018 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:33.304390 14020 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:33.308337 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.332962 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.284541 14023 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.314332 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.314985 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.315634 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.321703 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.323781 13988 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:33.328316 15488 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:33.335405 14016 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:33.333758 14024 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:33.339079 14019 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 2: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:33.342648 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.343400 14024 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:33.344281 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.345855 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.347568 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.347831 14023 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.348482 14022 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.348671 14024 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:33.352793 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.353516 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.355312 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:33.356487 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.357652 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.358743 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.368606 14020 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:33.369975 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.373035 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:33.365757 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.375746 14017 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.384016 15662 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:33.384727 15662 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.390455 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.393188 14017 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:33.379098 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.398722 14022 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:33.399952 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.400797 14022 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 2: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:33.401605 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.402352 15608 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.404980 14015 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:33.406241 14015 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:33.408402 14015 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:33.415277 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:33.415164 14021 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:33.403736 15662 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:33.417412 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:33.419785 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 47577.1us
W20260709 13:04:33.420523 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 47569.7us
I20260709 13:04:33.436010 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.422326 14020 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:33.424638 15633 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:33.426714 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:33.443030 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:33.443817 13945 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
W20260709 13:04:33.430671 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 175388us
W20260709 13:04:33.454800 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.449990 15633 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:33.465867 15633 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.469722 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.470976 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:33.471813 14016 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
W20260709 13:04:33.473362 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.479972 15492 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:33.487485 15670 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:33.490813 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.494243 14016 raft_consensus.cc:2376] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 2. Re-sending same reply.
I20260709 13:04:33.430884 15504 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.496027 15670 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:33.506804 14020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fdef734205364543b02ed4158216bf36"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:04:33.507555 14020 raft_consensus.cc:606] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:04:33.507903 14020 raft_consensus.cc:614] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:33.501088 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.472981 15662 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:04:33.514293 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.468487 15631 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:04:33.516645 14023 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:33.488526 15424 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:33.468192 15455 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:04:33.526643 14024 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:33.526605 14019 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:33.531529 13871 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:33.529155 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.536242 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:33.540362 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:33.560071 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.569676 14020 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:04:33.570398 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.563481 14019 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:33.563423 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.573514 14022 raft_consensus.cc:2376] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 3. Re-sending same reply.
I20260709 13:04:33.564110 13945 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:33.564262 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:33.574636 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.565069 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.565574 14015 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:33.561787 13915 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:33.565702 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.566879 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:33.566466 15670 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.567883 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:33.563000 14016 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:33.543962 15424 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:33.582578 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:33.605147 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:33.576046 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 105144us
I20260709 13:04:33.596724 15601 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:33.591230 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:33.609665 15662 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:33.610201 15662 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:33.611167 15662 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.612563 15662 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.614678 15489 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:33.585021 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:33.616022 15608 raft_consensus.cc:3055] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:33.616430 15608 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:33.587966 15606 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:33.617650 15608 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.607553 13840 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:33.618497 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:33.624055 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:33.618831 15674 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:04:33.627489 15424 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:33.630172 15635 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:33.630700 15635 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:33.631150 15635 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.632254 15633 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.633499 14555 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:33.637761 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:33.637861 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.638882 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.639142 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.646278 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:33.646318 13988 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:33.647148 13869 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:33.652980 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:33.619581 15436 consensus_queue.cc:1059] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:04:33.619031 15608 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.656702 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.617986 15504 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:33.618676 15508 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:33.660272 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:33.655964 14022 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:33.654558 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.654112 14018 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:33.658715 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:33.663635 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.663662 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.658463 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:33.665871 15631 raft_consensus.cc:3055] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:33.666348 15631 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:33.667352 15631 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.668722 15631 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.674927 15633 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:33.651568 15617 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:33.676213 15617 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:33.676291 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:33.676681 15617 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:33.681617 15673 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.683522 13840 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:33.684186 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:33.684327 15673 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:33.689607 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.686743 15492 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:33.690356 15492 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:33.657886 15602 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:33.690951 15602 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:33.691318 15602 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.691262 15492 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:33.692628 15492 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.661520 15667 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.702941 15628 raft_consensus.cc:3055] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:33.703512 15628 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:33.702833 13773 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 5 to 6, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 6 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.704378 15628 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:33.705585 15628 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.711304 15674 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:33.717272 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.718029 15508 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.718618 15508 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:33.717279 15680 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:33.720741 15508 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:33.688556 14015 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:33.654718 15424 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:33.724376 15667 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:33.726866 15674 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:33.730691 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.731559 14020 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:33.731866 14022 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:33.731590 14018 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:33.732125 14015 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:33.730654 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.733198 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:33.733935 14019 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:33.734458 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:33.685858 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:33.741195 14023 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
W20260709 13:04:33.745137 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 59356.6us
I20260709 13:04:33.746033 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:33.751361 13840 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:33.744580 15633 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.752622 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:33.751430 14024 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:04:33.752872 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.758107 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:33.758476 15591 consensus_queue.cc:1059] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:33.759572 15591 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:33.760005 15591 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:33.760937 15591 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:33.753465 15488 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:33.763093 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:33.763440 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 125766us
I20260709 13:04:33.765532 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.765770 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:33.766490 15673 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:33.766863 15668 raft_consensus.cc:3055] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:33.767325 15668 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:33.757138 15582 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:33.752498 15424 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.763531 15492 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:04:33.755384 15489 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:33.769295 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:33.774088 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.762668 15601 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.762794 15504 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:33.774991 15653 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:33.779733 15556 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:33.780337 15556 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:33.780586 15508 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:33.780982 15508 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:33.781878 15508 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.776698 15489 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:33.783388 15508 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.783988 15489 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.774883 15582 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.781349 15556 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:33.768289 15668 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:33.792286 15668 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.804116 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:33.805212 15674 raft_consensus.cc:2764] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:04:33.807549 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:33.808315 15436 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:33.808763 15436 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:33.809113 15436 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.808315 13772 catalog_manager.cc:5697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 1 to 2, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 2 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:33.813604 13773 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57642 (request call id 267) took 1011 ms. Trace:
I20260709 13:04:33.814023 13773 rpcz_store.cc:276] 0709 13:04:32.802665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.802806 (+   141us) service_pool.cc:224] Handling call
0709 13:04:32.810859 (+  8053us) spinlock_profiling.cc:243] Waited 5.88 ms on lock 0x7b5400002490. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030f0b2 00007fb2c9288a89 00007fb2c9220da6 00007fb2c91f2088 00007fb2c92f037e 00007fb2c77b12a4 00007fb2c77b1232 00007fb2c77b1161 00007fb2c77b10dc
0709 13:04:33.813529 (+1002670us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:33.718404 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:33.718992 (+   588us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:33.719034 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:04:33.719052 (+    18us) tablet.cc:662] Decoding operations
0709 13:04:33.719447 (+   395us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:33.719511 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:33.719536 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:33.719729 (+   193us) tablet.cc:701] Row locks acquired
0709 13:04:33.719747 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:04:33.719904 (+   157us) write_op.cc:270] Start()
0709 13:04:33.720020 (+   116us) write_op.cc:276] Timestamp: P: 1783602273719860 usec, L: 0
0709 13:04:33.720041 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:04:33.720714 (+   673us) log.cc:844] Serialized 536 byte log entry
0709 13:04:33.722821 (+  2107us) op_driver.cc:464] REPLICATION: finished
0709 13:04:33.765867 (+ 43046us) write_op.cc:301] APPLY: starting
0709 13:04:33.768685 (+  2818us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:33.777170 (+  8485us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:33.787668 (+ 10498us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:33.787990 (+   322us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:33.788163 (+   173us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:04:33.788255 (+    92us) write_op.cc:312] APPLY: finished
0709 13:04:33.788475 (+   220us) log.cc:844] Serialized 23 byte log entry
0709 13:04:33.788676 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:33.788775 (+    99us) write_op.cc:454] Released schema lock
0709 13:04:33.788820 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:33.789230 (+   410us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:33.789262 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3712,"reported_tablets":4,"spinlock_wait_cycles":13521664,"child_traces":[["op",{"apply.queue_time_us":32614,"apply.run_cpu_time_us":1971,"apply.run_wall_time_us":24436,"mutex_wait_us":28233,"num_ops":1,"prepare.queue_time_us":11812,"prepare.run_cpu_time_us":2602,"prepare.run_wall_time_us":2602,"replication_time_us":2634}]]}
I20260709 13:04:33.819444 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:33.820305 15674 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:33.820691 15674 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:33.821048 15674 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:33.824824 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47731.6us
I20260709 13:04:33.831928 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:33.832259 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.835947 13987 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:33.844244 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.824470 15653 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.842957 15601 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:33.841295 15608 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:33.846813 15606 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:33.865355 15436 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:33.850023 15639 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:33.844096 15488 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:33.883157 13950 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:33.885637 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 49809us
I20260709 13:04:33.886868 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.888393 13987 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.894497 15631 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:33.894929 15687 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.899729 15488 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.904971 15635 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:33.905700 15662 raft_consensus.cc:3055] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:33.906169 15662 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:33.906232 13947 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:33.907163 15662 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.908419 15662 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:33.910136 15685 consensus_queue.cc:1059] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:33.913120 15635 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:33.925228 15556 raft_consensus.cc:3055] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:33.925800 15556 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:33.927256 15556 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:33.929821 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 107366us
I20260709 13:04:33.919993 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.933974 13773 catalog_manager.cc:5697] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.936957 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.901480 15601 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.938541 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:33.941542 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:33.942791 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:33.939042 15687 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:33.939266 15608 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.950783 15683 raft_consensus.cc:2764] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:04:33.953745 15683 raft_consensus.cc:993] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:33.954257 15683 raft_consensus.cc:1081] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:33.957252 15653 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.936329 15617 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:33.959175 15424 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:33.959803 15424 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:33.960829 15424 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:33.962244 15436 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:33.962389 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:33.963261 14555 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:33.967037 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:33.968850 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:33.970858 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:33.974272 13947 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:33.976827 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:33.977128 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:33.978370 13838 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:33.978592 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.934504 15602 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:33.983690 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:33.975088 15617 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:33.984792 15667 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:33.979794 15653 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:33.961946 15674 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:33.997117 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.997562 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:33.989974 15556 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.999935 13987 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:33.988183 15689 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:33.994176 15436 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:34.004371 15674 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:33.999470 13942 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:34.000214 15608 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:34.009506 13759 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 545fc4aeb71d491985e296a0721d905d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d1b4fd04200247d99ba9ad1fbeb42b42 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:33.998476 15687 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:34.014429 15653 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:34.009223 15689 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:34.017769 15674 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:34.023121 15667 consensus_queue.cc:1059] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:34.024376 15602 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:34.025271 15692 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:34.031692 15667 raft_consensus.cc:3055] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:34.032182 15667 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:34.033121 15667 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.035048 15667 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:34.038214 15689 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:34.050367 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 55025.2us
W20260709 13:04:34.058543 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.059633 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:34.063570 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:34.065081 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:34.066920 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:34.067124 13840 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:34.068617 15591 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:34.069361 15668 raft_consensus.cc:993] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:34.069031 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:34.069949 15639 raft_consensus.cc:1081] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:34.070705 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:34.070959 15689 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.072981 13947 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:34.075032 15689 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:34.069108 15691 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:34.077265 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:34.077513 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:34.070758 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:34.083211 15662 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:34.083299 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:34.084195 15633 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:34.086424 15674 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:34.087074 15674 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:34.087594 15674 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:34.090940 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:34.093367 15662 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:34.094748 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:34.102437 15617 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:34.106185 15617 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:34.106818 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:34.095260 15689 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:34.095271 14524 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:34.103689 15504 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:34.115270 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:34.115150 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:34.107110 15617 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.122208 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:34.123715 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:34.091653 15508 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:34.125712 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.118494 15635 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:34.136042 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:34.136077 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:34.119546 15602 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:34.121032 15633 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:34.141568 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:34.129623 15691 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:34.143085 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:34.130839 15628 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:34.150007 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.146780 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:34.151961 14022 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:34.142444 14016 raft_consensus.cc:2376] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 2. Re-sending same reply.
I20260709 13:04:34.152283 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:34.137650 15662 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:34.118700 15673 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:04:34.153134 14023 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:34.137524 15639 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:04:34.170354 14020 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49628
I20260709 13:04:34.171168 14020 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:34.171627 14020 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.170876 13773 catalog_manager.cc:5697] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:34.154814 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:34.138047 15635 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.141284 15631 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:34.140237 15602 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:34.151332 15606 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:34.177655 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:34.146364 15689 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:34.156694 13987 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:34.177666 14024 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:34.149003 15628 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:34.187708 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:34.199537 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:34.218209 15436 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:34.203598 15628 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:34.184228 15606 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:34.190876 15696 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:34.207474 15591 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:34.155812 15670 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:04:34.225816 15684 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:34.191922 15662 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:34.179491 15635 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:34.218833 15582 raft_consensus.cc:3055] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:34.227907 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:34.209367 13840 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:34.182245 15602 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.243501 14020 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.305326 13947 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:34.193109 15631 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:04:34.323500 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 168749us
I20260709 13:04:34.234613 15556 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.240306 15633 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.338959 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 129721us
I20260709 13:04:34.344539 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:34.320773 15606 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.346045 14020 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:34.338406 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:34.350979 15679 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:34.357533 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:34.358546 13947 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.334421 15582 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:34.349323 15489 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:34.343679 15591 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:04:34.399446 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 171650us
I20260709 13:04:34.369175 13942 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.400794 15708 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.413169 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 74855.5us
I20260709 13:04:34.376777 15617 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:34.415728 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:34.368815 15582 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.385294 15689 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:34.349885 15556 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:34.354400 15633 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:34.403909 15508 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:34.417863 15700 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:34.420042 15697 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 13:04:34.419292 15694 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:04:34.406510 15708 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:34.437281 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:34.433740 15684 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:34.454613 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 46475us
I20260709 13:04:34.442951 15685 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:04:34.433251 15617 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:34.468883 14555 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:34.478415 14016 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.478633 13947 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:34.479579 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:34.481802 15685 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:34.482270 15685 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:34.483134 15685 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.483717 15674 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.484702 15685 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:34.486167 15674 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:34.486553 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:34.487727 15628 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.488443 13944 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:34.491506 15706 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:04:34.491595 15633 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:34.492130 15709 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:34.494733 15708 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:34.495788 15699 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.179s	user 0.005s	sys 0.000s
I20260709 13:04:34.499509 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:34.539080 13871 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:34.470450 15700 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:34.500249 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:34.506439 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:34.541409 14018 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:34.533185 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:34.510591 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:34.513525 14016 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49644
I20260709 13:04:34.546603 14016 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:34.546991 14016 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.547304 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:34.548002 13950 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:34.514084 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:34.514457 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:34.550498 14015 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:34.514911 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:34.436932 15631 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:34.517589 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:34.518769 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:34.521028 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:34.531908 13868 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.553822 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:34.554679 14021 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 2: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:34.549902 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:34.555408 13950 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:34.541280 14524 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:34.558005 13869 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:34.523923 15703 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.528178 15696 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:34.511675 15692 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:34.527716 15508 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.562903 14016 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.553345 13875 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:34.465173 15691 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 13:04:34.569816 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:34.542486 15679 raft_consensus.cc:697] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:34.583530 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:34.585342 13870 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:34.455190 15591 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.552805 15424 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:34.594971 14016 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:34.596328 15693 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:34.581565 15703 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:34.586158 15508 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.600958 15710 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:04:34.666662 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:34.579531 15692 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.651371 15608 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:34.571415 15689 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.650238 15679 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:34.654009 15606 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.606180 15617 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:34.637142 15708 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:34.628981 15693 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:34.666860 13950 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:34.633800 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.672279 13944 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:34.672279 13873 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:34.604048 15424 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.705832 14019 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:34.705205 13944 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:34.708726 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:34.629602 15696 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:34.714144 13950 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:34.719738 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 178527us
I20260709 13:04:34.673875 15639 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:34.728536 15709 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:34.698551 15556 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:34.729606 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 89384.3us
I20260709 13:04:34.700490 15708 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.736323 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:34.730294 13944 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.737699 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:34.740360 15708 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:34.697906 15608 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.747309 13942 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:34.747440 13875 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:34.749511 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:34.768863 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:34.667372 15635 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:34.771001 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:34.759563 13944 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.764060 13874 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:34.731235 15700 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.792927 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:34.747671 15670 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:34.761950 15608 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:34.731834 15668 consensus_queue.cc:1059] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:04:34.759615 15692 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:34.746675 15709 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:34.812117 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.813429 15424 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:34.814965 15606 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:34.748728 15556 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:04:34.815550 15724 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:04:34.747435 15628 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:34.821604 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.789325 15717 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:34.735095 15639 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:34.795017 15706 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:34.791008 15683 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:34.803203 15670 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.786751 15673 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
W20260709 13:04:34.836688 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 46998.7us
I20260709 13:04:34.748005 15716 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 13:04:34.835053 15710 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:34.835507 15683 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.843497 15683 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:34.842770 15710 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:34.847330 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:34.852077 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:34.861410 13875 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.861155 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:34.841455 15670 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:34.862624 13874 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.838475 15717 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:34.874128 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:34.850628 15703 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:34.874814 13866 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:34.875237 13866 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:34.875428 15639 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:34.874701 15717 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.876020 13866 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.877219 13866 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.869524 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:04:34.849478 15602 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.003s
I20260709 13:04:34.880038 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:34.881678 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:34.884403 14555 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 3: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:34.886744 13868 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:34.886816 13944 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:34.861937 15716 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:34.899924 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:34.751863 15696 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.891407 15508 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:34.851173 15698 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:04:34.946580 15693 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:34.947971 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 396781us
I20260709 13:04:34.889415 15602 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:34.987380 15602 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:34.954891 13866 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:34.954983 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:34.989001 13947 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:34.958488 15710 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:34.953648 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:34.886821 15608 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.965109 15606 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.966135 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.952447 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:34.966590 15732 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:04:34.794571 15700 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.016484 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:34.975035 13874 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:35.012948 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:34.901497 15703 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:34.941100 15692 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.988296 15709 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.012723 15608 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.030098 13949 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:35.025856 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:35.014576 15602 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.034937 13945 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:35.019507 15556 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.013423 15424 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:04:35.021821 13942 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:35.018844 15606 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:35.049736 15736 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:04:35.053649 15606 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:35.030587 15709 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.058367 15628 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.058967 15628 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.060688 15709 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:35.063174 14555 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:35.060907 15628 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:35.068672 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:35.063889 15700 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:35.074566 15718 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:35.075525 15608 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:35.075486 15718 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:35.079622 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:35.082181 13869 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:35.082371 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:35.045748 15556 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.083949 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:35.084033 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:35.085068 15556 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:35.085493 15693 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:35.086040 15693 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:35.086264 15662 consensus_meta.cc:287] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.749s	user 0.013s	sys 0.001s
I20260709 13:04:35.086587 15693 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.086717 15662 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:35.087350 13868 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:35.025955 15692 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.092135 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:35.095543 15582 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.095974 15667 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:35.096724 15692 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:35.096545 15667 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.099649 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:35.103338 15667 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:35.102931 15718 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.036615 15602 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.121917 15635 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:35.137468 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 195426us
I20260709 13:04:35.127326 15662 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.124040 15556 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.140333 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:35.127182 15707 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:35.101035 15683 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.144176 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:35.144594 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:35.145763 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:35.146445 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:35.147032 13875 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:35.149576 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:35.156396 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:35.157059 13870 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:35.139338 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:35.142535 15707 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:35.153841 13772 catalog_manager.cc:5697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 2 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:35.143162 15556 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.143303 15662 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.191179 15556 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:35.192821 15662 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:35.184158 15683 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:35.178954 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:35.156860 15713 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.138s
I20260709 13:04:35.176019 15667 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.205439 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:35.206827 15667 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.207144 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:35.209435 15667 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:35.198573 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:35.168277 14554 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:35.202126 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:35.203164 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:35.213070 13947 raft_consensus.cc:2368] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4.
I20260709 13:04:35.176355 15691 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:35.219177 14018 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:35.234861 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:35.237206 13875 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:35.237468 15691 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.244298 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:35.246290 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:35.213140 15718 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.220412 15722 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.249868 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:35.225184 15731 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:35.254474 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:35.257815 14021 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:35.259564 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:35.259727 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:35.260100 14019 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:35.260268 14024 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:35.261234 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:35.256210 14018 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:35.258818 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:35.262802 14021 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:35.263499 14016 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:35.265478 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:35.266827 14020 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:35.267175 15707 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.267869 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:35.268574 14020 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:04:35.269542 15602 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:35.269495 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:35.263248 15722 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.272241 14020 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:35.273312 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:35.273748 15722 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:35.274449 14022 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:35.275143 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:35.276656 14020 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:35.276865 14023 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:35.268590 15707 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.187366 15670 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.265913 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:35.279629 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:35.281263 14021 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:35.282949 14024 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:35.278872 15731 raft_consensus.cc:697] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:35.287976 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:35.287667 15731 consensus_queue.cc:237] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.289043 15424 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:35.290588 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:35.290885 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:35.291404 15424 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:35.292405 15504 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:35.292591 15670 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:35.292663 15556 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:35.292898 15504 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:35.293290 15556 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:35.293699 13987 leader_election.cc:400] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4.
I20260709 13:04:35.294394 13987 leader_election.cc:403] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:35.254406 15635 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:35.297603 15693 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.282333 15601 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.299122 15601 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.301640 14554 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:35.306188 15635 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:35.307518 15635 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.313776 14024 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:35.314023 13945 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:35.315138 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:35.315833 13947 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:35.316563 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:35.317174 13875 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.317272 13950 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:35.318457 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:35.293717 15556 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.327860 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:35.329965 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:35.331321 13943 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:35.333442 15693 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:35.334386 15711 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:35.337002 15728 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:35.337273 15725 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.337461 15728 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:35.338848 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:35.335768 15708 consensus_meta.cc:287] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.583s	user 0.014s	sys 0.000s
I20260709 13:04:35.341274 15728 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.342150 15708 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.343179 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:35.343816 13948 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:35.339715 13949 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:35.344481 15708 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:35.301981 15601 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:35.293866 15504 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.347394 15504 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.349685 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:35.351593 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:35.352316 13871 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:35.356460 15556 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.358214 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:35.360728 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.295130 15722 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
W20260709 13:04:35.362767 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.363339 15728 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:35.364329 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:35.366804 13987 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:35.365252 13871 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.257746 15718 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.374245 15722 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:35.373822 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:35.374711 15718 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:35.376658 15556 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:35.378194 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:35.378405 14561 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:35.378527 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:35.379021 14553 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 3: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:35.379334 13866 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.379794 15556 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:35.383514 14017 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 2: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:35.391745 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:35.398708 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:35.399807 14024 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:35.382402 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:35.400626 13873 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:35.402114 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:35.403378 15591 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:35.365026 15693 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:35.408190 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:35.410485 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.411446 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:35.413160 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:35.414171 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:35.415503 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:35.416443 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:35.328138 15670 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:35.417966 14015 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:35.413213 15556 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:35.417747 13948 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.427027 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:35.427739 14024 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.429419 14015 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:35.434918 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:35.435640 13949 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.436995 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:35.439720 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:35.442292 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:35.443524 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:35.451104 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:35.451467 13875 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:35.459439 13866 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:35.460289 13947 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:35.463563 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:35.464869 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:35.466249 13948 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:35.473613 14024 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:35.442466 15683 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.429458 15685 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:35.440699 15668 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:35.443161 15691 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:35.490875 13871 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:35.467695 15488 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:35.467448 15718 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:35.476644 15683 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.429388 15662 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:04:35.429388 15673 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:04:35.471478 15722 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.508387 13914 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:35.471336 15692 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:35.480173 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:35.486334 15601 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.467588 15710 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:04:35.454262 15591 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:04:35.552747 14015 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.557300 13870 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:35.473246 15639 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:35.488018 15668 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.467015 15711 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:04:35.486588 15685 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.443243 15670 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:35.525821 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 83656.6us
I20260709 13:04:35.534245 13942 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
W20260709 13:04:35.580470 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.531668 15725 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4.
I20260709 13:04:35.543565 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:35.588466 13875 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:35.543951 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:35.563254 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:35.550536 13773 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57648 (request call id 276) took 1410 ms. Trace:
I20260709 13:04:35.486006 15582 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:04:35.527045 15492 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:35.626641 15492 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:35.573289 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:35.629170 14017 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
W20260709 13:04:35.574378 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 99686.4us
I20260709 13:04:35.574513 13949 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
W20260709 13:04:35.575354 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 133038us
I20260709 13:04:35.579001 13873 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:35.522912 15722 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.537799 15721 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:35.584584 15670 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:35.570341 15668 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.577637 15685 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.589534 13871 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.544222 15601 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.568734 15696 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.566108 15639 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.709266 14021 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:35.579299 14524 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:35.639652 14015 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:35.639942 14522 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:35.640620 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 187483us
I20260709 13:04:35.641656 13947 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:35.648790 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:35.713256 14019 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 3: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:35.653116 13873 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:35.655936 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:35.657852 13987 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:04:35.715487 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 57764.2us
I20260709 13:04:35.630085 15713 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.621457 13773 rpcz_store.cc:276] 0709 13:04:34.140035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.140175 (+   140us) service_pool.cc:224] Handling call
0709 13:04:35.548661 (+1408486us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:34.202999 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:34.226454 (+ 23455us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:34.234213 (+  7759us) write_op.cc:435] Acquired schema lock
0709 13:04:34.241284 (+  7071us) tablet.cc:662] Decoding operations
0709 13:04:34.354330 (+113046us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:34.361302 (+  6972us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:34.366485 (+  5183us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:34.410773 (+ 44288us) tablet.cc:701] Row locks acquired
0709 13:04:34.413263 (+  2490us) write_op.cc:260] PREPARE: finished
0709 13:04:34.432802 (+ 19539us) write_op.cc:270] Start()
0709 13:04:34.439927 (+  7125us) write_op.cc:276] Timestamp: P: 1783602274430699 usec, L: 0
0709 13:04:34.442324 (+  2397us) op_driver.cc:348] REPLICATION: starting
0709 13:04:34.562345 (+120021us) log.cc:844] Serialized 541 byte log entry
0709 13:04:34.942933 (+380588us) op_driver.cc:464] REPLICATION: finished
0709 13:04:35.125582 (+182649us) write_op.cc:301] APPLY: starting
0709 13:04:35.126749 (+  1167us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:35.136770 (+ 10021us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:35.148050 (+ 11280us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:35.179427 (+ 31377us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:35.180732 (+  1305us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:04:35.180812 (+    80us) write_op.cc:312] APPLY: finished
0709 13:04:35.181137 (+   325us) log.cc:844] Serialized 23 byte log entry
0709 13:04:35.181388 (+   251us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:35.181513 (+   125us) write_op.cc:454] Released schema lock
0709 13:04:35.181589 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:35.182130 (+   541us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:35.182160 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":163983,"apply.run_cpu_time_us":3057,"apply.run_wall_time_us":62600,"num_ops":1,"prepare.queue_time_us":1328,"prepare.run_cpu_time_us":5032,"prepare.run_wall_time_us":397374,"replication_time_us":495125,"thread_start_us":87150,"threads_started":1}]]}
W20260709 13:04:35.717011 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 61182.8us
I20260709 13:04:35.736258 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:35.627804 15492 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.647001 15721 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:35.745294 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 105440us
I20260709 13:04:35.765884 14022 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
W20260709 13:04:35.745085 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 165834us
I20260709 13:04:35.654402 15708 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:35.649122 15691 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:35.744445 13838 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:35.768827 13871 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:35.654485 15725 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:35.804412 15761 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:35.806100 15693 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.806658 15693 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.809710 15683 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:35.733942 15696 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.822425 15696 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:35.823719 15693 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:35.651593 15673 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.116s
I20260709 13:04:35.742311 15424 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:35.738847 15674 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:35.733534 15639 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.767050 15721 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.735394 15718 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:35.883622 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 139266us
I20260709 13:04:35.884651 13838 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:35.803217 15691 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:35.886224 15504 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:35.766902 15670 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:35.886399 15710 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:35.886713 15504 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:35.887120 15670 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:35.887266 15504 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.888163 15670 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.888661 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:35.889981 15718 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.764297 15741 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:35.891245 15759 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:04:35.743906 15713 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.893361 15759 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.890919 15667 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:35.886547 15691 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.894449 15667 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:35.895113 15691 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.895485 15722 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:35.895622 15667 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.901069 13947 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:35.772019 15633 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:35.849218 15424 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.859252 15436 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:35.871381 15754 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.291s	user 0.002s	sys 0.000s
I20260709 13:04:35.848778 15725 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.893867 15759 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.931576 15633 raft_consensus.cc:697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:35.922696 15424 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.906744 15761 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:04:35.928459 15722 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:35.947083 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 210921us
I20260709 13:04:35.958401 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:35.962994 15674 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:35.964105 15556 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.535s	user 0.005s	sys 0.005s
I20260709 13:04:35.964617 15556 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:35.930855 15436 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:35.966879 15722 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:35.968444 15436 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:35.969247 15722 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:35.969377 15489 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:35.969977 15756 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
W20260709 13:04:35.950902 15707 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.673s	user 0.011s	sys 0.003s
I20260709 13:04:35.986968 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:35.989243 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:35.971107 15488 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:35.993705 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:35.992444 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 63700.2us
I20260709 13:04:35.993862 15692 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.016461 14556 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:35.994867 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:36.011742 14017 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:36.020304 13988 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:36.019098 15488 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:36.014622 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:36.025254 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:36.027220 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:36.031946 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:36.009415 15713 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:35.994930 15685 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.055840 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:36.021473 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:36.022382 15770 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:36.035844 15693 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.027782 15774 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.054596 15674 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.038329 15601 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:36.061511 15436 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:36.057855 15698 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:36.023953 15692 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:36.071909 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 55247.9us
I20260709 13:04:36.055152 14554 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:36.065088 15741 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:36.076871 15693 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.079491 15674 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.075811 13942 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
W20260709 13:04:36.132727 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 56286.2us
I20260709 13:04:36.136842 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:36.138041 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:36.138424 13950 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:04:36.138641 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:36.104683 15693 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.119642 15674 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.081723 15770 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.157168 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.157766 13873 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:36.158556 13869 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:36.160669 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.148262 14522 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:36.120407 15761 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:36.165009 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.064299 15633 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.167197 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.141214 15781 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:36.170881 15775 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:04:36.170609 15781 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.173379 15781 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:36.174196 13867 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:36.162305 14522 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:36.176962 15773 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.177448 15773 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.177855 15773 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.190405 13870 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:36.191063 13866 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 1.
I20260709 13:04:36.179427 15683 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:04:36.195648 15667 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.196316 15667 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.198402 15670 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:36.198863 15670 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.199241 15670 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:36.199728 15602 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:36.201553 15683 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:36.202026 15683 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:36.206295 15725 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:36.209703 15758 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.408s	user 0.001s	sys 0.000s
W20260709 13:04:36.212304 15757 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.557s	user 0.002s	sys 0.000s
I20260709 13:04:36.214315 15761 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.216266 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:36.203188 15683 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:36.234790 15668 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.575s	user 0.014s	sys 0.000s
I20260709 13:04:36.237320 15773 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.206187 15721 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.263242 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:36.082370 15774 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.247689 13869 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:36.151289 15767 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
W20260709 13:04:36.230350 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 78029.1us
I20260709 13:04:36.248093 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.248454 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.266120 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.144025 15608 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.274045 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:36.269532 13867 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:36.279031 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.151700 15711 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.278800 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.299132 14023 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:36.243866 15759 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:36.261502 15683 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:36.235435 15668 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.240622 15674 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.241752 15696 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:36.261046 15701 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:36.309562 14555 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:36.273445 13943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:36.270323 15761 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:36.275931 15608 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.343128 15608 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.335057 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.352703 13871 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:36.329741 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:36.336544 15701 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.276976 15766 consensus_queue.cc:1059] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:04:36.342463 13943 raft_consensus.cc:606] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:36.267269 15713 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
W20260709 13:04:36.366469 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 122940us
I20260709 13:04:36.360292 13875 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:04:36.367903 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:36.307983 15785 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:36.380002 15766 raft_consensus.cc:3055] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:36.380667 15766 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:36.381937 15766 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.383890 15766 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:36.395556 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:36.357448 15701 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.402910 13989 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:36.405289 15693 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.362452 15667 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:36.308707 15711 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.408303 15771 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:36.355192 15698 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:04:36.364912 13942 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:36.410900 15773 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:36.411993 15674 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:36.414198 15774 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:36.415099 15692 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:36.434819 15692 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.435300 15692 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.435719 15692 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.374903 15685 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:36.407567 15668 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:36.413050 15721 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:36.377702 15785 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.416500 15792 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:36.430379 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:36.368695 15639 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.432857 15488 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.423702 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:36.357175 15696 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.421657 15701 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.467051 13873 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
W20260709 13:04:36.468842 15424 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.000s
I20260709 13:04:36.395720 15725 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 13:04:36.373677 15504 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:36.473816 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 210688us
I20260709 13:04:36.475240 14021 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:04:36.497954 14022 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:36.545508 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:36.542251 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 298531us
I20260709 13:04:36.574599 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:36.463083 15667 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.578016 14017 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 2: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:36.477063 15601 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:36.473318 15785 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:36.583048 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 316989us
I20260709 13:04:36.542217 15685 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.570386 15492 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.541324 15771 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:36.607789 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.608649 13869 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:36.579481 15790 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.609776 15790 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.606966 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:04:36.596076 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 175264us
I20260709 13:04:36.611874 13870 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:36.584374 15674 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.574942 15424 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.577577 15667 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.630587 15693 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:36.614609 15492 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.631953 15488 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:36.632644 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:36.655594 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:04:36.643682 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 70675.5us
I20260709 13:04:36.628932 15608 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.646461 13868 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:36.660651 14016 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:04:36.603317 15685 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:36.672094 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 95437.6us
I20260709 13:04:36.679796 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:36.680730 13869 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.684567 13867 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:36.687459 15804 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:36.688172 15773 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:36.692751 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:36.693746 14556 raft_consensus.cc:2410] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:36.642206 15674 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:36.706931 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.704442 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:36.706678 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:36.698122 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:36.710203 15608 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:36.712488 14556 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:36.713824 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:36.714366 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:36.716728 14019 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
W20260709 13:04:36.736176 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 99670.3us
I20260709 13:04:36.665333 15746 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:04:36.724309 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:36.703004 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:36.684198 15770 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:36.739913 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:36.757561 14559 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.758828 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:04:36.741344 15670 consensus_meta.cc:287] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.005s
W20260709 13:04:36.768267 13759 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 545fc4aeb71d491985e296a0721d905d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d1b4fd04200247d99ba9ad1fbeb42b42 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:36.741691 15773 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:36.744485 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:36.745218 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:36.746759 15708 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:36.773681 15708 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.750684 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:36.753566 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:36.762858 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:36.765318 15424 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:36.738374 15639 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:36.766839 13943 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.772701 15760 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:36.793964 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:36.703946 15799 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:36.749711 15804 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:36.795657 15804 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.799067 15795 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:04:36.797838 15582 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:04:36.802722 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.779335 13949 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:36.783450 14552 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:36.780937 15792 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:36.789798 14556 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:36.770833 15770 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:36.773046 14554 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:36.794409 15760 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.801949 15582 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:36.814080 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:36.778736 15696 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:36.816779 13949 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:36.691223 15771 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.825721 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:36.825970 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:36.827034 13949 raft_consensus.cc:2368] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4.
I20260709 13:04:36.787750 15504 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:36.838620 15692 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.840999 15692 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:36.797868 15804 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:36.771618 15670 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.841732 15785 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.835510 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:36.832208 15792 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.843835 15639 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:04:36.862473 15683 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:36.862973 15683 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.863338 15683 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:36.833434 15770 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.867363 13950 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 5: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:36.830228 15582 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:36.817438 15799 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:36.873747 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.874465 13874 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:36.878275 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:36.878906 13867 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:36.869864 13947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:36.879640 13947 raft_consensus.cc:606] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:36.881176 15709 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:36.881850 15709 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:36.890035 13943 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
W20260709 13:04:36.890601 15808 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:04:36.892655 15718 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:36.893422 15785 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:36.897127 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:36.893472 15790 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:36.898741 14522 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:36.904165 14559 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
W20260709 13:04:36.874814 15628 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.017s	sys 0.016s
I20260709 13:04:36.913726 14019 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:36.892692 13869 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:36.916190 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:36.917590 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:36.921442 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:36.922187 13870 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:36.922495 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:36.918265 15628 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.017s	sys 0.016s
I20260709 13:04:36.859932 15674 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:36.899555 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.880663 15602 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:04:36.914639 15799 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.927523 15776 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.923900 15504 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:36.925433 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:36.901780 15718 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:36.955106 15718 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.925154 15639 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:36.914636 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:36.959290 14552 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:36.911470 15685 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:36.916446 15721 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:36.884387 15769 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:04:37.005112 15721 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:37.008133 13772 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54072 (request call id 134) took 1987 ms. Trace:
I20260709 13:04:36.989774 13870 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:37.006536 13771 catalog_manager.cc:5697] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:36.942663 15806 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:36.982122 15504 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.940658 15815 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:37.035766 15815 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.036192 15815 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.037225 15582 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.037561 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1202) took 1038 ms. Trace:
I20260709 13:04:37.038102 14016 rpcz_store.cc:276] 0709 13:04:35.999629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.999752 (+   123us) service_pool.cc:224] Handling call
0709 13:04:36.000034 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.661311 (+661277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.037285 (+375974us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b540023c154. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:37.037494 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1518410496}
I20260709 13:04:37.039513 15701 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:37.040978 13944 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 4: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:37.041586 13838 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:37.030356 15790 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.042505 15685 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:37.042562 15790 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:37.043184 15685 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.043826 15806 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:37.043823 15685 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:37.045058 15806 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:36.945195 15776 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.029476 15700 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:04:37.006469 15721 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:37.088481 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 123421us
I20260709 13:04:37.095993 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:37.096971 13949 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:37.098153 15770 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:37.098690 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:37.099159 15718 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:37.058319 13838 leader_election.cc:400] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4.
I20260709 13:04:37.100028 13838 leader_election.cc:403] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:04:37.100164 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:37.100775 15781 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4.
I20260709 13:04:37.100917 13870 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:37.008569 13772 rpcz_store.cc:276] 0709 13:04:35.021215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.021492 (+   277us) service_pool.cc:224] Handling call
0709 13:04:37.008063 (+1986571us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:35.239002 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:35.248077 (+  9075us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:35.250128 (+  2051us) write_op.cc:435] Acquired schema lock
0709 13:04:35.251364 (+  1236us) tablet.cc:662] Decoding operations
0709 13:04:35.252073 (+   709us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:35.252207 (+   134us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:35.252270 (+    63us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:35.252630 (+   360us) tablet.cc:701] Row locks acquired
0709 13:04:35.252663 (+    33us) write_op.cc:260] PREPARE: finished
0709 13:04:35.252983 (+   320us) write_op.cc:270] Start()
0709 13:04:35.253202 (+   219us) write_op.cc:276] Timestamp: P: 1783602275252880 usec, L: 0
0709 13:04:35.253248 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 13:04:35.307252 (+ 54004us) log.cc:844] Serialized 536 byte log entry
0709 13:04:35.333990 (+ 26738us) op_driver.cc:464] REPLICATION: finished
0709 13:04:35.334475 (+   485us) write_op.cc:301] APPLY: starting
0709 13:04:35.334556 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:35.334662 (+   106us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:35.334793 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:35.343035 (+  8242us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:35.343235 (+   200us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:04:35.343287 (+    52us) write_op.cc:312] APPLY: finished
0709 13:04:35.343501 (+   214us) log.cc:844] Serialized 23 byte log entry
0709 13:04:35.343691 (+   190us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:35.343787 (+    96us) write_op.cc:454] Released schema lock
0709 13:04:35.343835 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:35.344259 (+   424us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:35.344296 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":207659,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":250,"apply.run_cpu_time_us":1741,"apply.run_wall_time_us":9692,"num_ops":1,"prepare.queue_time_us":74008,"prepare.run_cpu_time_us":4483,"prepare.run_wall_time_us":68606,"replication_time_us":80584,"thread_start_us":76356,"threads_started":1}]]}
W20260709 13:04:37.101886 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 43831.9us
I20260709 13:04:37.103447 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:37.104876 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:37.105551 14556 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:04:37.106657 13914 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.651s (SENT) [suppressed 47 similar messages]
I20260709 13:04:37.107676 15662 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:37.113325 13950 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "e1649107c781428692458f10597c5742"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:37.113423 13949 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:04:37.114166 13950 raft_consensus.cc:606] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Received request to transfer leadership to TS e1649107c781428692458f10597c5742
I20260709 13:04:37.114672 13950 raft_consensus.cc:614] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:37.119021 13942 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:37.075693 15776 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.111096 15803 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:37.125694 15803 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.128013 15803 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:37.130916 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:37.134140 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:37.132485 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:37.136273 15763 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.083304 15792 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:37.083571 15701 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:37.153117 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:37.154970 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:37.158439 15770 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.156404 15722 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:37.192929 13867 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:37.074260 15683 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.168118 14016 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.122509 15692 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:37.143101 15639 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.153623 15710 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:37.197549 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:37.167140 14019 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:37.157368 15760 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:37.217772 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:37.222383 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:04:37.216990 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 110344us
I20260709 13:04:37.229306 14016 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:37.172654 15504 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:37.238271 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:37.163745 15673 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
W20260709 13:04:37.213838 15735 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 2.248s	user 0.013s	sys 0.015s
I20260709 13:04:37.170167 15708 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:37.278962 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 50264.5us
I20260709 13:04:37.282725 13950 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 3: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:37.389026 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:37.389446 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:37.411196 14016 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:37.344687 14019 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:37.413602 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:37.414803 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:37.380657 15817 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 13:04:37.206845 15770 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.243647 15710 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:37.471021 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:37.379693 15820 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:04:37.404414 15803 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.685913 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:37.686622 14016 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:37.683156 15683 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:37.690349 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:37.267562 13943 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:37.222082 15639 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.395632 14522 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:37.221415 15692 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.690609 15710 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:37.691247 15815 consensus_meta.cc:287] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.655s	user 0.012s	sys 0.000s
I20260709 13:04:37.230650 15806 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.774282 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:37.637959 13874 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:37.222707 15722 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.643333 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:37.660058 13988 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:37.693078 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:37.695000 14016 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:37.694909 13941 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:37.677950 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 273981us
W20260709 13:04:37.685019 15685 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.640s	user 0.005s	sys 0.003s
I20260709 13:04:37.789675 14016 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:37.780907 13942 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:37.670168 15792 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:37.714668 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:37.684700 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:37.644483 15768 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:37.795908 14555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 463) took 1764 ms (client timeout 1651 ms). Trace:
W20260709 13:04:37.797289 14555 rpcz_store.cc:269] 0709 13:04:36.026634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.026810 (+   176us) service_pool.cc:224] Handling call
0709 13:04:37.791200 (+1764390us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54007148d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:37.791300 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640888704}
I20260709 13:04:37.799154 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:37.727672 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 332085us
I20260709 13:04:37.233616 15760 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:37.680124 15782 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 1.355s	user 0.003s	sys 0.000s
I20260709 13:04:37.763025 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 205) took 1605 ms. Trace:
I20260709 13:04:37.772822 14554 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:37.402390 15701 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:04:37.698298 13840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.013s	sys 0.029s
W20260709 13:04:37.805860 13840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.013s	sys 0.030s
I20260709 13:04:37.245430 15785 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:37.709829 15556 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.787792 14018 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:37.806422 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 641532us
W20260709 13:04:37.788347 13873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2084) took 1623 ms (client timeout 1760 ms). Trace:
I20260709 13:04:37.416043 15764 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:04:37.817551 13873 rpcz_store.cc:269] 0709 13:04:36.164663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.164781 (+   118us) service_pool.cc:224] Handling call
0709 13:04:37.788163 (+1623382us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b540029e354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:37.788282 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693007232}
I20260709 13:04:37.793102 14015 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:37.647469 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1191) took 2004 ms. Trace:
I20260709 13:04:37.204399 15763 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.793613 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:37.636121 15780 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.822863 14016 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:37.716396 15683 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:37.622850 15436 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:37.771886 15804 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.822337 15701 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.833524 14016 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:04:37.819207 14024 rpcz_store.cc:276] 0709 13:04:35.643273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.643445 (+   172us) service_pool.cc:224] Handling call
0709 13:04:35.647387 (+  3942us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:35.915984 (+268597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:35.916035 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:35.927694 (+ 11659us) log.cc:844] Serialized 26 byte log entry
0709 13:04:36.057685 (+129991us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:36.259959 (+202274us) log.cc:844] Serialized 14 byte log entry
0709 13:04:36.324343 (+ 64384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.443702 (+1119359us) spinlock_profiling.cc:243] Waited 68.1 ms on lock 0x7b540023a354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:37.623458 (+179756us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.623508 (+    50us) raft_consensus.cc:1685] finished
0709 13:04:37.646995 (+ 23487us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.647330 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4988,"prepare.queue_time_us":260642,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":113733,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":26201,"spinlock_wait_cycles":156626304,"thread_start_us":537368,"threads_started":4,"wal-append.queue_time_us":200930,"wal-append.run_cpu_time_us":3421,"wal-append.run_wall_time_us":1243330}
I20260709 13:04:37.869627 13869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:37.793601 15799 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.692435 15662 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:37.499502 15766 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.387576 15582 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:37.807896 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:37.812122 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:37.575037 15711 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:37.637501 15772 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:37.815740 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:37.804625 13875 rpcz_store.cc:276] 0709 13:04:36.157327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.158492 (+  1165us) service_pool.cc:224] Handling call
0709 13:04:36.158818 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.367706 (+208888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.762681 (+1394975us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54000d9b54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:37.762947 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462580608}
I20260709 13:04:37.786985 15765 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:37.835435 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 49652.1us
I20260709 13:04:37.895550 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:37.896216 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:37.802265 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:37.876585 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 59452.1us
I20260709 13:04:37.692090 15803 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.796231 15721 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:04:37.888990 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 76985.8us
I20260709 13:04:37.892778 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 1995) took 1471 ms. Trace:
I20260709 13:04:37.898252 13941 raft_consensus.cc:2368] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3.
I20260709 13:04:37.916808 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:37.792924 15806 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.908622 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1210) took 3065 ms. Trace:
I20260709 13:04:37.908010 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:37.926587 13942 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.859174 15780 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.899636 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:37.918666 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:37.940529 14021 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:37.925637 14023 rpcz_store.cc:276] 0709 13:04:36.421113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.708936 (+287823us) service_pool.cc:224] Handling call
0709 13:04:36.709376 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.710311 (+   935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.710346 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:36.710849 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:04:36.925122 (+214273us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:36.925177 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.925245 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.892553 (+967308us) raft_consensus.cc:1685] finished
0709 13:04:37.892576 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.892705 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":172,"thread_start_us":213903,"threads_started":1,"wal-append.queue_time_us":409505}
I20260709 13:04:37.942878 14561 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:37.883123 14553 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:37.800280 15685 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:37.921841 15776 consensus_meta.cc:287] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.768s	user 0.010s	sys 0.000s
I20260709 13:04:37.822561 15760 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.901012 15801 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:37.874472 15436 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:37.814855 15768 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.792503 15815 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.926894 13946 rpcz_store.cc:276] 0709 13:04:34.827539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.828001 (+   462us) service_pool.cc:224] Handling call
0709 13:04:34.828387 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:34.828817 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:34.828859 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:34.835590 (+  6731us) log.cc:844] Serialized 26 byte log entry
0709 13:04:34.867985 (+ 32395us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:34.868031 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.868124 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.156793 (+2288669us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b2400249fc8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c01cc9c4 00007fb2c01caacb 00007fb2c0385dc9 00007fb2c0385a55 00007fb2c038509d 00007fb2c2fb710e 00007fb2c2fce88c 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221
0709 13:04:37.876171 (+719378us) raft_consensus.cc:1685] finished
0709 13:04:37.876205 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.892531 (+ 16326us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7546,"raft.queue_time_us":2773,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":5483,"spinlock_wait_cycles":16635264,"thread_start_us":21406,"threads_started":1,"wal-append.queue_time_us":118909}
I20260709 13:04:37.946601 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:37.907015 14018 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:37.804648 15792 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:37.815402 15785 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.829506 15556 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:37.965517 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 65953.4us
I20260709 13:04:38.062536 13949 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:38.063073 15667 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.063736 15667 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:38.065366 15770 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:38.066015 15770 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:38.066536 15770 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.067410 15582 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:38.068310 15582 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:38.068679 15582 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:38.069052 15770 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:38.069664 15770 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:38.070528 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:38.071259 15668 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:38.071712 15668 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:38.072328 15803 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:38.072906 15803 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.072625 15668 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.920053 15701 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:37.893864 15492 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:37.911197 15812 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.035884 13869 raft_consensus.cc:606] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:04:38.036276 13941 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:38.085453 13869 raft_consensus.cc:614] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:37.920069 15790 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:38.042917 15806 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:37.915905 15639 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:37.925200 15772 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.045435 15685 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:37.985879 15721 raft_consensus.cc:697] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:38.046288 15831 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:38.108160 15721 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:37.890224 15662 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:38.083460 15781 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.114303 15771 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:38.108682 15831 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.116987 15771 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.118266 15692 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:38.118636 15831 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.119825 15771 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:37.885126 15804 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:38.047977 15708 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:37.985932 15710 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:38.050897 15670 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.052175 15768 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:37.880429 15683 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.059693 15815 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.023730 13947 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:37.918483 15722 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:37.909195 15829 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:37.907343 15602 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:38.081353 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:38.162007 13866 raft_consensus.cc:2410] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:04:38.024945 13942 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:38.163336 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:38.081559 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:38.164577 13874 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:38.165735 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:38.166488 13874 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:38.081966 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:38.073372 15803 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
I20260709 13:04:38.167514 13875 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:38.168668 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:37.827080 15764 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.170567 13950 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:37.895391 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:37.915956 15709 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:38.061939 15556 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.087330 15582 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:38.092617 14554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 292) took 1002 ms. Trace:
I20260709 13:04:38.199808 14554 rpcz_store.cc:276] 0709 13:04:37.089874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.090395 (+   521us) service_pool.cc:224] Handling call
0709 13:04:37.090750 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:37.805531 (+714781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.092243 (+286712us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b5400730354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:38.092507 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1568007552}
I20260709 13:04:37.886509 15799 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:37.886673 15795 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.193s
I20260709 13:04:38.096515 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:38.074025 15668 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:38.083232 15770 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:38.105531 13773 catalog_manager.cc:5697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:38.230751 13915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.004s	sys 0.022s
W20260709 13:04:38.231371 13915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.004s	sys 0.022s
I20260709 13:04:38.231781 13773 catalog_manager.cc:5697] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 3, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.116861 15781 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:38.025790 15776 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.047161 13948 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:38.108731 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:38.291715 14017 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:38.110023 15790 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:38.120522 15760 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.115396 15780 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.308461 15771 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:38.309021 15771 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.201670 15722 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:38.113257 15674 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:38.345328 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:38.346159 13868 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:38.113004 15772 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.123131 15662 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.123626 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:38.125196 15763 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:37.898874 15766 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:38.129364 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:38.094417 15812 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.115609 15436 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:38.222846 15709 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.147s
I20260709 13:04:38.188143 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:38.352645 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:38.247157 13773 catalog_manager.cc:5697] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 3, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.270129 15770 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:38.290220 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 630235us
I20260709 13:04:38.526826 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:38.297123 13875 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:38.305831 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.305933 15815 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.310472 15790 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.350509 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:38.309834 15683 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.082669 15785 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:38.171959 15829 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.434891 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:38.386300 13870 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:38.608979 14024 raft_consensus.cc:2368] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4.
I20260709 13:04:37.907644 15765 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:38.567440 14523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.036s	sys 0.027s
W20260709 13:04:38.616062 14523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.970s	user 0.036s	sys 0.028s
I20260709 13:04:38.616710 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:38.305249 15556 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:38.543911 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:38.587831 13874 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:38.612558 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:38.581836 13869 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:38.435365 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:38.807713 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:39.576829 13942 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 2: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:39.577013 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2106) took 1282 ms. Trace:
W20260709 13:04:38.724596 13914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.015s	sys 0.056s
W20260709 13:04:38.619517 13867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.029s	sys 0.029s
I20260709 13:04:39.032845 15792 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:39.578472 13867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.752s	user 0.047s	sys 0.074s
W20260709 13:04:38.899571 15711 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.047s	sys 0.064s
W20260709 13:04:39.579291 15711 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.508s	user 0.052s	sys 0.072s
I20260709 13:04:38.806332 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:38.806862 15684 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.351984 15713 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 1.238s
I20260709 13:04:39.582100 15684 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:39.113485 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.621116 15815 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:39.586727 13875 raft_consensus.cc:2410] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:39.587402 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2110) took 1009 ms. Trace:
I20260709 13:04:39.587924 13875 rpcz_store.cc:276] 0709 13:04:38.578075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.612154 (+ 34079us) service_pool.cc:224] Handling call
0709 13:04:39.587317 (+975163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:39.589953 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:38.729899 13944 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:38.827373 15801 consensus_meta.cc:287] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.791s	user 0.009s	sys 0.002s
W20260709 13:04:39.052062 15721 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.000s	sys 0.021s
W20260709 13:04:39.600863 15721 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.297s	user 0.004s	sys 0.033s
W20260709 13:04:39.577736 13914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.675s	user 0.028s	sys 0.092s
W20260709 13:04:39.032066 13989 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.953s	user 0.009s	sys 0.012s
I20260709 13:04:39.592669 14017 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:04:39.602262 13989 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.523s	user 0.009s	sys 0.028s
I20260709 13:04:38.760923 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:39.604168 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 1945) took 1519 ms. Trace:
I20260709 13:04:38.306499 15692 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:39.133764 15504 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.006s	sys 0.014s
I20260709 13:04:38.615482 15662 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:38.993571 15635 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.816066 15668 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.000s
I20260709 13:04:39.290431 15837 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:39.593333 15556 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:38.820483 15776 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:39.298499 15436 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:39.300230 15812 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.591524 13773 catalog_manager.cc:5697] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.805841 15602 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:38.809726 13947 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:04:39.642843 13947 pending_rounds.cc:85] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 1
I20260709 13:04:38.807224 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:39.599305 15815 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.617231 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:39.643774 14021 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:39.644328 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1213) took 1241 ms. Trace:
I20260709 13:04:38.797876 15683 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:38.620685 14524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.009s	sys 0.018s
I20260709 13:04:39.627660 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:38.619897 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 1908) took 1218 ms. Trace:
I20260709 13:04:39.601528 15763 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.601593 15790 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:39.576514 13869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 1946) took 1486 ms (client timeout 1930 ms). Trace:
I20260709 13:04:39.604679 13874 rpcz_store.cc:276] 0709 13:04:38.085162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.187749 (+102587us) service_pool.cc:224] Handling call
0709 13:04:39.604104 (+1416355us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:39.604969 13868 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.249s	user 0.021s	sys 0.017s
I20260709 13:04:39.678896 13868 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:39.620440 15692 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.993958 15772 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:39.601117 13838 leader_election.cc:400] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3.
I20260709 13:04:39.612231 13941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
W20260709 13:04:39.597788 15670 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.017s	sys 0.013s
I20260709 13:04:39.617682 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1208) took 1253 ms. Trace:
W20260709 13:04:39.141139 15731 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.008s	sys 0.010s
I20260709 13:04:39.576406 14023 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:39.624486 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 1931) took 2487 ms. Trace:
I20260709 13:04:38.760632 15806 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:38.344422 15674 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:04:39.625731 14019 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.439s	user 0.000s	sys 0.014s
I20260709 13:04:39.628583 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:39.625780 14523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.009s	user 0.002s	sys 0.013s
I20260709 13:04:39.621213 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2109) took 1325 ms. Trace:
W20260709 13:04:38.948900 15617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.893s	user 0.045s	sys 0.089s
I20260709 13:04:38.798017 15628 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:38.728976 15701 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.730401 15811 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:39.577492 13866 rpcz_store.cc:276] 0709 13:04:38.294745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.741772 (+447027us) service_pool.cc:224] Handling call
0709 13:04:38.743683 (+  1911us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:39.576591 (+832908us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.576630 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:39.576694 (+    64us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:39.576743 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.576846 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.576957 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:39.618472 15504 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.355s	user 0.016s	sys 0.018s
I20260709 13:04:39.604091 15685 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:39.634138 13872 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.021s	sys 0.009s
I20260709 13:04:38.816378 15829 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:39.307236 15843 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:39.637034 13840 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.015s	sys 0.018s
I20260709 13:04:39.600893 15801 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:38.352355 15722 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:38.823063 15799 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:38.760279 15492 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:38.797742 15760 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:39.618395 15792 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.640286 13773 catalog_manager.cc:5697] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.803129 13773 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 3, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.598543 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 1942) took 1518 ms. Trace:
I20260709 13:04:38.966264 15591 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:38.799981 15835 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:38.605612 15768 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.643139 13947 pending_rounds.cc:107] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:04:39.644629 14021 rpcz_store.cc:276] 0709 13:04:38.402803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.402955 (+   152us) service_pool.cc:224] Handling call
0709 13:04:39.644247 (+1241292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:39.812821 13947 raft_consensus.cc:2889] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:39.646589 14524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.573s	user 0.035s	sys 0.056s
I20260709 13:04:39.805547 13773 catalog_manager.cc:5697] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 3, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:39.813454 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 1.73989e+06us
I20260709 13:04:39.813748 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 1923) took 1005 ms. Trace:
W20260709 13:04:38.720808 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:39.815214 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 202757us
I20260709 13:04:39.815923 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:39.816226 13989 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:39.646852 15766 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:39.820519 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:39.822413 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:39.823782 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:39.824455 13942 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:39.825407 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:39.646967 13948 rpcz_store.cc:276] 0709 13:04:37.390191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.390320 (+   129us) service_pool.cc:224] Handling call
0709 13:04:37.390612 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:38.291123 (+900511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.607833 (+316710us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b5400229054. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:38.608073 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1509058816}
W20260709 13:04:39.658656 14525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.063s	user 0.008s	sys 0.008s
W20260709 13:04:39.827756 14525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.232s	user 0.008s	sys 0.008s
W20260709 13:04:39.659096 14522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.000s	sys 0.015s
W20260709 13:04:39.828678 14522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.000s	sys 0.016s
I20260709 13:04:39.814195 13947 rpcz_store.cc:276] 0709 13:04:38.808234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.808860 (+   626us) service_pool.cc:224] Handling call
0709 13:04:38.809220 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:39.813456 (+1004236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.813685 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:39.816552 13946 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:39.621133 15839 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:04:39.678131 14024 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.005s	sys 0.013s
I20260709 13:04:39.679481 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2103) took 1386 ms. Trace:
I20260709 13:04:39.605497 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:39.679924 13869 rpcz_store.cc:269] 0709 13:04:38.089685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.129071 (+ 39386us) service_pool.cc:224] Handling call
0709 13:04:39.576435 (+1447364us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:39.624279 15662 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:39.624650 15838 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:04:39.685811 13838 leader_election.cc:403] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:04:39.686393 13941 raft_consensus.cc:606] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:39.682132 15790 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:39.795864 15841 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:39.777539 15811 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:39.708225 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:39.712862 14015 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.005s	sys 0.021s
I20260709 13:04:39.736529 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1214) took 1311 ms. Trace:
I20260709 13:04:39.712622 14017 rpcz_store.cc:276] 0709 13:04:38.349541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.349747 (+   206us) service_pool.cc:224] Handling call
0709 13:04:39.602581 (+1252834us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:39.633937 15635 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:39.711321 15771 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:39.726168 15731 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.384s	user 0.008s	sys 0.024s
I20260709 13:04:39.809935 15492 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:39.739368 14019 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.595s	user 0.000s	sys 0.015s
I20260709 13:04:39.830031 15812 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:39.739916 14020 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 2: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:04:39.761868 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 140628us
I20260709 13:04:39.740317 13875 rpcz_store.cc:276] 0709 13:04:38.295830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.620238 (+1324408us) service_pool.cc:224] Handling call
0709 13:04:39.620615 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:39.620845 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.620876 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:39.620924 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:39.620967 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.621029 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.621151 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:40.002187 15811 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.004375 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2115) took 1242 ms. Trace:
I20260709 13:04:40.004779 13875 rpcz_store.cc:276] 0709 13:04:38.762091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.003446 (+1241355us) service_pool.cc:224] Handling call
0709 13:04:40.003830 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.004034 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.004064 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.004107 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:40.004150 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.004205 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.004317 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:39.740134 14523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.123s	user 0.002s	sys 0.013s
I20260709 13:04:39.770908 13871 rpcz_store.cc:276] 0709 13:04:37.130428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.130615 (+   187us) service_pool.cc:224] Handling call
0709 13:04:37.159514 (+ 28899us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:37.877205 (+717691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.877238 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:37.892269 (+ 15031us) log.cc:844] Serialized 26 byte log entry
0709 13:04:38.158884 (+266615us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:38.158931 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.187614 (+ 28683us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54002e77d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:38.187670 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:39.577237 (+1389567us) raft_consensus.cc:1685] finished
0709 13:04:39.577259 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.617366 (+ 40107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70355,"raft.queue_time_us":102061,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":183,"spinlock_wait_cycles":1487614336,"thread_start_us":250322,"threads_started":1,"wal-append.queue_time_us":1581021}
I20260709 13:04:39.919395 15722 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:39.829667 15792 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:39.855820 15839 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:40.008508 15839 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.008706 15792 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:39.776309 13866 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:40.009662 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2112) took 1278 ms. Trace:
I20260709 13:04:39.647032 15602 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:40.009979 13866 rpcz_store.cc:276] 0709 13:04:38.730904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.775480 (+1044576us) service_pool.cc:224] Handling call
0709 13:04:39.775876 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.009375 (+233499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.009602 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:39.705935 15713 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.009943 15582 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:40.015509 15866 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:40.015445 15602 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:39.743616 15617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.816s	user 0.067s	sys 0.128s
I20260709 13:04:39.709503 15804 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:39.936194 15781 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:39.806200 13873 rpcz_store.cc:276] 0709 13:04:38.080125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.080249 (+   124us) service_pool.cc:224] Handling call
0709 13:04:38.080510 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:38.816679 (+736169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:38.816728 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:38.816788 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:38.816847 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.842862 (+ 26015us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b540026d9d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:38.842933 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.598469 (+755536us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1692313472}
I20260709 13:04:39.831851 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:40.019892 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:40.020469 13946 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:40.021291 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:39.901516 15841 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:39.853626 13868 rpcz_store.cc:276] 0709 13:04:38.293424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.296273 (+  2849us) service_pool.cc:224] Handling call
0709 13:04:39.679410 (+1383137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:39.854465 14016 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:40.023159 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1218) took 1270 ms. Trace:
I20260709 13:04:40.023515 14016 rpcz_store.cc:276] 0709 13:04:38.752906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.784969 (+ 32063us) service_pool.cc:224] Handling call
0709 13:04:40.023076 (+1238107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6784}
I20260709 13:04:40.023828 15841 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.024611 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:40.025350 14016 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
W20260709 13:04:39.853878 14024 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.009s	sys 0.015s
I20260709 13:04:39.636160 15837 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:39.834964 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:39.844857 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:39.802443 15848 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.676771 15683 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:39.860137 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 259380us
I20260709 13:04:39.860356 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:40.037173 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:40.037657 13947 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:39.798678 15815 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:39.742798 15806 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:39.679368 15763 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } }
W20260709 13:04:39.866173 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51963.5us
I20260709 13:04:40.047395 13838 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:39.779948 15685 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:39.786180 15701 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:39.810591 15684 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:39.785861 15628 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:39.738145 15772 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:39.772989 13872 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.977s	user 0.022s	sys 0.009s
I20260709 13:04:39.829025 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:39.966075 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:39.983736 14023 rpcz_store.cc:276] 0709 13:04:38.424919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.426708 (+  1789us) service_pool.cc:224] Handling call
0709 13:04:39.736427 (+1309719us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:39.901862 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 85697.7us
W20260709 13:04:39.876886 14015 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.007s	sys 0.021s
W20260709 13:04:39.962191 13870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 1938) took 2807 ms (client timeout 2388 ms). Trace:
I20260709 13:04:39.797364 15843 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:39.813023 15674 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:39.810127 15760 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:39.974236 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2101) took 2564 ms. Trace:
I20260709 13:04:39.876936 14018 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:39.961648 15867 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:39.993484 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 175) took 2174 ms. Trace:
W20260709 13:04:39.922591 15785 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.190s	user 0.014s	sys 0.002s
I20260709 13:04:40.202140 13867 rpcz_store.cc:276] 0709 13:04:37.410260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.410465 (+   205us) service_pool.cc:224] Handling call
0709 13:04:37.411027 (+   562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:37.625808 (+214781us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.635776 (+  9968us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:37.767000 (+131224us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.579122 (+1812122us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:39.579225 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.579306 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:39.973973 (+394667us) raft_consensus.cc:1685] finished
0709 13:04:39.973994 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.974148 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":212685,"raft.queue_time_us":223738,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":119,"thread_start_us":1752256,"threads_started":1,"wal-append.queue_time_us":2176424}
I20260709 13:04:39.989986 14522 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:04:40.171207 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:40.205298 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 215365us
I20260709 13:04:40.201611 14019 rpcz_store.cc:276] 0709 13:04:37.808976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.812764 (+  3788us) service_pool.cc:224] Handling call
0709 13:04:37.813025 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:37.813204 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:38.143791 (+330587us) log.cc:844] Serialized 14 byte log entry
0709 13:04:39.983109 (+1839318us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:39.983199 (+    90us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:39.983305 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.983432 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.983605 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1082528,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":715,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":1941,"thread_start_us":2114675,"threads_started":2,"wal-append.queue_time_us":1826986}
I20260709 13:04:39.966780 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:39.920636 15851 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 13:04:40.209407 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:40.210206 14018 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:39.962630 15662 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:39.901603 15862 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.858455 15591 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:39.829474 15835 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:39.867899 15790 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:39.869216 15766 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:40.236492 15766 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:39.858462 15801 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:40.238432 15801 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:40.241604 15801 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:39.856159 15692 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:39.962860 15764 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:39.997243 15812 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:39.712154 15670 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.061s	user 0.018s	sys 0.013s
I20260709 13:04:40.010488 15635 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:39.785296 13840 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.056s	user 0.016s	sys 0.018s
I20260709 13:04:40.012681 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:39.876938 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:40.015429 15713 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:40.015765 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:39.857082 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 149809us
I20260709 13:04:40.028075 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:40.021955 13946 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:40.031533 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:40.035562 14021 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:40.035673 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:39.900539 15860 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.958626 15556 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:40.036612 15799 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:04:40.040820 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:40.041093 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:40.042372 15837 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:40.039508 15848 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:40.052001 13838 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:39.852907 15768 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:40.166985 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 244) took 3072 ms. Trace:
I20260709 13:04:39.934010 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36586 (request call id 23) took 1096 ms. Trace:
I20260709 13:04:40.072440 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:40.202924 13989 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:40.163552 13870 rpcz_store.cc:269] 0709 13:04:37.154972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.197250 (+ 42278us) service_pool.cc:224] Handling call
0709 13:04:39.959154 (+2761904us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002e5ed4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:39.962084 (+  2930us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2654161152}
I20260709 13:04:40.218600 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:40.219226 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1220) took 1325 ms. Trace:
I20260709 13:04:40.221936 14022 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:40.222654 13942 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:40.309885 14016 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:40.277841 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 230768us
W20260709 13:04:40.311015 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 1.58195e+06us
I20260709 13:04:40.312996 14554 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:40.225077 14019 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:40.073222 15504 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:40.228375 15701 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:40.332576 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 304602us
I20260709 13:04:40.311950 13944 rpcz_store.cc:276] 0709 13:04:37.094589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.043348 (+948759us) service_pool.cc:224] Handling call
0709 13:04:38.043669 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:39.600405 (+1556736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.166574 (+566169us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b540004bfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:40.166889 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577485696}
I20260709 13:04:40.309969 13941 rpcz_store.cc:276] 0709 13:04:38.804467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.612080 (+807613us) service_pool.cc:224] Handling call
0709 13:04:39.901154 (+289074us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:40.335041 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:40.230113 15772 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:39.868126 15711 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3.
I20260709 13:04:40.242681 15683 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.351825 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:40.053033 15763 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:40.337863 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 135044us
I20260709 13:04:40.313951 14017 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:40.138782 15806 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.309484 13948 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
W20260709 13:04:40.362421 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 310480us
I20260709 13:04:40.345973 13771 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57642 (request call id 270) took 5016 ms. Trace:
I20260709 13:04:40.215514 15858 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:40.362968 14016 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:40.214669 13943 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
W20260709 13:04:40.336999 14524 connection.cc:256] RPC call timeout handler was delayed by -1.1176076039999998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
I20260709 13:04:40.367846 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:40.369076 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:40.051250 15815 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:40.354647 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:40.347196 15801 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
W20260709 13:04:40.415218 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.248525 15766 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.331125 15591 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.355780 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:40.418284 15780 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.419005 15780 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:40.215924 15785 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:40.336759 15836 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:04:40.423470 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:40.425452 15685 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.366693 13947 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:40.426609 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:40.427616 14561 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:40.430644 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:40.431414 14560 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:40.367324 15858 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:40.432610 15858 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:40.223857 15862 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:40.267125 15492 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.073381 15710 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.277s
I20260709 13:04:40.439198 13945 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 3: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:40.348438 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:40.441331 13944 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:40.220583 15674 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:40.387796 14022 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:40.338318 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:40.364099 13838 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:40.351752 15839 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:40.331943 15504 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:40.337713 15860 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:40.409958 15847 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:40.413332 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:40.369906 14556 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:40.249418 15803 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:40.247460 15835 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:40.377601 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:40.487353 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 149116us
W20260709 13:04:40.253101 15863 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.292s	user 0.002s	sys 0.000s
I20260709 13:04:40.358834 15628 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.351542 15867 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:40.523052 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:40.535460 15862 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:40.816800 15812 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.859319 15692 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:40.529852 15504 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:40.855268 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:40.862671 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 485135us
I20260709 13:04:40.863286 14559 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:40.866861 15685 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:40.485862 15771 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:40.866978 15709 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:40.867425 15685 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.867630 15771 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:40.867882 15685 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:40.868124 15771 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:40.873591 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:40.875874 15668 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.876231 15867 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:40.876700 15867 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:40.876540 15668 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:40.879034 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1216) took 2450 ms. Trace:
I20260709 13:04:40.879452 14015 rpcz_store.cc:276] 0709 13:04:38.428464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.429318 (+   854us) service_pool.cc:224] Handling call
0709 13:04:38.544493 (+115175us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:38.584056 (+ 39563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:38.584092 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:38.614620 (+ 30528us) log.cc:844] Serialized 26 byte log entry
0709 13:04:40.201989 (+1587369us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:40.221285 (+ 19296us) log.cc:844] Serialized 14 byte log entry
0709 13:04:40.221535 (+   250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.221608 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.252262 (+ 30654us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:04:40.535161 (+282899us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b180039d2a0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:04:40.878792 (+343631us) raft_consensus.cc:1685] finished
0709 13:04:40.878817 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.878965 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119241,"prepare.queue_time_us":18898,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":723134,"raft.run_cpu_time_us":476,"raft.run_wall_time_us":97295,"spinlock_wait_cycles":77747200,"thread_start_us":1223221,"threads_started":1,"wal-append.queue_time_us":1062193}
W20260709 13:04:40.878717 13840 connection.cc:256] RPC call timeout handler was delayed by -1.4051944049999747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19830000000000003s for a response
W20260709 13:04:40.881968 13840 connection.cc:256] RPC call timeout handler was delayed by -1.3871944050000593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
I20260709 13:04:40.522261 15785 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:40.884475 13840 connection.cc:256] RPC call timeout handler was delayed by -1.3421944049999865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
I20260709 13:04:40.891558 15711 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:04:40.843308 14018 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.010s	sys 0.032s
W20260709 13:04:40.865751 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 501699us
I20260709 13:04:40.892580 15711 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:40.893395 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:40.894750 15591 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:40.895358 15591 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:04:40.895489 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2170944050000116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
I20260709 13:04:40.895787 15591 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:40.896072 13840 connection.cc:256] RPC call timeout handler was delayed by -1.172994405000054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22410000000000002s for a response
I20260709 13:04:40.896607 15763 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:40.897123 15763 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:40.898936 15771 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:40.883471 15668 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.899196 15763 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:40.641453 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.899153 13838 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:40.901868 13838 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:40.444443 15772 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:40.902729 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 552139us
I20260709 13:04:40.687415 15860 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:40.905920 14018 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.010s	sys 0.034s
W20260709 13:04:40.907976 14558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.008s	sys 0.011s
I20260709 13:04:40.423532 15713 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.863633 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:40.860464 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:40.443907 15790 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:40.363435 13771 rpcz_store.cc:276] 0709 13:04:35.306363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.306651 (+   288us) service_pool.cc:224] Handling call
0709 13:04:40.323010 (+5016359us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:37.575121 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:37.910448 (+335327us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:37.910508 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:04:37.910534 (+    26us) tablet.cc:662] Decoding operations
0709 13:04:37.911128 (+   594us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:37.911233 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:37.911277 (+    44us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:37.911593 (+   316us) tablet.cc:701] Row locks acquired
0709 13:04:37.911631 (+    38us) write_op.cc:260] PREPARE: finished
0709 13:04:37.911919 (+   288us) write_op.cc:270] Start()
0709 13:04:37.912094 (+   175us) write_op.cc:276] Timestamp: P: 1783602277911828 usec, L: 0
0709 13:04:37.912133 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:04:37.913161 (+  1028us) log.cc:844] Serialized 541 byte log entry
0709 13:04:39.917172 (+2004011us) op_driver.cc:464] REPLICATION: finished
0709 13:04:40.118576 (+201404us) write_op.cc:301] APPLY: starting
0709 13:04:40.118735 (+   159us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:40.137420 (+ 18685us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:40.137581 (+   161us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:40.137924 (+   343us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:40.138493 (+   569us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:04:40.138539 (+    46us) write_op.cc:312] APPLY: finished
0709 13:04:40.138775 (+   236us) log.cc:844] Serialized 23 byte log entry
0709 13:04:40.138959 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:40.139061 (+   102us) write_op.cc:454] Released schema lock
0709 13:04:40.139099 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:40.139557 (+   458us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:40.139590 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1376279,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":153811,"apply.run_cpu_time_us":2563,"apply.run_wall_time_us":43965,"num_ops":1,"prepare.queue_time_us":293592,"prepare.run_cpu_time_us":25507,"prepare.run_wall_time_us":779605,"raft.queue_time_us":262836,"raft.run_cpu_time_us":8170,"raft.run_wall_time_us":119364,"replication_time_us":2004820,"thread_start_us":642163,"threads_started":4,"wal-append.queue_time_us":1571300}]]}
W20260709 13:04:40.930562 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 67044.1us
W20260709 13:04:40.912407 14558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.008s	sys 0.011s
I20260709 13:04:40.524907 15770 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:40.934674 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:40.935128 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1261) took 1119 ms. Trace:
I20260709 13:04:40.935678 13950 rpcz_store.cc:276] 0709 13:04:39.816204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.826289 (+ 10085us) service_pool.cc:224] Handling call
0709 13:04:39.826643 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.047715 (+221072us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:40.095845 (+ 48130us) log.cc:844] Serialized 14 byte log entry
0709 13:04:40.222405 (+126560us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.222465 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:40.223493 (+  1028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.934813 (+711320us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400088254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:40.934885 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.935044 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":657,"prepare.queue_time_us":53900,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":56,"raft.queue_time_us":762,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":46602,"spinlock_wait_cycles":509376640,"thread_start_us":125448,"threads_started":1,"wal-append.queue_time_us":592557}
I20260709 13:04:40.313040 14024 rpcz_store.cc:276] 0709 13:04:38.895433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.939969 (+ 44536us) service_pool.cc:224] Handling call
0709 13:04:38.966264 (+ 26295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:38.966620 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:40.026419 (+1059799us) log.cc:844] Serialized 14 byte log entry
0709 13:04:40.218641 (+192222us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.218684 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:40.219030 (+   346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.219080 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.219174 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63537,"prepare.queue_time_us":146180,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":15444,"raft.queue_time_us":419,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":5698,"thread_start_us":1186005,"threads_started":2}
W20260709 13:04:40.943048 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 82668.3us
I20260709 13:04:40.944926 13987 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:40.946672 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:40.948215 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:40.485678 15662 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.483s
I20260709 13:04:40.485388 15764 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.435s
I20260709 13:04:40.927644 15668 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:40.855509 15848 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.949100 15692 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:40.957677 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2114) took 2196 ms. Trace:
I20260709 13:04:40.910879 15772 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.961870 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:40.964823 13869 rpcz_store.cc:276] 0709 13:04:38.761746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.862407 (+1100661us) service_pool.cc:224] Handling call
0709 13:04:39.864839 (+  2432us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:40.047709 (+182870us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.047744 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:40.071472 (+ 23728us) log.cc:844] Serialized 590 byte log entry
0709 13:04:40.230811 (+159339us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:40.230859 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.957189 (+726330us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54002e7cd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:40.957241 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.957311 (+    70us) raft_consensus.cc:1685] finished
0709 13:04:40.957324 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.957592 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2992,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":46960,"spinlock_wait_cycles":419153792,"thread_start_us":154838,"threads_started":1,"wal-append.queue_time_us":154928}
I20260709 13:04:40.958843 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1269) took 1115 ms. Trace:
I20260709 13:04:40.956384 15790 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:40.977116 13942 rpcz_store.cc:276] 0709 13:04:39.843522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.844544 (+  1022us) service_pool.cc:224] Handling call
0709 13:04:40.958661 (+1114117us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400087854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:40.958780 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429436672}
I20260709 13:04:40.929136 15556 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.920842 15763 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:40.928789 15792 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:40.967440 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:40.927305 15617 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.960007 15835 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:40.950032 15764 consensus_queue.cc:1243] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Peer 4dd265d61fc141a48ca9d3d5c8f52921 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:40.942982 15836 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:40.950488 15668 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:40.963685 15803 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:40.966481 15837 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:40.966773 15760 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:40.982661 15772 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:41.036947 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 102356us
I20260709 13:04:40.996222 15763 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:40.964751 15692 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:40.976723 15815 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:40.994725 15722 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:40.996012 15792 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.999159 15801 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:40.983956 15866 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:04:41.011863 15843 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:41.026768 15668 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:41.060061 15763 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:41.058521 15692 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:41.039337 15835 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:41.012190 15617 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:41.078236 15792 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.062839 15635 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:41.092664 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.094154 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.127166 13950 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
W20260709 13:04:41.091589 13915 connection.cc:256] RPC call timeout handler was delayed by -2.6794181669999944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
W20260709 13:04:41.127761 13915 connection.cc:256] RPC call timeout handler was delayed by -2.6650181669999711s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
I20260709 13:04:41.128355 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.096463 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2107) took 2801 ms. Trace:
I20260709 13:04:41.009562 15556 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.129880 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.133342 13950 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:41.084990 15760 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:41.089294 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1228) took 1476 ms. Trace:
W20260709 13:04:41.118695 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 303889us
W20260709 13:04:41.135492 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 209666us
I20260709 13:04:41.061733 15722 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:41.138654 15722 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:41.134965 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.111663 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.136382 15768 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:41.098182 15628 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.082917 15801 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.332433 13950 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:41.348268 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.356644 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.130399 13872 rpcz_store.cc:276] 0709 13:04:38.295120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.760060 (+464940us) service_pool.cc:224] Handling call
0709 13:04:38.795339 (+ 35279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:38.795603 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:04:40.207469 (+1411866us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.096094 (+888625us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.096154 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:41.096221 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.096291 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.096405 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":382590,"prepare.queue_time_us":632312,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":493961,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":5569,"thread_start_us":1912176,"threads_started":3,"wal-append.queue_time_us":735230}
I20260709 13:04:41.378410 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:41.141275 13988 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:04:41.382254 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 241035us
I20260709 13:04:41.383579 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:41.139954 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:41.387238 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.402487 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:41.274993 15883 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
W20260709 13:04:41.276716 15881 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 13:04:41.408980 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:41.386093 14018 rpcz_store.cc:276] 0709 13:04:39.612493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.210916 (+598423us) service_pool.cc:224] Handling call
0709 13:04:40.211268 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.211535 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:40.911860 (+700325us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.088772 (+176912us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.088849 (+    77us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:41.088932 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.089032 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.089215 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10562,"prepare.queue_time_us":125504,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":5751,"raft.queue_time_us":4666,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":25216,"thread_start_us":777174,"threads_started":2}
I20260709 13:04:41.417623 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.269587 15771 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:41.473119 15722 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:41.347652 15839 raft_consensus.cc:697] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:41.181408 15635 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:41.401856 15835 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:41.518934 15835 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:41.518416 15839 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.519345 15835 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.518414 15635 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:41.482772 15860 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:41.139003 15760 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:41.505717 15891 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.526556 15891 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.525384 15860 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.247345 15867 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:41.534024 14558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 481) took 1659 ms. Trace:
I20260709 13:04:41.534688 14558 rpcz_store.cc:276] 0709 13:04:39.874510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.876730 (+  2220us) service_pool.cc:224] Handling call
0709 13:04:39.919113 (+ 42383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.185439 (+266326us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:04:40.185488 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:40.203965 (+ 18477us) log.cc:844] Serialized 26 byte log entry
0709 13:04:40.932975 (+729010us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:40.933019 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.933094 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.533677 (+600583us) raft_consensus.cc:1685] finished
0709 13:04:41.533713 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.533930 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43213,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":201,"thread_start_us":869731,"threads_started":2,"wal-append.queue_time_us":859106}
I20260709 13:04:41.537210 15891 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:41.537815 15891 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.538305 15891 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.540398 15891 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:41.541181 15760 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:41.544389 15674 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.020s	user 0.011s	sys 0.002s
W20260709 13:04:41.546653 15685 consensus_meta.cc:287] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.678s	user 0.014s	sys 0.003s
I20260709 13:04:41.547364 15685 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:41.513590 15801 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:41.402256 15713 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:41.548910 15713 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.549326 15713 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.551280 15867 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:41.551792 15860 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:41.552886 15591 consensus_meta.cc:287] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.657s	user 0.011s	sys 0.000s
I20260709 13:04:41.517060 15556 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:41.515365 15884 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:04:41.555316 15504 consensus_meta.cc:287] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.693s	user 0.005s	sys 0.006s
I20260709 13:04:41.555826 15504 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:41.558094 15504 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:41.402748 15628 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.559038 15713 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.560391 15628 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:41.553579 15591 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:41.561919 15713 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:41.562563 15591 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:41.564415 15790 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.580s	user 0.007s	sys 0.004s
I20260709 13:04:41.565073 15790 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:41.567528 15685 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:41.567255 15835 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:41.565228 15858 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.132s	user 0.016s	sys 0.001s
I20260709 13:04:41.450332 15768 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:04:41.596194 15701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.000s	sys 0.014s
I20260709 13:04:41.474742 15890 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:04:41.617678 15668 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.525s	user 0.003s	sys 0.011s
I20260709 13:04:41.604835 15768 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:04:41.715627 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 547385us
I20260709 13:04:41.809836 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:41.809376 15668 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:41.811020 13869 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:41.638159 15692 consensus_meta.cc:287] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.008s
I20260709 13:04:41.812860 15692 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:41.717381 13871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.005s	sys 0.026s
I20260709 13:04:41.098388 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:04:41.814173 13871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.006s	sys 0.027s
I20260709 13:04:41.814560 15713 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:41.815646 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 225) took 1449 ms. Trace:
I20260709 13:04:41.349444 13944 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:41.816433 15692 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:41.607656 15701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.000s	sys 0.015s
I20260709 13:04:41.628644 15722 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:41.818246 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.728049 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 255) took 1758 ms. Trace:
W20260709 13:04:41.728978 13987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.005s	sys 0.009s
I20260709 13:04:41.730141 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.732131 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.735710 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1262) took 1919 ms. Trace:
W20260709 13:04:41.736570 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:41.737515 13947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1271) took 1868 ms (client timeout 1862 ms). Trace:
I20260709 13:04:41.722700 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2111) took 2992 ms. Trace:
W20260709 13:04:41.746354 13989 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.007s	sys 0.010s
W20260709 13:04:41.747756 13945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 1910) took 4356 ms (client timeout 1544 ms). Trace:
I20260709 13:04:41.750223 13870 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:41.648208 15835 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:41.753173 14022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1210) took 3402 ms (client timeout 2241 ms). Trace:
I20260709 13:04:41.767354 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:41.770717 15617 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:41.781616 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:41.782886 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:41.795305 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 298) took 1383 ms. Trace:
I20260709 13:04:41.797307 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:41.800587 13868 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:41.800561 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:41.802510 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:41.838439 14559 raft_consensus.cc:2410] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:04:41.838281 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 71439.2us
I20260709 13:04:41.804271 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:41.840054 14551 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
W20260709 13:04:41.721432 13915 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.001s	sys 0.001s
I20260709 13:04:41.841125 15685 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:41.815593 15713 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:41.720362 13914 leader_election.cc:336] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.651s (SENT)
W20260709 13:04:41.844527 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 124510us
I20260709 13:04:41.845005 15713 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:41.845743 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:41.640663 15772 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.580s	user 0.017s	sys 0.000s
I20260709 13:04:41.851061 15772 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:41.627039 15792 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:41.816453 13871 rpcz_store.cc:276] 0709 13:04:40.366315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.367145 (+   830us) service_pool.cc:224] Handling call
0709 13:04:40.429813 (+ 62668us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:40.862590 (+432777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.862625 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:40.949597 (+ 86972us) log.cc:844] Serialized 590 byte log entry
0709 13:04:41.815040 (+865443us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:41.815100 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.815212 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.815276 (+    64us) raft_consensus.cc:1685] finished
0709 13:04:41.815288 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.815556 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57993,"raft.queue_time_us":435875,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":126,"thread_start_us":864737,"threads_started":1,"wal-append.queue_time_us":579054}
I20260709 13:04:41.854689 15722 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:41.725684 13942 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:41.813832 15668 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:41.604347 15858 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.619303 15890 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.870488 15772 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:41.751044 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1223) took 2395 ms. Trace:
I20260709 13:04:41.871783 14553 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:41.872014 14019 rpcz_store.cc:276] 0709 13:04:39.355831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.216567 (+860736us) service_pool.cc:224] Handling call
0709 13:04:40.217023 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.314497 (+ 97474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.749905 (+1435408us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b5400412654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:41.750616 (+   711us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17701120}
W20260709 13:04:41.828071 13989 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.007s	sys 0.010s
W20260709 13:04:41.821779 13947 rpcz_store.cc:269] 0709 13:04:39.868796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.040544 (+171748us) service_pool.cc:224] Handling call
0709 13:04:41.736945 (+1696401us) spinlock_profiling.cc:243] Waited 56.9 ms on lock 0x7b5400090454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:41.737212 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130897792}
I20260709 13:04:41.822503 13870 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:04:41.826040 13987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.006s	sys 0.009s
I20260709 13:04:41.821275 13949 rpcz_store.cc:276] 0709 13:04:39.816527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.826898 (+ 10371us) service_pool.cc:224] Handling call
0709 13:04:39.827206 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.209687 (+382481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.209719 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.209792 (+    73us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:40.209835 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.734998 (+1525163us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54001c6e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:41.735097 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.735422 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879085312}
I20260709 13:04:41.822149 13874 rpcz_store.cc:276] 0709 13:04:38.730545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.683365 (+952820us) service_pool.cc:224] Handling call
0709 13:04:39.683828 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.238339 (+554511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.238385 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.238445 (+    60us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:40.238513 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.721707 (+1483194us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54001aac54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:41.721866 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.722368 (+   502us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1274554624}
W20260709 13:04:41.841501 13915 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.002s	sys 0.001s
I20260709 13:04:41.724364 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 252) took 1755 ms. Trace:
I20260709 13:04:41.724666 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.846451 14555 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:04:41.846637 15768 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:41.847236 13873 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 2: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:41.847692 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:41.847896 14561 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
W20260709 13:04:41.835196 14022 rpcz_store.cc:269] 0709 13:04:38.350350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.351262 (+   912us) service_pool.cc:224] Handling call
0709 13:04:41.752625 (+3401363us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54003250d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:41.752884 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485568256}
W20260709 13:04:41.822351 13945 rpcz_store.cc:269] 0709 13:04:37.391356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.391476 (+   120us) service_pool.cc:224] Handling call
0709 13:04:41.747158 (+4355682us) spinlock_profiling.cc:243] Waited 3.02 s on lock 0x7b54000f5354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:41.747472 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6951592832}
I20260709 13:04:41.857977 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:41.858480 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:41.858706 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:41.846184 14552 rpcz_store.cc:276] 0709 13:04:40.411169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.424800 (+ 13631us) service_pool.cc:224] Handling call
0709 13:04:41.784696 (+1359896us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b5800004898. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:04:41.785184 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.785826 (+   642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.785876 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.785958 (+    82us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:41.786023 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.794695 (+  8672us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.794955 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14276480}
I20260709 13:04:41.880218 15890 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:41.881279 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 913825us
I20260709 13:04:41.882478 13950 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:41.820830 13943 rpcz_store.cc:276] 0709 13:04:39.969146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.409584 (+440438us) service_pool.cc:224] Handling call
0709 13:04:40.410076 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.905756 (+495680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.926101 (+ 20345us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:40.968291 (+ 42190us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.726703 (+758412us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:41.727020 (+   317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.727226 (+   206us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.727361 (+   135us) raft_consensus.cc:1685] finished
0709 13:04:41.727389 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.727785 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":166536,"raft.queue_time_us":491923,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":13287,"thread_start_us":381250,"threads_started":1,"wal-append.queue_time_us":558552}
I20260709 13:04:41.867983 13875 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:04:41.883994 13868 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:41.885442 14554 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:41.881916 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 753605us
I20260709 13:04:41.889216 13872 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:41.906819 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:41.912057 15711 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:41.913077 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:41.915228 13946 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.855464 15685 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:41.897316 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 920920us
I20260709 13:04:41.918367 14561 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:41.874782 13941 rpcz_store.cc:276] 0709 13:04:39.968458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.343114 (+374656us) service_pool.cc:224] Handling call
0709 13:04:40.343739 (+   625us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:40.445821 (+102082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.445867 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:40.532641 (+ 86774us) log.cc:844] Serialized 590 byte log entry
0709 13:04:41.031308 (+498667us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:41.031369 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.031489 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.723451 (+691962us) raft_consensus.cc:1685] finished
0709 13:04:41.723515 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.723941 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":458509,"raft.queue_time_us":130771,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":19906,"thread_start_us":230557,"threads_started":1,"wal-append.queue_time_us":582265}
I20260709 13:04:41.930639 13870 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.931979 13870 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:41.932475 15721 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:41.932981 15721 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:04:41.932348 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 75686.7us
I20260709 13:04:41.881904 15765 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:41.939289 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:41.940711 15692 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:41.942394 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:41.946312 13915 connection.cc:256] RPC call timeout handler was delayed by -2.4832181670000182s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19230000000000003s for a response
I20260709 13:04:41.947829 15858 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:41.948777 13915 connection.cc:256] RPC call timeout handler was delayed by -1.3075502799999867s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1638s for a response
W20260709 13:04:41.949234 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0780502799999567s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18930000000000002s for a response
I20260709 13:04:41.963424 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:41.965621 15764 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:41.966455 15908 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:41.967129 15908 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.966562 15764 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.967504 15908 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.970551 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:41.971555 13872 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:41.972427 14553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 297) took 1561 ms. Trace:
I20260709 13:04:41.972903 14553 rpcz_store.cc:276] 0709 13:04:40.410856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.410957 (+   101us) service_pool.cc:224] Handling call
0709 13:04:40.959029 (+548072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.872399 (+913370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.957841 (+ 85442us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b54004b0fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:41.972356 (+ 14515us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2088776448}
I20260709 13:04:41.975677 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:41.975932 15768 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.976446 13871 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:41.981498 15685 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:41.965801 15876 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.882s	user 0.007s	sys 0.018s
I20260709 13:04:41.988981 15766 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:42.001362 15721 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:41.983004 15908 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:42.010250 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.010722 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.014024 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:42.019389 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.022670 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.027136 15836 raft_consensus.cc:2955] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:42.029930 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:42.030501 15780 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:42.081317 15721 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:42.031324 13948 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:42.082854 13869 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:04:42.083159 13873 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:42.083846 13867 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:42.083909 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.084146 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.067637 15766 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:42.085585 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:42.032207 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:42.038619 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.086697 15860 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:42.086937 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.087597 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.087484 15860 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:42.088218 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.088704 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.088897 13872 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:42.039399 15792 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:42.041239 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.090894 13874 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:42.038367 15764 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:42.052611 15763 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.792s	user 0.013s	sys 0.010s
I20260709 13:04:42.055292 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:42.073321 14554 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:42.082360 15908 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:42.084834 13869 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:42.089850 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:42.050889 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44924.3us
I20260709 13:04:42.089936 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.106194 13871 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:42.103015 13946 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:04:42.104259 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:42.108719 14555 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:42.110675 13988 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:42.112186 13873 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:42.111936 15763 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:42.116971 13840 leader_election.cc:336] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.536s (SENT)
I20260709 13:04:42.122627 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:42.124449 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.099644 15860 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:42.141271 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 86477.3us
W20260709 13:04:42.142531 13838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.012s	sys 0.009s
I20260709 13:04:42.153512 13870 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:04:42.153950 13870 pending_rounds.cc:85] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 5
W20260709 13:04:42.156257 13838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.057s	user 0.012s	sys 0.010s
I20260709 13:04:42.139714 15768 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:42.158159 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.140043 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:42.155738 13870 pending_rounds.cc:107] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:04:42.163100 13870 raft_consensus.cc:2889] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:42.163316 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:42.085976 15766 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:42.166407 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.171216 15910 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:42.171509 13773 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57626 (request call id 244) took 5769 ms. Trace:
W20260709 13:04:42.191182 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 87176.8us
I20260709 13:04:42.176649 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1285) took 1033 ms. Trace:
W20260709 13:04:42.177565 15710 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:04:42.178123 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 221) took 1841 ms. Trace:
I20260709 13:04:42.179627 13771 catalog_manager.cc:5697] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:42.182950 13759 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 545fc4aeb71d491985e296a0721d905d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d1b4fd04200247d99ba9ad1fbeb42b42 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:42.173782 13871 raft_consensus.cc:2955] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:42.185235 15635 consensus_queue.cc:1059] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:42.186103 15764 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:42.186501 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 86513.2us
I20260709 13:04:42.211180 15909 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:42.212924 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:42.192538 13870 rpcz_store.cc:276] 0709 13:04:40.336237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.337120 (+   883us) service_pool.cc:224] Handling call
0709 13:04:40.337430 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.749888 (+1412458us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b54004a86a0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33aadc0 00007fb2c34351c0 00007fb2c3436231 00007fb2c3432d8f 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c
0709 13:04:42.177179 (+427291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.178034 (+   855us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1552682752}
I20260709 13:04:42.162766 15792 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:04:42.211665 13840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.580s (SENT) [suppressed 24 similar messages]
W20260709 13:04:42.218532 13838 connection.cc:256] RPC call timeout handler was delayed by -1.1068656649999866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
I20260709 13:04:42.192158 13945 rpcz_store.cc:276] 0709 13:04:41.130104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.908346 (+778242us) service_pool.cc:224] Handling call
0709 13:04:41.908943 (+   597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.909369 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:41.909851 (+   482us) log.cc:844] Serialized 14 byte log entry
0709 13:04:42.163422 (+253571us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:42.163492 (+    70us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:42.163563 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.163651 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.163816 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11764,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":4216,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":207,"thread_start_us":236361,"threads_started":1}
W20260709 13:04:42.236045 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 45728.2us
I20260709 13:04:42.190410 15910 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:42.239568 13759 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 56938.2us
I20260709 13:04:42.192287 15860 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:04:42.242949 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:42.237979 15763 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:42.247290 15858 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:42.247813 15763 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.248389 15763 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:42.254954 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.255720 13866 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
W20260709 13:04:42.259671 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.240563 15792 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:42.243526 15635 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:42.244786 15910 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:42.257973 15858 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:42.218659 13771 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 5 to 6, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 6 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:42.279031 15768 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:42.318310 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 260154us
W20260709 13:04:42.321730 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 83204.3us
I20260709 13:04:42.325771 13870 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:42.336959 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.339344 15792 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:42.346210 15635 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:42.347872 15804 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:42.348388 15804 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:42.354202 15763 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:42.357182 15766 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:42.362514 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:42.364271 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.326406 13771 catalog_manager.cc:5697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 1 to 3, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68), now has a pending config: VOTER e1649107c781428692458f10597c5742 (127.12.109.65), VOTER 3779649036364744823fc035dac892c7 (127.12.109.66), VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67), VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 3 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:42.190717 13773 rpcz_store.cc:276] 0709 13:04:36.401735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.410767 (+  9032us) service_pool.cc:224] Handling call
0709 13:04:42.171229 (+5760462us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:40.137602 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:40.138485 (+   883us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:40.138529 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:04:40.138559 (+    30us) tablet.cc:662] Decoding operations
0709 13:04:40.139602 (+  1043us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:40.139708 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:40.139744 (+    36us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:04:40.140497 (+   753us) tablet.cc:701] Row locks acquired
0709 13:04:40.140520 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:04:40.140716 (+   196us) write_op.cc:270] Start()
0709 13:04:40.140839 (+   123us) write_op.cc:276] Timestamp: P: 1783602280140663 usec, L: 0
0709 13:04:40.140869 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:04:40.208902 (+ 68033us) log.cc:844] Serialized 2855 byte log entry
0709 13:04:40.239027 (+ 30125us) op_driver.cc:464] REPLICATION: finished
0709 13:04:40.239650 (+   623us) write_op.cc:301] APPLY: starting
0709 13:04:40.239755 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:40.240022 (+   267us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:40.240154 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:40.241199 (+  1045us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:40.241515 (+   316us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:04:40.241583 (+    68us) write_op.cc:312] APPLY: finished
0709 13:04:40.241957 (+   374us) log.cc:844] Serialized 59 byte log entry
0709 13:04:40.341366 (+ 99409us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:40.360270 (+ 18904us) write_op.cc:454] Released schema lock
0709 13:04:40.440714 (+ 80444us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:40.441187 (+   473us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:40.441237 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":584981,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":386,"apply.run_cpu_time_us":3474,"apply.run_wall_time_us":201480,"num_ops":7,"prepare.queue_time_us":280939,"prepare.run_cpu_time_us":4908,"prepare.run_wall_time_us":71757,"replication_time_us":98014,"thread_start_us":174818,"threads_started":1}]]}
I20260709 13:04:42.365080 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.366712 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.367842 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.368956 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.369705 13873 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:42.338127 15858 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:42.370635 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.370718 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.371486 13870 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:42.372406 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.373126 13870 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:42.363658 13874 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:04:42.376636 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.371516 13869 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:42.373919 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:42.386011 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:04:42.390987 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.391685 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.392956 13868 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:42.400043 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.406687 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:42.408327 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:42.411008 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.398257 13947 raft_consensus.cc:1217] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[3.6-3.6]   Dedup: 3.6->[]
I20260709 13:04:42.415078 13947 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:04:42.419127 13869 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:42.407552 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:42.427042 13948 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:42.427807 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:42.428638 13946 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:04:42.433236 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.434377 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:42.390393 13771 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:42.446534 13941 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:42.440680 13942 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:42.349524 15804 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:42.454718 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.441388 13949 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:42.466924 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:42.435585 15768 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:04:42.476076 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:42.476595 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:42.477566 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:42.477818 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:42.478261 13947 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:42.478559 13870 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:42.479087 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:42.480324 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:42.481410 13839 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:42.481251 13771 catalog_manager.cc:5697] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 2 to 4, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 4 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:42.488319 13947 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:42.489120 13870 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:42.490218 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:42.493016 13870 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:42.494699 15804 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:42.498445 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:42.500061 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:42.500891 13869 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:04:42.502442 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 44407.6us
I20260709 13:04:42.502703 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:42.511269 13950 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:42.515136 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.520110 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:42.522306 15768 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:42.523664 15768 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:42.524132 15768 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:42.524950 15771 raft_consensus.cc:3055] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:42.524610 15812 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:42.525492 15771 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:42.526676 15771 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:42.524612 15768 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:42.528298 15771 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:42.532269 13987 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:42.533828 15770 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:42.536029 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:42.501783 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:42.469445 15909 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 13:04:42.478415 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:42.541031 13873 raft_consensus.cc:2393] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:42.546479 13869 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:42.521708 13989 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:42.555352 15701 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:42.555531 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:42.584024 13941 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:42.586148 13942 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
W20260709 13:04:42.594277 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 72654.6us
W20260709 13:04:42.597719 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 143954us
I20260709 13:04:42.603327 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:42.604650 15909 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:42.605181 15909 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:42.605654 15909 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:42.556324 13943 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:42.625324 13870 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
W20260709 13:04:42.622828 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 340283us
I20260709 13:04:42.641913 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:42.647591 13870 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:42.617070 15685 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:42.664503 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:42.675469 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:42.676613 15713 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:42.677078 15713 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:42.665485 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 129592us
I20260709 13:04:42.679059 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:42.681023 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:42.682087 15892 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:42.667914 15770 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:04:42.682832 15892 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:42.677419 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:42.687393 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 305666us
I20260709 13:04:42.663288 15685 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:42.696161 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.698208 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:42.661669 15710 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:42.677467 15713 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:42.683236 15892 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.711205 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:42.694350 15685 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:42.664896 15835 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:42.646354 15635 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:42.720791 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 72428.5us
I20260709 13:04:42.726130 13950 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:42.645612 15910 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:42.719803 15836 consensus_meta.cc:287] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.532s	user 0.014s	sys 0.000s
I20260709 13:04:42.702433 15701 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:42.713048 15710 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:42.726405 15892 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:42.738888 15862 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:42.763018 15922 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.096s	user 0.000s	sys 0.003s
I20260709 13:04:42.737314 15929 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:04:42.769111 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.769505 13868 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:42.734583 15835 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.756672 14553 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:42.777889 13838 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:42.781867 13948 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:42.775386 15710 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:42.786094 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.767006 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:42.736493 15860 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:04:42.791388 15835 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:42.779500 15910 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:42.820218 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:42.817656 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:42.806638 15812 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:42.834538 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:42.835485 13948 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:42.832084 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:42.833283 15812 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:42.896370 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 49114us
I20260709 13:04:42.879746 15812 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:42.849929 15909 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:42.936532 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 201719us
W20260709 13:04:42.963927 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 232301us
I20260709 13:04:42.930310 15792 consensus_queue.cc:1059] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
W20260709 13:04:42.966666 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 271878us
I20260709 13:04:42.973325 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:42.974249 13948 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 5: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:04:42.965384 15766 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.608s	user 0.007s	sys 0.006s
W20260709 13:04:42.977607 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.981385 14555 raft_consensus.cc:1217] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Deduplicated request from leader. Original: 2.5->[3.6-3.6]   Dedup: 3.6->[]
I20260709 13:04:42.982007 14555 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:04:42.987802 13942 raft_consensus.cc:2955] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:42.959601 15806 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:42.996480 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:43.005725 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:43.003539 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 225743us
I20260709 13:04:43.010908 14561 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:43.016323 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.018162 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:43.000767 15806 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.035283 15806 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:43.036620 15934 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:04:43.025028 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:43.038794 14552 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
W20260709 13:04:43.039130 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.042284 14524 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.964s (SENT)
I20260709 13:04:43.043205 14561 raft_consensus.cc:2955] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:43.025516 15932 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:04:43.026744 15909 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:43.045977 15909 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.046598 15909 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.048844 15909 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:43.048758 15713 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.049511 15909 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.030419 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:43.030833 15928 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:04:43.034576 15933 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:04:43.000541 15770 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:43.061753 14558 raft_consensus.cc:2368] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4.
W20260709 13:04:43.066586 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.037917 15768 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.510s	user 0.012s	sys 0.000s
I20260709 13:04:43.078428 15768 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.069223 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.059674 15770 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:43.081822 15768 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.047310 15635 raft_consensus.cc:2955] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:43.074442 15942 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:43.075367 13838 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:43.076517 15713 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:43.073668 13943 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
W20260709 13:04:43.087886 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 416500us
I20260709 13:04:43.090587 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 236) took 1066 ms. Trace:
W20260709 13:04:43.095430 14524 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.001s (SENT)
I20260709 13:04:43.095993 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:04:43.109433 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.123031 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:43.059542 15909 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.135485 13989 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:04:43.141934 15804 consensus_meta.cc:287] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.647s	user 0.001s	sys 0.014s
W20260709 13:04:43.158497 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 72330.8us
I20260709 13:04:43.134169 15768 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:43.144970 15771 consensus_meta.cc:287] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.616s	user 0.012s	sys 0.004s
I20260709 13:04:43.161993 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:43.166908 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:43.167765 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 128443us
I20260709 13:04:43.175823 13839 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:43.180476 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.187455 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:43.188246 13874 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:43.094306 13867 rpcz_store.cc:276] 0709 13:04:42.023815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.114739 (+ 90924us) service_pool.cc:224] Handling call
0709 13:04:42.123565 (+  8826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.189645 (+ 66080us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:42.378905 (+189260us) log.cc:844] Serialized 14 byte log entry
0709 13:04:43.086946 (+708041us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:43.087009 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:43.087084 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.087150 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.090481 (+  3331us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":403649,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":72,"raft.queue_time_us":339896,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":65,"thread_start_us":815764,"threads_started":3,"wal-append.queue_time_us":511670}
I20260709 13:04:43.183358 15771 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:43.183504 15951 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:43.210675 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 80238.6us
I20260709 13:04:43.212890 13869 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:43.193089 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.221415 13866 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:43.218101 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:43.220328 13988 leader_election.cc:400] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4.
I20260709 13:04:43.223256 13988 leader_election.cc:403] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:04:43.231192 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:43.206365 15947 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:43.213064 15792 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:43.177189 15770 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.291535 15792 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:43.291683 15770 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:43.224586 15804 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.294466 15804 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.296953 15804 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:43.218351 15771 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:43.232555 15930 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:04:43.266443 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 70294.5us
I20260709 13:04:43.254426 15953 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:43.271050 15909 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:43.298583 15953 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:43.299525 15953 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.253876 15946 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:04:43.273857 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48863.2us
I20260709 13:04:43.255393 15948 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:43.302749 15948 consensus_queue.cc:1243] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer 4dd265d61fc141a48ca9d3d5c8f52921 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:43.303325 15948 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.303857 15948 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.189217 15768 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.278450 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:04:43.298357 15892 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.006s
I20260709 13:04:43.306514 14558 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:43.306722 15909 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:43.307127 15909 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:43.307534 15768 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.308109 15909 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.309564 15909 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:43.275978 13870 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:43.279502 15947 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.313997 15947 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:43.218343 15835 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.320977 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:43.321828 13867 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:43.266616 15961 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:04:43.283018 13943 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:43.283595 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.283670 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.285064 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.298759 15771 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.303727 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:43.278219 15710 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:43.272660 15685 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.004s
I20260709 13:04:43.327857 13874 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:43.338876 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:43.342347 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 122624us
I20260709 13:04:43.347127 14558 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:04:43.348610 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:43.349951 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.350219 15801 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:43.350746 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:43.352981 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.350257 15955 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:43.306526 15892 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.354290 13874 raft_consensus.cc:2955] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:43.354338 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:43.356626 15892 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:43.337448 15960 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:43.358856 15960 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:43.359359 15960 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:43.344542 15812 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:43.366173 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.367395 13838 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:43.367844 15812 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:43.367964 15764 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:43.368490 15764 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:43.369807 15949 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:43.328914 15930 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.350857 15801 raft_consensus.cc:697] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:43.365769 15835 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.369437 15764 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.388468 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:43.391978 15764 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:43.393888 15963 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:43.390239 15875 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.396490 15713 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:43.396656 15768 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:43.396315 15875 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.343353 15685 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.400331 15801 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.400921 15875 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:43.401659 14555 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:43.402424 13872 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:43.402704 15685 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.387689 15806 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.403770 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.388494 15930 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:43.408501 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.409919 15930 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:43.411430 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.412056 13949 raft_consensus.cc:2368] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4.
I20260709 13:04:43.412763 15835 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:43.413232 15835 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:43.413605 15946 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:43.414255 15946 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:43.414145 15835 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.376325 15948 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.418533 15941 raft_consensus.cc:3055] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:43.418992 15941 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:43.419837 14555 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:04:43.419978 15941 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.409792 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.422469 13950 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
W20260709 13:04:43.424053 15950 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.237s	user 0.006s	sys 0.002s
I20260709 13:04:43.431568 15806 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:43.465015 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:43.466043 13872 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:04:43.476632 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:43.477438 13866 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
W20260709 13:04:43.478238 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 89199.2us
W20260709 13:04:43.478392 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 218936us
I20260709 13:04:43.332999 15940 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 13:04:43.486608 15792 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4.
I20260709 13:04:43.487124 15770 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:43.487560 15770 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:43.486632 15947 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.530395 15947 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.509987 15770 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.574342 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:43.574398 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:43.575517 14558 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:43.569813 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:43.602263 15835 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:43.595826 14553 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:43.605973 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 262946us
I20260709 13:04:43.654441 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:43.683118 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.682700 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 134425us
W20260709 13:04:43.671384 15965 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 13:04:43.731335 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 252382us
W20260709 13:04:43.725689 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 156027us
W20260709 13:04:43.731122 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.717829 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:43.657385 14553 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:43.608414 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:43.747321 14558 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:43.747416 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 367467us
I20260709 13:04:43.745158 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:43.751996 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 67776.2us
I20260709 13:04:43.605000 15875 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:43.685933 15770 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:43.754794 13840 leader_election.cc:336] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.606s (SENT)
W20260709 13:04:43.752593 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.755576 13839 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:43.651121 15955 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:43.725258 15941 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.725028 15710 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.759241 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:43.743043 15685 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:43.668838 15960 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.772783 15960 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.773581 15963 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:04:43.776502 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.777132 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.780331 15940 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:43.780970 15940 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.782227 15909 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:43.782816 15909 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.784176 15969 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.784947 15948 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:43.784795 15969 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.785518 15948 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.787464 15969 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:43.787814 15948 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.788174 15963 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:43.789435 15963 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.790242 15962 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:43.791123 15436 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:04:43.791973 15812 consensus_queue.cc:1059] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:43.794657 15955 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.796831 15948 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.797646 15948 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.783550 15940 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:43.799861 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.799971 13866 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:43.670742 15953 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:43.814278 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.761644 15943 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:43.677392 15946 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.816562 13942 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:43.764899 15956 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:43.819626 13948 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:04:43.813678 15804 raft_consensus.cc:3055] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:43.822008 13943 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:43.823066 15953 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:43.836745 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.784806 15909 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:43.838866 13949 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
W20260709 13:04:43.837863 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 76135.8us
W20260709 13:04:43.840642 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 72112.8us
I20260709 13:04:43.817672 15943 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.858893 15971 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:43.859782 15971 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.864710 13838 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:43.863896 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 109122us
I20260709 13:04:43.869313 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:43.836189 15804 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:43.882023 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:43.883055 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:43.886293 13950 raft_consensus.cc:2368] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3.
W20260709 13:04:43.882705 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 99280.2us
I20260709 13:04:43.883615 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.890476 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:43.890594 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:43.891113 13949 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:43.814409 15685 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.893585 13941 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:43.894186 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:43.871587 15792 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.867367 15943 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:43.887005 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 121669us
I20260709 13:04:43.907651 15975 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:43.908445 15685 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.913136 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:43.894920 15963 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:43.928929 15963 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.917627 15948 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.917088 15804 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.932684 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:43.932960 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.933593 14023 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:43.918313 15986 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:04:43.934774 14021 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:43.919297 15908 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:43.921940 15987 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.922474 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:43.923447 14019 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:43.911456 15792 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:43.934674 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:43.919872 15975 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:43.935099 15986 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:43.940655 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:43.942471 15962 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:43.945101 14015 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:43.947460 15946 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:43.949719 15792 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.913651 15971 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:43.951097 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:43.952247 15975 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:43.944069 14016 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:43.979357 14017 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:43.982532 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:43.983189 14017 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:43.983582 14018 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 4 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:04:43.942404 15987 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:43.954245 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58977.7us
I20260709 13:04:43.962703 14021 raft_consensus.cc:2410] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:43.966475 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:43.973102 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:43.987165 13871 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:43.974470 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:43.988020 13872 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:43.974251 15701 consensus_queue.cc:1059] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:43.975448 13870 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:43.976131 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:43.991472 13869 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:43.976832 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:43.987309 14015 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:43.954456 15986 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:04:43.992036 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 88031.6us
W20260709 13:04:44.006927 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.010535 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.011587 14021 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:44.020731 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:44.022317 15986 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:44.023864 15986 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:44.026436 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:44.029198 13875 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:43.998729 15963 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:44.034683 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.034756 14022 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:44.035451 14021 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:44.034281 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:44.047116 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:44.046486 13872 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:44.048995 13869 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:44.051210 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.052953 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:44.055833 14018 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:44.056341 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.057480 14023 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:44.066936 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.070928 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:44.061046 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:44.073318 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.074296 14023 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:44.074455 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:44.075907 15908 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:44.076326 15908 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.068058 14018 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:44.076670 15908 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:44.099637 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.100533 14021 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:44.076682 15987 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:44.101976 15977 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.279s	user 0.005s	sys 0.000s
I20260709 13:04:44.103909 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.104681 14017 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:44.105556 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.101377 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.106621 14021 raft_consensus.cc:2376] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 4. Re-sending same reply.
I20260709 13:04:44.107577 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.108045 14017 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:44.108160 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.109090 14023 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:44.114490 15804 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.119349 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.120554 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.122125 14016 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:44.121516 14021 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:44.122630 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.123245 14022 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:44.123646 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.124154 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.124437 14021 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:44.124722 14022 raft_consensus.cc:2376] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 3. Re-sending same reply.
I20260709 13:04:44.125406 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.126178 14021 raft_consensus.cc:2368] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5.
I20260709 13:04:44.125538 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.134824 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:44.135651 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.134927 14017 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:44.136814 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.137359 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.138113 14017 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:44.138947 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:44.140872 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55106.6us
I20260709 13:04:44.141110 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.173529 14021 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:44.140913 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.175493 14022 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:44.143770 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.172904 14016 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:44.176075 14017 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:44.171787 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 92550us
W20260709 13:04:44.178306 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 313655us
I20260709 13:04:44.137766 14024 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:44.168586 15492 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:04:44.177096 15956 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:04:44.186841 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 170636us
I20260709 13:04:44.181354 15949 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:44.196091 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.196974 14021 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:44.198272 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:44.198633 13840 leader_election.cc:336] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.647s (SENT)
I20260709 13:04:44.199153 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:44.199975 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:44.200428 13867 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:44.207674 15710 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:44.207917 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.208580 14019 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:44.209254 15986 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:44.209666 15986 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:44.179332 15943 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:44.212334 14552 raft_consensus.cc:2368] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8.
I20260709 13:04:44.212628 15956 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:44.213469 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.214109 14015 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:44.214973 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.215528 14015 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:44.216271 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:44.217810 13941 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:44.224472 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:44.198305 15949 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.225484 15949 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:44.225728 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:44.209969 15983 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:44.226874 13838 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:44.226969 15983 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:44.228017 13838 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:44.228276 15947 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:44.221392 13942 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:04:44.222860 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:44.223286 15908 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:44.229178 15685 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:44.235623 15685 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:44.229293 15984 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:44.236582 15685 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:44.237298 15984 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:44.236129 15908 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:44.234375 13948 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:44.238348 15984 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:44.216296 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.208536 15710 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:44.228883 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:44.250641 15710 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:44.252923 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.254613 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 59980.3us
I20260709 13:04:44.263067 15953 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:44.264220 15763 raft_consensus.cc:3055] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:44.264637 15763 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:44.265638 15763 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:44.267253 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.273411 14020 raft_consensus.cc:2410] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:44.266984 15986 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:44.228011 15983 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:44.252877 15710 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:44.199069 14024 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
W20260709 13:04:44.285576 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.277338 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 75174.3us
I20260709 13:04:44.285401 14019 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:04:44.296295 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.297003 14017 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:44.297024 14018 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:04:44.288313 13840 leader_election.cc:336] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.816s (SENT)
W20260709 13:04:44.287460 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58638.5us
I20260709 13:04:44.299940 13838 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:44.288044 13915 leader_election.cc:336] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.753s (SENT)
I20260709 13:04:44.300319 15943 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:44.302676 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.303399 14024 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 1.
I20260709 13:04:44.303546 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:44.304311 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.304445 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:44.305151 15908 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:44.305496 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.306329 15908 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:44.307534 15908 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:44.279039 13948 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:04:44.289631 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:44.311810 13870 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:44.289557 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:44.291404 13988 leader_election.cc:400] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8.
I20260709 13:04:44.315152 13988 leader_election.cc:403] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:44.316262 15770 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:44.291852 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.292944 15983 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:44.286837 15713 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:04:44.298566 15764 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:44.292989 13869 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:44.293622 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.294828 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.295027 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:44.319763 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.321262 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:44.298355 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:44.328223 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.320572 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2155) took 1101 ms. Trace:
W20260709 13:04:44.329625 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.329845 14017 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:44.323042 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.333935 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:44.340008 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.328500 14021 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:44.329870 13874 rpcz_store.cc:276] 0709 13:04:43.219096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.344435 (+125339us) service_pool.cc:224] Handling call
0709 13:04:43.353910 (+  9475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.354154 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:43.386851 (+ 32697us) log.cc:844] Serialized 14 byte log entry
0709 13:04:44.172254 (+785403us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b18002bd980. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c730758f 00007fb2c0346571 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd
0709 13:04:44.220226 (+ 47972us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b18002bd980. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c6e2b3c7 00007fb2c343d9a7 00007fb2c344d0bf 00007fb2c344d038
0709 13:04:44.298216 (+ 77990us) log.cc:844] Serialized 14 byte log entry
0709 13:04:44.319827 (+ 21611us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:44.319886 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:44.319945 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.320412 (+   467us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.320518 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29399,"prepare.queue_time_us":71251,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":42,"raft.queue_time_us":36552,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":6253,"spinlock_wait_cycles":11410048,"thread_start_us":238633,"threads_started":2,"wal-append.queue_time_us":221182}
I20260709 13:04:44.344655 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:44.350167 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 50308.2us
I20260709 13:04:44.350260 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.350756 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.345582 14018 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:44.351552 14017 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 1.
I20260709 13:04:44.352468 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.355834 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:44.356415 13867 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:44.356603 14024 raft_consensus.cc:2410] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:04:44.358917 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.359138 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.359768 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.357846 13838 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:44.360474 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.360466 14017 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:44.361534 13871 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:44.363055 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.363505 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.363693 14018 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:44.364287 14017 raft_consensus.cc:2410] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:44.366545 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.367292 14019 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:44.368566 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.369280 14021 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
W20260709 13:04:44.370226 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.370272 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.371297 14021 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 4 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:04:44.371377 15792 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.371716 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.372145 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.372545 14020 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:44.373433 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.373834 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.373998 14020 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:44.375016 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.371944 15792 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:44.373447 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.381145 13870 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:04:44.389183 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.327698 15947 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:44.389778 14021 raft_consensus.cc:2376] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 4. Re-sending same reply.
I20260709 13:04:44.376626 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:44.691545 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.641355 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.549503 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.420938 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 132634us
W20260709 13:04:44.920274 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.920811 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.449870 13867 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:44.346519 15892 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:04:44.835880 14524 leader_election.cc:336] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.658s (SENT)
W20260709 13:04:44.920749 13840 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.862s (SENT)
I20260709 13:04:44.923121 14020 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:44.416546 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.936583 14018 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:44.937707 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.353981 15943 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:44.355459 15812 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:44.420677 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.949251 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.949572 14017 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:44.950722 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:44.883929 15770 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.567s	user 0.003s	sys 0.008s
I20260709 13:04:44.414597 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:44.964109 14022 raft_consensus.cc:2376] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6. Re-sending same reply.
I20260709 13:04:44.918689 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:44.674816 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.359117 15960 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:44.391584 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:44.811422 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.803403 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.931090 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:44.941998 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.359184 15948 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.496665 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.919065 15986 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:44.952486 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:44.945295 13840 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.913s (SENT)
W20260709 13:04:44.968456 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 617541us
I20260709 13:04:44.480408 15768 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.436761 13772 catalog_manager.cc:5697] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 1 to 2, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:44.975478 13838 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:44.970497 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.970755 14021 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
W20260709 13:04:44.972100 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.974012 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:44.982179 14016 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:44.954133 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:44.971984 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 136169us
I20260709 13:04:44.983728 14017 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:44.501688 15436 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:44.967424 14020 raft_consensus.cc:2410] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:44.945194 15943 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.991760 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.993019 14019 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:44.972502 15770 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8.
I20260709 13:04:44.949054 15953 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:44.996565 14016 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:04:45.006104 14016 pending_rounds.cc:85] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 5
I20260709 13:04:45.006438 14016 pending_rounds.cc:107] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:04:45.006727 14016 raft_consensus.cc:2889] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:44.998871 14015 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.622s	user 0.012s	sys 0.000s
I20260709 13:04:45.000993 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.003530 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.008776 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.008590 14015 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
W20260709 13:04:45.004175 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 342133us
I20260709 13:04:44.976747 15710 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:04:45.008188 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.975864 15963 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.679669 15949 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.010742 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.013806 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.015324 14019 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:44.990515 15960 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:45.002022 15943 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:45.019533 15492 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.019356 15963 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.021337 15492 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.015669 14017 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:45.029757 16010 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.030491 16010 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:45.033372 16010 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:45.018879 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:44.994618 15768 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.040309 16015 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:45.040570 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 69023.2us
I20260709 13:04:45.041077 15892 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:45.042778 16015 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.043831 15892 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.045142 16015 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:45.041863 16016 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:44.996846 15948 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.000001 15436 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:45.093032 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.098399 14015 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
W20260709 13:04:45.097040 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 77101.6us
I20260709 13:04:45.102133 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.105026 15948 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:45.105984 15492 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:45.027077 15710 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:45.118602 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.052655 15984 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.121065 14020 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:45.088092 15792 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:45.126798 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:45.106757 16002 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:04:45.130709 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:45.135649 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.100632 16016 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:45.117110 15956 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:45.138268 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.138564 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:45.121721 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 146329us
I20260709 13:04:45.143762 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.103150 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:45.147439 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 44371.6us
I20260709 13:04:45.117359 15963 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:45.142805 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.155174 14019 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:45.154714 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.156984 14015 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:04:45.158551 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 213280us
I20260709 13:04:45.159128 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:45.159257 13840 leader_election.cc:336] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.971s (SENT)
I20260709 13:04:45.161785 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:45.163427 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:45.164242 14015 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:04:45.165859 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 67914.2us
I20260709 13:04:45.153777 15701 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:45.171820 15710 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.153321 15960 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.175114 16016 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.159816 15985 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:04:45.171862 15701 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:45.179744 15701 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.152701 15764 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:45.182493 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.183238 14019 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:45.184101 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.185274 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.187345 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.174291 16022 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:45.187698 14024 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:45.188125 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:45.170773 14524 leader_election.cc:336] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.862s (SENT)
I20260709 13:04:45.198551 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:45.198639 14018 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:04:45.198881 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.200618 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.203824 14019 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:45.133579 15984 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:45.194020 15436 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:45.215441 13866 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:45.165035 13915 leader_election.cc:336] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.638s (SENT)
I20260709 13:04:45.222138 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:45.223060 14019 raft_consensus.cc:2368] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4.
I20260709 13:04:45.165424 15812 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:45.234755 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:45.251889 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:45.214766 16022 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:45.239023 15764 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.264714 15949 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:45.268406 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 97657.7us
I20260709 13:04:45.272408 13873 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
W20260709 13:04:45.279163 14524 leader_election.cc:336] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.912s (SENT)
I20260709 13:04:45.280436 14016 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:45.282961 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:45.284992 14024 raft_consensus.cc:2368] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3.
I20260709 13:04:45.262972 15436 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:45.279781 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:45.267864 15875 raft_consensus.cc:3055] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:04:45.301471 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 141609us
W20260709 13:04:45.302907 13915 leader_election.cc:336] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.705s (SENT)
W20260709 13:04:45.303959 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 205578us
I20260709 13:04:45.304934 14015 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:45.307273 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.308061 14024 raft_consensus.cc:2376] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 2. Re-sending same reply.
I20260709 13:04:45.306499 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.309556 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:45.294322 15875 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:04:45.327091 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.320529 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.328439 14024 raft_consensus.cc:2376] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 3. Re-sending same reply.
I20260709 13:04:45.264792 15812 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:45.318415 16015 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:45.346364 16035 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:45.346805 15984 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:45.347479 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.348161 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 69065.6us
I20260709 13:04:45.408875 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.375041 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:45.370018 13915 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.721s (SENT)
I20260709 13:04:45.420580 13870 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:45.420915 14015 raft_consensus.cc:2410] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:04:45.348815 14524 leader_election.cc:336] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.937s (SENT)
I20260709 13:04:45.358758 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:45.361524 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.421907 14524 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:45.387154 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 209) took 1022 ms. Trace:
W20260709 13:04:45.423569 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 261582us
I20260709 13:04:45.416800 13772 catalog_manager.cc:5697] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:45.426908 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:45.427556 13948 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
I20260709 13:04:45.428534 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:45.427928 13772 catalog_manager.cc:5697] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 3, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:45.430302 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:45.430874 14015 raft_consensus.cc:2376] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 3. Re-sending same reply.
W20260709 13:04:45.431414 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 86206.8us
W20260709 13:04:45.432142 13915 leader_election.cc:336] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.893s (SENT)
I20260709 13:04:45.434726 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:45.429352 13771 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54072 (request call id 135) took 3459 ms. Trace:
I20260709 13:04:45.430114 13772 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 4, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 4 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:45.430799 13870 raft_consensus.cc:2368] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4.
I20260709 13:04:45.425575 14017 rpcz_store.cc:276] 0709 13:04:44.364632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.987323 (+622691us) service_pool.cc:224] Handling call
0709 13:04:45.011464 (+ 24141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.024492 (+ 13028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.386938 (+362446us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b54003255d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:45.387083 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54873984}
W20260709 13:04:45.378902 16027 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:04:45.442359 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:45.444000 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:45.315646 16030 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:04:45.447209 13915 leader_election.cc:336] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.923s (SENT)
I20260709 13:04:45.452869 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:45.448800 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:04:45.384296 16023 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:04:45.449841 13942 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:45.450953 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:45.387650 15983 consensus_meta.cc:287] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.061s	user 0.008s	sys 0.008s
I20260709 13:04:45.348667 16015 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.447595 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.457877 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:45.470872 14024 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:45.471930 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:45.472858 14017 raft_consensus.cc:2376] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 3. Re-sending same reply.
I20260709 13:04:45.435827 13771 rpcz_store.cc:276] 0709 13:04:41.957299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.957679 (+   380us) service_pool.cc:224] Handling call
0709 13:04:45.415958 (+3458279us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:43.031325 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:43.032167 (+   842us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:43.032233 (+    66us) write_op.cc:435] Acquired schema lock
0709 13:04:43.032258 (+    25us) tablet.cc:662] Decoding operations
0709 13:04:43.033183 (+   925us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:43.033269 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:43.033297 (+    28us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:04:43.034001 (+   704us) tablet.cc:701] Row locks acquired
0709 13:04:43.050108 (+ 16107us) write_op.cc:260] PREPARE: finished
0709 13:04:43.050432 (+   324us) write_op.cc:270] Start()
0709 13:04:43.050681 (+   249us) write_op.cc:276] Timestamp: P: 1783602283050313 usec, L: 0
0709 13:04:43.050702 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:04:43.122239 (+ 71537us) log.cc:844] Serialized 2446 byte log entry
0709 13:04:43.934742 (+812503us) op_driver.cc:464] REPLICATION: finished
0709 13:04:44.171578 (+236836us) write_op.cc:301] APPLY: starting
0709 13:04:44.171934 (+   356us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:44.172159 (+   225us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:44.172330 (+   171us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:44.250608 (+ 78278us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:44.250948 (+   340us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:04:44.250997 (+    49us) write_op.cc:312] APPLY: finished
0709 13:04:44.251379 (+   382us) log.cc:844] Serialized 47 byte log entry
0709 13:04:44.251602 (+   223us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:44.258833 (+  7231us) write_op.cc:454] Released schema lock
0709 13:04:44.258904 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:44.260392 (+  1488us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:44.260422 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":39536,"reported_tablets":11,"spinlock_wait_cycles":2432,"child_traces":[["op",{"apply.queue_time_us":150501,"apply.run_cpu_time_us":4174,"apply.run_wall_time_us":89614,"num_ops":5,"prepare.queue_time_us":463460,"prepare.run_cpu_time_us":51804,"prepare.run_wall_time_us":579291,"raft.queue_time_us":225326,"raft.run_cpu_time_us":18519,"raft.run_wall_time_us":80028,"replication_time_us":883774,"thread_start_us":1020722,"threads_started":4,"wal-append.queue_time_us":541205,"wal-append.run_cpu_time_us":19220,"wal-append.run_wall_time_us":1575918}]]}
I20260709 13:04:45.458845 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.379629 15768 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:45.385908 15763 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:45.474285 14015 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:45.349819 16032 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:45.480786 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:45.480754 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:04:45.482451 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.489583 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:45.490824 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:45.475812 15701 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:45.501228 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:45.504711 14015 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:45.504774 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 57628.2us
I20260709 13:04:45.474581 16035 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:45.505522 13915 leader_election.cc:336] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.017s (SENT)
I20260709 13:04:45.505985 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:45.508059 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.508286 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:45.508988 13872 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:45.511256 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:45.499450 15983 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:45.515033 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:45.514861 15983 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:45.517407 15983 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:45.528642 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.530507 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:45.523064 16035 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:45.536418 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.537002 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.587067 13942 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:45.461217 15875 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:45.586691 13872 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:45.598407 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:45.598722 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 93225.8us
I20260709 13:04:45.507555 16039 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:45.493737 16030 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:04:45.581816 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 233027us
I20260709 13:04:45.479262 16015 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:45.602577 14524 leader_election.cc:336] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.988s (SENT)
I20260709 13:04:45.602947 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:45.599088 14017 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:45.600378 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 188) took 1332 ms. Trace:
W20260709 13:04:45.604419 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.609306 16015 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:45.601943 15986 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.619s	user 0.011s	sys 0.011s
I20260709 13:04:45.610414 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:45.611588 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:45.597473 15983 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:45.614835 16029 log.cc:927] Time spent T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:04:45.611668 15875 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:45.617241 16035 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:45.597843 16022 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:45.621452 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:45.624888 13948 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
W20260709 13:04:45.629266 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.599545 16028 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.235s	user 0.003s	sys 0.005s
W20260709 13:04:45.631817 14524 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.073s (SENT)
I20260709 13:04:45.632628 14017 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:45.633008 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:45.604038 14021 rpcz_store.cc:276] 0709 13:04:44.265048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.981829 (+716781us) service_pool.cc:224] Handling call
0709 13:04:45.005555 (+ 23726us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:45.023750 (+ 18195us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:45.023778 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:45.026072 (+  2294us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.151723 (+125651us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:45.151772 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.151854 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.597259 (+445405us) raft_consensus.cc:1685] finished
0709 13:04:45.597278 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.597412 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5992,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":12150,"thread_start_us":44928,"threads_started":1,"wal-append.queue_time_us":124340}
I20260709 13:04:45.619558 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:45.634249 13942 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:45.636950 14020 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:45.639658 14015 raft_consensus.cc:2410] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:45.640383 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:45.640518 14524 leader_election.cc:336] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.075s (SENT)
I20260709 13:04:45.641069 16035 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:45.637362 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2050) took 1536 ms. Trace:
I20260709 13:04:45.649760 13870 raft_consensus.cc:2410] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:04:45.644837 14024 raft_consensus.cc:2376] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 4. Re-sending same reply.
I20260709 13:04:45.651916 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:45.652354 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:45.653010 14024 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:04:45.653458 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 54013.1us
I20260709 13:04:45.654543 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:45.655263 13866 raft_consensus.cc:2368] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8.
I20260709 13:04:45.656281 13866 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:45.656893 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.657384 13866 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:45.657704 14016 raft_consensus.cc:2376] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 3. Re-sending same reply.
I20260709 13:04:45.658147 13868 raft_consensus.cc:2393] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:45.658735 16030 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:45.659482 13868 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:45.659896 13873 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:45.661299 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:45.661541 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:45.662438 14019 raft_consensus.cc:2410] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:04:45.667526 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2972675470000468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1723s for a response
I20260709 13:04:45.669513 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:45.672632 13873 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:45.650923 15943 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.616s	user 0.009s	sys 0.003s
I20260709 13:04:45.673271 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:45.674772 13868 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
W20260709 13:04:45.667961 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2352489090000063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
W20260709 13:04:45.676107 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2230140320000373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2719s for a response
W20260709 13:04:45.677009 13840 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.580s (SENT)
I20260709 13:04:45.678581 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:45.677048 16016 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:45.649616 14023 rpcz_store.cc:276] 0709 13:04:44.100989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.109974 (+  8985us) service_pool.cc:224] Handling call
0709 13:04:44.155082 (+ 45108us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.155637 (+   555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.155680 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:45.253255 (+1097575us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b2c000e8788. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:04:45.275792 (+ 22537us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.423162 (+147370us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:45.426521 (+  3359us) log.cc:844] Serialized 14 byte log entry
0709 13:04:45.426776 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.426851 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.619130 (+192279us) raft_consensus.cc:1685] finished
0709 13:04:45.637140 (+ 18010us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.637297 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10022,"raft.queue_time_us":6790,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":10398,"spinlock_wait_cycles":44680448,"thread_start_us":115891,"threads_started":1,"wal-append.queue_time_us":205234}
W20260709 13:04:45.683319 13840 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.631s (SENT)
I20260709 13:04:45.683964 13840 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:45.676327 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:45.679804 16016 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:45.682926 15436 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:45.685225 15892 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:45.693307 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47179.7us
I20260709 13:04:45.693897 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.694917 14019 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:45.696996 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:45.698836 15999 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.328s	user 0.019s	sys 0.042s
I20260709 13:04:45.685370 15768 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:45.700306 15768 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:45.697042 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:45.697590 14023 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:45.711855 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:45.713827 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:45.714324 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:45.692442 15436 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:45.715991 13840 leader_election.cc:336] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.661s (SENT)
I20260709 13:04:45.717633 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:45.678663 15943 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:45.720194 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:45.696229 15960 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:45.690464 15986 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:04:45.722213 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:45.711300 16035 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:45.724304 16035 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.724668 16035 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:45.725709 15998 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.355s	user 0.009s	sys 0.017s
I20260709 13:04:45.731061 15436 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:45.735989 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.694561 15892 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:04:45.742022 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.743202 13840 leader_election.cc:336] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.670s (SENT)
I20260709 13:04:45.747324 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:45.748903 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:45.759132 13838 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:45.710278 15768 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:45.762465 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.768967 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:45.724681 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:45.772049 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:45.772842 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.740978 15951 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:45.757321 15892 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:45.757787 15983 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:04:45.783301 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.785604 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:45.786829 15946 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:45.789183 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 46037us
I20260709 13:04:45.768595 15986 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:45.799475 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:45.799852 14524 leader_election.cc:336] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.609s (SENT)
I20260709 13:04:45.800386 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:45.801056 15951 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:04:45.791198 13840 leader_election.cc:336] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.983s (SENT)
I20260709 13:04:45.773162 15953 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:45.801237 15987 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:45.802301 13840 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.003s (SENT)
I20260709 13:04:45.802793 13838 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:04:45.803378 13840 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.008s (SENT)
I20260709 13:04:45.800293 14023 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:45.781818 15983 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:45.805068 13838 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:45.890498 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:45.897821 16035 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:45.959412 14524 leader_election.cc:336] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.780s (SENT)
I20260709 13:04:46.001291 14551 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:46.000963 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:45.924059 14018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 210) took 1483 ms (client timeout 1886 ms). Trace:
I20260709 13:04:45.992985 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:45.831983 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 1952) took 1618 ms. Trace:
W20260709 13:04:45.984510 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 181139us
I20260709 13:04:45.993561 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:45.992280 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 187290us
I20260709 13:04:45.995620 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:45.996177 16022 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:45.996773 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.996970 15892 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:45.946444 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:45.983690 16014 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.878569 15943 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:46.003031 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 43692.2us
W20260709 13:04:45.918380 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.941524 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.991065 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.986764 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:46.164045 14019 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
W20260709 13:04:46.164203 14524 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.848s (SENT)
I20260709 13:04:46.164742 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:46.164927 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:46.165077 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:46.165788 14024 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:46.165746 14019 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:45.856799 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:45.924659 14022 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 8. Ops: [6.8-6.8]
W20260709 13:04:45.912798 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 134403us
I20260709 13:04:46.004123 13875 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:46.006556 14021 raft_consensus.cc:2376] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 5. Re-sending same reply.
I20260709 13:04:45.815003 15946 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:46.006870 14018 rpcz_store.cc:269] 0709 13:04:44.440574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.937395 (+496821us) service_pool.cc:224] Handling call
0709 13:04:45.923813 (+986418us) spinlock_profiling.cc:243] Waited 75.4 ms on lock 0x7b540023bc54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:45.923979 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173413120}
W20260709 13:04:46.073323 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.786829 15764 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:45.858670 15701 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:46.075922 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.807873 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:45.888669 15436 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:46.004160 15987 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:46.041822 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:46.093765 13840 leader_election.cc:336] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.053s (SENT)
I20260709 13:04:46.093746 14017 raft_consensus.cc:2376] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 3. Re-sending same reply.
I20260709 13:04:46.117844 14020 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:46.024416 13946 rpcz_store.cc:276] 0709 13:04:44.213461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.213908 (+   447us) service_pool.cc:224] Handling call
0709 13:04:44.214285 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.214691 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.214719 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.215191 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.286270 (+ 71079us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:44.299151 (+ 12881us) log.cc:844] Serialized 14 byte log entry
0709 13:04:44.299358 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.299461 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.725073 (+1425612us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:04:45.814244 (+ 89171us) raft_consensus.cc:1685] finished
0709 13:04:45.831736 (+ 17492us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.831910 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16927,"prepare.queue_time_us":23214,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":33769,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":289,"spinlock_wait_cycles":13096320,"thread_start_us":70521,"threads_started":1,"wal-append.queue_time_us":112385}
I20260709 13:04:45.806598 15986 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:46.151075 15763 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.152005 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:46.152540 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.586540 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:46.591383 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 427169us
I20260709 13:04:46.592572 13948 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:46.593547 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:46.612792 14015 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:46.585114 13989 connection.cc:256] RPC call timeout handler was delayed by -1.2612078699999074s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
W20260709 13:04:46.618498 13989 connection.cc:256] RPC call timeout handler was delayed by -1.1406078699999398s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
W20260709 13:04:46.618901 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.608131 14023 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:04:46.619406 13989 connection.cc:256] RPC call timeout handler was delayed by -1.0109654159999764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
I20260709 13:04:46.621348 13989 leader_election.cc:400] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4.
I20260709 13:04:46.621831 13989 leader_election.cc:403] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:04:46.617292 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.200639 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:46.627094 14019 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:46.608083 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:46.627913 13866 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:46.616232 14560 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
W20260709 13:04:46.555688 16057 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
W20260709 13:04:46.330798 16053 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:46.116425 16035 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:46.650769 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:46.558516 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 02dcf286c6bd48e3a6e548def7d786cc with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:04:46.636008 15763 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:46.518878 15875 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:46.651530 15908 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:46.661723 13875 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:46.277720 13943 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:46.505733 15792 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.567051 15892 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:46.662603 13944 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:46.166671 16022 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:46.201152 16050 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.685252 14015 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:46.705252 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:46.705942 14015 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:46.259889 15801 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:46.656503 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.688585 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:46.654175 16016 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:46.707736 14016 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:46.678442 15985 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.236843 15983 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:46.686355 16025 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:46.694206 13758 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 136013us
I20260709 13:04:46.203899 16014 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:46.712703 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:46.822549 14016 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:46.749228 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 176) took 2487 ms. Trace:
I20260709 13:04:46.825469 14022 rpcz_store.cc:276] 0709 13:04:44.261641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.974593 (+712952us) service_pool.cc:224] Handling call
0709 13:04:45.924284 (+949691us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:46.613008 (+688724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.748939 (+135931us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b540023bc50. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:46.749141 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2183042304}
I20260709 13:04:46.694821 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:46.636634 15701 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:46.844179 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:46.845227 14022 raft_consensus.cc:2376] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 3. Re-sending same reply.
I20260709 13:04:46.847198 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:46.778302 16016 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:46.705075 15792 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:46.877341 14015 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:46.842047 15983 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:46.696630 15875 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:46.694643 15908 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:46.873000 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:46.915380 14016 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:46.921372 14016 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:46.779218 15985 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:46.748806 16050 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:46.878863 15946 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.837445 16025 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:46.948706 13941 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:46.950755 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 329569us
W20260709 13:04:46.955200 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 304242us
I20260709 13:04:46.959692 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:04:46.956938 13915 leader_election.cc:336] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.762s (SENT)
W20260709 13:04:46.986968 13915 leader_election.cc:336] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.824s (SENT)
I20260709 13:04:46.987624 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:46.971058 15875 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:46.995482 13872 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:46.971903 15436 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:46.980588 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:46.975634 14522 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:46.999799 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:46.940197 15946 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:47.005093 15892 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:47.005636 13875 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:47.006114 13873 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:47.007469 15985 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:46.936659 16022 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:46.967674 16059 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:47.016623 13946 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:46.971123 15792 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:47.021636 13874 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:47.025200 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:04:47.026469 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 932672us
I20260709 13:04:47.027993 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:47.038033 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.040084 13840 leader_election.cc:336] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.057s (SENT)
W20260709 13:04:47.043934 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 57018.1us
W20260709 13:04:47.044896 13915 leader_election.cc:336] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.830s (SENT)
I20260709 13:04:47.045363 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:47.039062 13948 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 2.
W20260709 13:04:47.059875 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 83859.4us
W20260709 13:04:47.060518 14524 leader_election.cc:336] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.199s (SENT)
I20260709 13:04:47.008301 16014 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:47.008699 15875 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:47.064283 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:47.018399 15986 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:47.025079 15763 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:47.070221 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 110717us
W20260709 13:04:47.073531 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.074750 13987 connection.cc:256] RPC call timeout handler was delayed by -2.5844725050000079s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
W20260709 13:04:47.075407 13987 connection.cc:256] RPC call timeout handler was delayed by -2.5619725050000852s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260709 13:04:47.075804 13987 connection.cc:256] RPC call timeout handler was delayed by -2.50707250500011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260709 13:04:47.076188 13987 connection.cc:256] RPC call timeout handler was delayed by -2.4701725050000505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
W20260709 13:04:47.076574 13987 connection.cc:256] RPC call timeout handler was delayed by -2.4260725050000929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1698s for a response
W20260709 13:04:47.076948 13987 connection.cc:256] RPC call timeout handler was delayed by -2.3072725050000145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
W20260709 13:04:47.077334 13987 connection.cc:256] RPC call timeout handler was delayed by -2.2442725050000263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260709 13:04:47.077709 13987 connection.cc:256] RPC call timeout handler was delayed by -2.2334725050000088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1912s for a response
W20260709 13:04:47.078104 13987 connection.cc:256] RPC call timeout handler was delayed by -2.2271725050001123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
W20260709 13:04:47.078503 13987 connection.cc:256] RPC call timeout handler was delayed by -2.2208725050001021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1926s for a response
W20260709 13:04:47.078874 13987 connection.cc:256] RPC call timeout handler was delayed by -2.2172725050000963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.193s for a response
W20260709 13:04:47.079260 13987 connection.cc:256] RPC call timeout handler was delayed by -2.1911725050000541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
W20260709 13:04:47.079649 13987 connection.cc:256] RPC call timeout handler was delayed by -2.1803725050000367s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19710000000000003s for a response
W20260709 13:04:47.080022 13987 connection.cc:256] RPC call timeout handler was delayed by -2.1785725050000337s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19730000000000003s for a response
W20260709 13:04:47.080413 13987 connection.cc:256] RPC call timeout handler was delayed by -2.163272505000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.199s for a response
W20260709 13:04:47.080776 13987 connection.cc:256] RPC call timeout handler was delayed by -2.1236725050000587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
W20260709 13:04:47.081171 13987 connection.cc:256] RPC call timeout handler was delayed by -2.04177250500004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21250000000000002s for a response
W20260709 13:04:47.081547 13987 connection.cc:256] RPC call timeout handler was delayed by -2.0219725050000079s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2147s for a response
W20260709 13:04:47.081914 13987 connection.cc:256] RPC call timeout handler was delayed by -1.9805725050000547s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21930000000000002s for a response
W20260709 13:04:47.082744 13840 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.114s (SENT)
W20260709 13:04:47.083128 13987 connection.cc:256] RPC call timeout handler was delayed by -1.9463725050001131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2231s for a response
I20260709 13:04:47.059724 15792 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:47.065055 16014 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:47.058909 16061 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4.
I20260709 13:04:47.100414 16061 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.100924 16061 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:47.102813 15701 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:47.106287 16050 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:47.106503 16069 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.107049 16069 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:47.107748 16061 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:47.109508 16069 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:47.094832 13987 connection.cc:256] RPC call timeout handler was delayed by -1.4450725050000983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2788s for a response
W20260709 13:04:47.110389 13987 connection.cc:256] RPC call timeout handler was delayed by -1.3163725050001176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2931s for a response
W20260709 13:04:47.098099 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 53209.5us
I20260709 13:04:47.066114 15801 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:47.098610 13870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 1986) took 1609 ms (client timeout 1874 ms). Trace:
I20260709 13:04:47.098803 15792 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:47.110713 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.114022 13840 leader_election.cc:336] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.119s (SENT)
W20260709 13:04:47.110731 13987 connection.cc:256] RPC call timeout handler was delayed by -1.2083725050000567s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30510000000000004s for a response
W20260709 13:04:47.113765 13870 rpcz_store.cc:269] 0709 13:04:45.486984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.514686 (+ 27702us) service_pool.cc:224] Handling call
0709 13:04:47.096009 (+1581323us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54000ea454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:47.096188 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301548416}
W20260709 13:04:47.117918 13987 connection.cc:256] RPC call timeout handler was delayed by -1.0859725050000861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3187s for a response
I20260709 13:04:47.114778 13866 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
W20260709 13:04:47.120118 13915 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.893s (SENT)
I20260709 13:04:47.065763 16064 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:47.120236 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:47.120800 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:47.091657 16017 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:47.115513 15801 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:47.137576 16017 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:47.137189 13987 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:47.138783 13915 leader_election.cc:336] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.915s (SENT)
I20260709 13:04:47.138722 16017 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:47.139353 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:47.139463 16064 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:47.139721 15801 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:47.122822 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:47.140462 13915 leader_election.cc:336] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.951s (SENT)
I20260709 13:04:47.123327 15875 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:47.140880 15983 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:47.141074 15701 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:47.124907 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.142187 13870 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:47.143038 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:47.127251 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.057813 16059 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:47.144043 13873 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:47.140884 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:47.145985 16059 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:47.146236 13915 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.040s (SENT)
I20260709 13:04:47.129913 15946 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:47.136025 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:47.124749 15792 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:47.147265 13915 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.078s (SENT)
I20260709 13:04:47.147653 15875 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.147727 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:47.143198 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.084626 15908 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:47.148470 15801 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:47.134996 13866 raft_consensus.cc:2368] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4.
I20260709 13:04:47.148979 15892 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:47.145206 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.152139 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.148797 15908 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:47.149462 13870 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:47.151932 15801 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:04:47.149000 13915 leader_election.cc:336] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.102s (SENT)
I20260709 13:04:47.154160 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:47.155098 15908 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:47.155853 15875 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:47.157236 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.158514 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:47.159409 13944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1330) took 1411 ms (client timeout 1500 ms). Trace:
W20260709 13:04:47.159865 13944 rpcz_store.cc:269] 0709 13:04:45.748330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.748527 (+   197us) service_pool.cc:224] Handling call
0709 13:04:47.159251 (+1410724us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400229f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:47.159352 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2098938112}
I20260709 13:04:47.160622 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:47.161616 13915 leader_election.cc:336] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.145s (SENT)
I20260709 13:04:47.162079 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:04:47.163717 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 49733.5us
W20260709 13:04:47.164832 13840 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.173s (SENT)
I20260709 13:04:47.165217 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:47.193130 16052 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:47.197698 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.198951 13840 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.192s (SENT)
I20260709 13:04:47.199507 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:47.200235 15947 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:47.200546 16060 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:47.201773 13840 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.192s (SENT)
I20260709 13:04:47.202696 16060 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.203176 16060 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:47.208695 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:47.210358 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 48765.9us
W20260709 13:04:47.211536 13915 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.184s (SENT)
I20260709 13:04:47.209757 13944 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
W20260709 13:04:47.214780 13840 leader_election.cc:336] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.241s (SENT)
W20260709 13:04:47.216317 13915 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.198s (SENT)
I20260709 13:04:47.216933 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:47.219193 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:47.221519 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:04:47.230623 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 87667.8us
I20260709 13:04:47.231571 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:47.232648 14560 raft_consensus.cc:2368] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4.
I20260709 13:04:47.233482 16022 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:47.234390 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:47.235422 14551 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:47.217922 16067 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:47.210825 16025 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:47.237980 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:47.206032 16060 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:47.239609 13987 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:47.233618 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:47.241108 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 156271us
I20260709 13:04:47.242765 16067 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:47.245445 13914 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:04:47.247886 13989 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.782s (SENT) [suppressed 66 similar messages]
I20260709 13:04:47.250396 13914 leader_election.cc:400] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4.
I20260709 13:04:47.250725 13914 leader_election.cc:403] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:04:47.250742 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.251981 13915 leader_election.cc:336] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.199s (SENT)
I20260709 13:04:47.252450 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:04:47.254086 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47679.7us
W20260709 13:04:47.255995 13840 connection.cc:256] RPC call timeout handler was delayed by -1.0498489089999339s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19890000000000002s for a response
W20260709 13:04:47.256383 13840 connection.cc:256] RPC call timeout handler was delayed by -1.0318489090000185s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
I20260709 13:04:47.255728 15792 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:47.267537 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.268088 15986 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:47.249064 16050 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:47.270612 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:47.271239 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:47.268105 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.273906 13838 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:04:47.268149 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.253635 15908 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:47.258104 15892 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:47.279484 13840 connection.cc:256] RPC call timeout handler was delayed by -1.6505147190000571s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2465s for a response
W20260709 13:04:47.296862 13840 connection.cc:256] RPC call timeout handler was delayed by -1.5624829880000561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2782s for a response
W20260709 13:04:47.297245 13840 connection.cc:256] RPC call timeout handler was delayed by -1.5247385679999752s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 13:04:47.297520 13840 connection.cc:256] RPC call timeout handler was delayed by -1.5229385679999723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16410000000000002s for a response
W20260709 13:04:47.297966 13840 leader_election.cc:336] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.635s (SENT)
I20260709 13:04:47.295785 15943 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:04:47.299408 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.284127 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.299816 16064 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:47.295990 15860 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:04:47.300653 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.300977 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.301239 15792 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:47.302425 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.301020 15860 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:47.303045 13944 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:47.303998 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.303793 15860 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:47.305178 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.306831 15949 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:47.309361 14556 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:47.309780 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:47.310443 14555 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:47.310784 13947 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:04:47.311376 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:47.311663 15962 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:47.312450 15962 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:47.317067 15953 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:47.318390 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:47.319379 15960 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:47.319916 15960 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.320283 15960 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:47.324402 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:47.327982 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:47.329813 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.330843 13872 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:47.332082 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.333017 13870 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:04:47.334400 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.295996 16067 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:04:47.339939 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:47.285049 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.286482 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.287931 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.289295 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:47.343312 13949 raft_consensus.cc:2410] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:47.343991 13950 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:47.344316 15943 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:47.345163 15962 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:47.346449 15943 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:47.346596 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.347092 15943 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.347602 15943 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:47.294564 13915 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.212s (SENT)
I20260709 13:04:47.351438 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:47.294682 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:47.353127 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.353922 13949 raft_consensus.cc:2368] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8.
W20260709 13:04:47.357218 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 62632.9us
I20260709 13:04:47.358453 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:47.360028 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:47.364259 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:47.366816 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.367504 13950 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:04:47.371677 15892 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4.
I20260709 13:04:47.293426 15436 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:47.373097 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 76779.7us
W20260709 13:04:47.375703 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 110729us
I20260709 13:04:47.376487 13987 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:47.373234 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 75289.4us
W20260709 13:04:47.379026 13840 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.672s (SENT)
W20260709 13:04:47.379807 13840 leader_election.cc:336] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.723s (SENT)
I20260709 13:04:47.296600 15710 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:04:47.350930 16040 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.627s	user 0.048s	sys 0.077s
W20260709 13:04:47.374225 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 79711.6us
I20260709 13:04:47.296772 15908 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:47.374892 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.381107 13950 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:47.362833 15764 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:47.382164 15436 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:47.390594 13840 leader_election.cc:336] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.783s (SENT)
W20260709 13:04:47.391836 13915 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.241s (SENT)
I20260709 13:04:47.400152 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:47.391371 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.401299 13942 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:47.367982 15984 raft_consensus.cc:2764] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:04:47.368216 16059 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:47.373063 16050 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:04:47.383164 16040 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.660s	user 0.048s	sys 0.077s
I20260709 13:04:47.373692 16025 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:47.382933 15801 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:47.404068 13840 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.985s (SENT)
I20260709 13:04:47.375464 15792 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:47.412576 13772 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57648 (request call id 277) took 5123 ms. Trace:
I20260709 13:04:47.383366 15908 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:47.421059 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.329877 15960 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:47.432389 16072 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:04:47.433506 13946 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:04:47.434710 15960 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:47.423182 16025 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.367585 15947 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:04:47.431777 15436 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:47.431650 15984 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:47.423372 15764 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:04:47.439682 13948 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:47.431116 13942 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:04:47.461021 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.401036 16069 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:47.401262 15710 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:47.454732 13838 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:47.456656 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52621.4us
W20260709 13:04:47.466276 13840 leader_election.cc:336] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.989s (SENT)
I20260709 13:04:47.466697 13840 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:47.480120 15960 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:47.480690 15960 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.480844 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.481148 15960 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:47.481630 13948 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:47.485321 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.411994 15701 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:47.500584 13949 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:04:47.462157 16073 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:04:47.485988 16052 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.009s
W20260709 13:04:47.486552 13840 connection.cc:256] RPC call timeout handler was delayed by -1.4599385679999841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
I20260709 13:04:47.423614 13772 rpcz_store.cc:276] 0709 13:04:42.289022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.294514 (+  5492us) service_pool.cc:224] Handling call
0709 13:04:42.536483 (+241969us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b5400002490. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030f0b2 00007fb2c9288a89 00007fb2c9220da6 00007fb2c91f2088 00007fb2c92f037e 00007fb2c77b12a4 00007fb2c77b1232 00007fb2c77b1161 00007fb2c77b10dc
0709 13:04:47.412490 (+4876007us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:45.630390 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:45.631051 (+   661us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:45.631091 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:04:45.631106 (+    15us) tablet.cc:662] Decoding operations
0709 13:04:45.631741 (+   635us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:45.631803 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:45.631832 (+    29us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:04:45.632246 (+   414us) tablet.cc:701] Row locks acquired
0709 13:04:45.632264 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:04:45.632420 (+   156us) write_op.cc:270] Start()
0709 13:04:45.632536 (+   116us) write_op.cc:276] Timestamp: P: 1783602285632365 usec, L: 0
0709 13:04:45.632557 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:04:45.686989 (+ 54432us) log.cc:844] Serialized 1701 byte log entry
0709 13:04:46.276929 (+589940us) op_driver.cc:464] REPLICATION: finished
0709 13:04:46.588393 (+311464us) write_op.cc:301] APPLY: starting
0709 13:04:46.588522 (+   129us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:46.588781 (+   259us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:46.588957 (+   176us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:46.589821 (+   864us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:46.637651 (+ 47830us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:04:46.648053 (+ 10402us) write_op.cc:312] APPLY: finished
0709 13:04:46.776529 (+128476us) log.cc:844] Serialized 41 byte log entry
0709 13:04:46.850900 (+ 74371us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:46.855198 (+  4298us) write_op.cc:454] Released schema lock
0709 13:04:46.863781 (+  8583us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:46.927058 (+ 63277us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:46.929159 (+  2101us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":545165,"reported_tablets":18,"spinlock_wait_cycles":93894400,"child_traces":[["op",{"apply.queue_time_us":310147,"apply.run_cpu_time_us":4344,"apply.run_wall_time_us":291014,"num_ops":4,"prepare.queue_time_us":135837,"prepare.run_cpu_time_us":24865,"prepare.run_wall_time_us":227427,"raft.queue_time_us":230996,"raft.run_cpu_time_us":6439,"raft.run_wall_time_us":462836,"replication_time_us":615621,"thread_start_us":558542,"threads_started":4,"wal-append.queue_time_us":140038}]]}
W20260709 13:04:47.513118 13840 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.009s (SENT)
I20260709 13:04:47.513556 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:47.514628 13840 connection.cc:256] RPC call timeout handler was delayed by -1.4509385679999696s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
I20260709 13:04:47.516112 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.516880 13941 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:47.517832 13941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:47.521306 13941 raft_consensus.cc:606] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:47.521430 15962 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.522248 13946 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:47.522082 15962 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:47.524427 15962 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:47.486733 15763 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.487119 16014 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:04:47.487373 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:47.486006 13948 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:47.494415 16067 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.504317 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:47.525750 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:47.531235 14553 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:04:47.531390 16059 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:47.403286 16061 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
W20260709 13:04:47.505812 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 129410us
I20260709 13:04:47.540489 16076 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:47.540988 16076 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:47.541041 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:47.542584 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:47.542225 16076 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:47.542757 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:47.450515 16025 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.543458 14551 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:04:47.543597 16064 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:47.543737 16076 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:47.544135 16064 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:47.545140 16064 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:47.546324 16077 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:47.547128 16077 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:47.505777 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 309) took 1749 ms. Trace:
I20260709 13:04:47.477444 16069 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:47.547902 13943 rpcz_store.cc:276] 0709 13:04:45.755438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.758414 (+  2976us) service_pool.cc:224] Handling call
0709 13:04:45.787820 (+ 29406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:46.695908 (+908088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.464746 (+768838us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b5400330754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:47.505082 (+ 40336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1083114752}
I20260709 13:04:47.548628 16069 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:47.550000 16069 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:47.510795 15710 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:47.514984 13840 connection.cc:256] RPC call timeout handler was delayed by -1.4113385680000192s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
I20260709 13:04:47.504657 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:47.553614 13840 leader_election.cc:336] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.719s (SENT)
I20260709 13:04:47.525442 15943 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:47.554144 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:47.555866 15943 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:47.555963 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2115385679999235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
I20260709 13:04:47.556741 15953 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:47.559026 15763 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:04:47.532606 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45315.3us
I20260709 13:04:47.431234 15792 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.565150 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:47.547585 15701 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:47.577525 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:47.580052 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:47.580809 13867 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:47.581238 13914 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:47.580340 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:47.587848 13915 leader_election.cc:400] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4.
I20260709 13:04:47.588795 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:47.521812 15960 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:47.620072 15701 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:47.621419 15892 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:47.622246 15892 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:47.596895 15908 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:47.623662 15892 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:47.625061 15892 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:47.625699 15908 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:47.607030 15986 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:47.623692 13947 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:47.627068 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 84577.4us
I20260709 13:04:47.629036 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:47.636787 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.640003 13867 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:47.647540 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:47.595492 15436 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:47.648574 15436 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.648972 15436 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:47.589875 15983 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:47.595692 13840 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:47.595539 15962 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:47.615499 16064 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.535385 16067 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:47.674660 15436 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:47.664040 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 72946.8us
I20260709 13:04:47.675817 13947 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:47.667953 13987 leader_election.cc:400] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4.
I20260709 13:04:47.677287 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:47.677736 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:47.614398 16050 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:04:47.628494 15943 consensus_queue.cc:1059] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:04:47.656090 15983 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.660579 16082 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:47.659579 15962 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:47.717024 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.662987 15984 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:47.780881 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.716434 15436 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:47.659545 16009 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:47.729605 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.795195 13872 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
W20260709 13:04:47.734982 16057 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:04:47.735072 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 53560us
I20260709 13:04:47.798447 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.799230 13867 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:47.752784 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.681360 16022 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:47.801573 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 106912us
I20260709 13:04:47.811487 13987 leader_election.cc:400] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3.
I20260709 13:04:47.812181 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:47.725142 15983 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:47.724875 16061 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:47.779222 15962 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:47.826923 16080 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:04:47.813275 16014 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
W20260709 13:04:47.831264 16078 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:04:47.843096 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.884395 15436 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:47.882731 13773 catalog_manager.cc:5697] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 3 to 4, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 4 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:47.898507 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 79822.5us
I20260709 13:04:47.890234 13773 catalog_manager.cc:5697] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 3 to 5, now has a pending config: VOTER e1649107c781428692458f10597c5742 (127.12.109.65), VOTER 3779649036364744823fc035dac892c7 (127.12.109.66), VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67), VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 5 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: NON_VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:47.907687 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.815165 16009 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:47.817301 16061 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:47.905239 13773 catalog_manager.cc:5697] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 3 to 5, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 5 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.916457 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:47.925231 13773 catalog_manager.cc:5697] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 3, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 3 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.931514 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.927580 13773 catalog_manager.cc:5697] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 3, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 3 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.932624 13870 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:47.933284 13773 catalog_manager.cc:5697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.939169 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:47.926496 16061 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:47.945634 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:47.953852 14551 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:47.954982 16009 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:47.961488 16069 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:47.972877 15983 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:47.979615 15960 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:47.983309 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.984967 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:47.900453 16090 consensus_queue.cc:1059] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
W20260709 13:04:47.985144 14524 leader_election.cc:336] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.705s (SENT)
I20260709 13:04:47.940410 13773 catalog_manager.cc:5697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.988312 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:47.934646 16076 raft_consensus.cc:2764] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:04:47.996310 13868 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:47.998548 16090 raft_consensus.cc:3055] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:47.999238 13872 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:04:48.000984 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 61900.9us
I20260709 13:04:48.001817 13773 catalog_manager.cc:5697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.001933 13870 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:48.013775 15763 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:48.021078 13987 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:47.987586 16039 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:48.014470 15763 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:48.029184 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:48.045192 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:48.047431 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:48.048445 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:48.049427 15792 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:48.049816 15792 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.046873 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:48.032864 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:48.036165 15983 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:48.036808 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:48.038290 16069 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.015058 16090 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:48.050449 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:48.050020 13840 leader_election.cc:336] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.765s (SENT)
I20260709 13:04:48.029911 16039 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:04:48.088351 13840 leader_election.cc:336] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.465s (SENT)
I20260709 13:04:48.089262 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:48.050177 15792 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:48.059157 16082 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.059441 16067 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:48.068856 16076 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:48.120517 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 72141.3us
I20260709 13:04:48.054773 16024 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:04:48.051172 16065 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
W20260709 13:04:48.127990 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 95226.7us
I20260709 13:04:48.129794 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:48.131508 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:04:48.128082 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 277820us
I20260709 13:04:48.132568 13947 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:48.136116 13948 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:48.137008 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.140445 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:48.142524 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:48.134825 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:48.160665 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:48.161734 13946 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:48.116763 16076 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:48.184959 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 50236.8us
W20260709 13:04:48.189006 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.142702 16082 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:48.124972 16067 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.126343 16062 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:48.185994 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 43555.1us
W20260709 13:04:48.201406 14524 leader_election.cc:336] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.177s (SENT)
W20260709 13:04:48.212316 14524 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.211s (SENT)
W20260709 13:04:48.212860 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 77728.1us
W20260709 13:04:48.213037 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.218268 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:48.219116 13868 raft_consensus.cc:2393] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:48.188568 16076 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:48.186098 15943 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:04:48.226542 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:48.226828 16109 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:48.228200 13875 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:48.119940 15763 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:48.213630 16069 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:48.122511 16090 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:48.239581 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:48.211982 16105 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:04:48.247082 13840 leader_election.cc:336] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.015s (SENT)
I20260709 13:04:48.255124 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:48.256772 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:48.230994 16111 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:48.243547 16094 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.225s	user 0.002s	sys 0.000s
I20260709 13:04:48.166009 16039 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:48.221803 16067 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:48.266816 13840 leader_election.cc:336] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.101s (SENT)
W20260709 13:04:48.293723 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 73669.8us
I20260709 13:04:48.254316 16105 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:48.338716 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71910.5us
I20260709 13:04:48.331274 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:48.257296 15986 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:48.343345 13915 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.629s (SENT)
I20260709 13:04:48.344355 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:04:48.331327 16025 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.787s	user 0.012s	sys 0.000s
I20260709 13:04:48.239840 15962 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:48.338579 16105 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:48.352936 15986 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:48.342257 16111 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:48.372738 15960 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:48.270699 16106 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.112s
I20260709 13:04:48.375204 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:48.237993 16082 consensus_queue.cc:1059] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:04:48.335516 16069 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:48.450389 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 107083us
W20260709 13:04:48.467880 13915 leader_election.cc:336] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.711s (SENT)
I20260709 13:04:48.468320 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:48.463313 13868 raft_consensus.cc:2368] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4.
W20260709 13:04:48.469760 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:48.453313 16085 log.cc:927] Time spent T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:04:48.472479 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:48.474749 13872 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:48.465397 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 289628us
I20260709 13:04:48.454952 16098 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:48.490662 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 312845us
I20260709 13:04:48.472465 16052 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:48.505704 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 316140us
I20260709 13:04:48.479743 16062 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:48.498878 13915 leader_election.cc:336] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.138s (SENT)
W20260709 13:04:48.478472 13943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.008s	sys 0.016s
I20260709 13:04:48.439236 16025 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:48.510993 15984 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:48.497797 13989 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:48.492442 16069 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:48.534111 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:48.534425 16069 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:48.534793 14555 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:48.535269 15986 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:48.507011 15908 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.535408 16069 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:48.536052 15908 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:48.536556 16069 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:48.529332 13989 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:48.531574 15962 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:48.532142 13945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:48.538417 16106 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:48.538679 13945 raft_consensus.cc:606] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:48.539146 13945 raft_consensus.cc:614] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:48.539239 16106 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.539351 15792 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:48.541508 15792 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:48.545210 14555 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
W20260709 13:04:48.545624 16061 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.002s
I20260709 13:04:48.547256 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:48.525784 16062 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:48.547186 13872 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:48.547956 13868 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:48.549808 13914 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:48.501420 16111 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:48.523082 13943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.009s	sys 0.016s
I20260709 13:04:48.558184 13948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "765348f2471a493d92e998132c0c746b"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:48.562718 13948 raft_consensus.cc:606] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:48.565109 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:48.558244 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:48.553076 15960 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:48.589151 13915 leader_election.cc:400] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4.
I20260709 13:04:48.589546 13915 leader_election.cc:403] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:04:48.568873 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.576440 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:48.581048 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:48.593086 13872 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:48.564877 16015 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:48.602069 16052 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4.
I20260709 13:04:48.603713 13866 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:48.605540 16093 raft_consensus.cc:993] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
W20260709 13:04:48.610670 16107 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:04:48.566664 16111 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:48.616652 16111 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:48.618508 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:48.629581 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1355) took 1071 ms. Trace:
I20260709 13:04:48.584121 13868 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:48.630125 13943 rpcz_store.cc:276] 0709 13:04:47.557953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.576458 (+ 18505us) service_pool.cc:224] Handling call
0709 13:04:47.660631 (+ 84173us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.921306 (+260675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.921344 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:47.921927 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:04:48.574522 (+652595us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:48.587281 (+ 12759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.594302 (+  7021us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:48.618773 (+ 24471us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b180040fc60. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:04:48.622035 (+  3262us) raft_consensus.cc:1685] finished
0709 13:04:48.622079 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.629495 (+  7416us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48463,"raft.queue_time_us":41154,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":66111,"spinlock_wait_cycles":3262208,"thread_start_us":827640,"threads_started":2,"wal-append.queue_time_us":638467}
W20260709 13:04:48.587168 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 330497us
I20260709 13:04:48.632510 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:48.546046 16061 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:48.633437 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:48.646442 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:48.645323 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:48.656122 16061 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:48.592361 15960 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:48.659083 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:48.659318 13866 raft_consensus.cc:606] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:48.659824 13866 raft_consensus.cc:614] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:48.659941 13987 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:48.661413 16062 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:48.661988 16015 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:48.663015 16015 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:48.670481 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 161743us
I20260709 13:04:48.594911 16089 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 13:04:48.671614 16127 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:48.623564 16093 raft_consensus.cc:1081] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
W20260709 13:04:48.633452 16064 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.962s	user 0.006s	sys 0.007s
I20260709 13:04:48.679941 13987 leader_election.cc:400] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8.
I20260709 13:04:48.674417 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:48.680794 13868 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:48.681151 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:48.634220 16039 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:48.694396 13868 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:04:48.696008 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:48.648350 16067 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:48.697238 14551 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:48.685348 15908 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:48.685432 15792 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:48.698355 16119 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.687508 15892 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:48.648077 16111 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:48.701508 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:48.702214 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:48.656216 16025 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:48.675997 16061 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:48.716284 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:48.725437 16067 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:48.735117 16067 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.735586 16067 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:48.702436 16127 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:48.740469 15960 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:48.745048 16076 consensus_meta.cc:287] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.000s
I20260709 13:04:48.745566 16076 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:48.746790 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:48.747570 13868 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:48.752203 16111 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:48.710309 16017 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:48.748881 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:48.764822 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48678.6us
I20260709 13:04:48.747061 15892 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:48.703944 16069 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:04:48.767915 15892 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:48.762149 16127 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.766638 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:48.770758 16118 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.771625 16017 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:48.773815 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:48.777279 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:48.777972 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.778682 14020 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:48.778578 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.779959 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.780040 14017 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:48.780668 14018 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:48.781529 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.782405 14018 raft_consensus.cc:2410] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:48.783782 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.784406 14022 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:48.784621 14016 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:04:48.784993 14016 pending_rounds.cc:85] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 1
I20260709 13:04:48.785209 14016 pending_rounds.cc:107] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:04:48.785451 14016 raft_consensus.cc:2889] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:48.785211 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.786623 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:48.787375 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.788002 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:48.788678 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.789409 16127 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:48.789711 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.791374 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:48.792289 14023 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:48.793713 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:48.794327 14023 raft_consensus.cc:2376] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 4. Re-sending same reply.
I20260709 13:04:48.786456 14022 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:48.795859 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.796324 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.796473 14015 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:04:48.796802 14018 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:48.797605 14022 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:48.801846 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:48.801996 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:48.806154 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.808799 14017 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:48.804749 14018 raft_consensus.cc:2955] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:48.811368 14020 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:48.811295 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:48.812534 14017 raft_consensus.cc:2410] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:48.814342 14021 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:48.793959 15701 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:48.826601 13948 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:48.819079 15701 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.827294 15701 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:48.832233 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:48.833470 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:48.747339 16119 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:48.840409 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.772687 16017 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:48.841353 14022 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:48.846184 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:48.846966 13944 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:48.843544 16118 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:48.851441 16118 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:48.832672 16131 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.181s	user 0.000s	sys 0.005s
I20260709 13:04:48.856033 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:48.806766 15960 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:04:48.862726 13868 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:48.795513 16111 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:48.877542 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:48.899976 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 49090.4us
I20260709 13:04:48.903398 14024 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:48.906688 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.907539 14024 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:48.910900 16119 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:48.912004 16109 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.685s	user 0.014s	sys 0.000s
I20260709 13:04:48.912499 16109 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:48.913657 16135 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:04:48.918757 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.919507 14021 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:48.920403 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.920960 14021 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:48.921720 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.926322 16076 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:48.928507 16118 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:48.932305 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:48.936798 13915 leader_election.cc:336] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.793s (SENT)
I20260709 13:04:48.931680 14021 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:04:48.965094 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:48.966630 13946 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:04:48.967154 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.980084 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:48.991585 14024 raft_consensus.cc:2410] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:48.904510 16120 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:04:49.012570 13868 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:49.013273 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:49.017510 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:49.018985 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:49.019793 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:49.020393 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:48.977896 14021 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49644
I20260709 13:04:49.031203 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.031498 14021 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:49.031899 14021 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:49.031982 14024 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:49.031221 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:49.034577 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:49.035125 13946 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:49.078282 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:49.011327 16120 consensus_queue.cc:1243] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:04:49.088420 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47994us
I20260709 13:04:49.098109 13946 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:49.098912 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:49.101104 14524 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.886s (SENT)
I20260709 13:04:49.101080 14018 raft_consensus.cc:2410] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:49.115020 13948 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:04:49.127884 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.129377 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:49.133071 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:49.133834 14016 raft_consensus.cc:2376] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 5. Re-sending same reply.
I20260709 13:04:49.135746 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.136361 14018 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:49.136634 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:49.137213 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.137467 14022 raft_consensus.cc:2376] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8. Re-sending same reply.
I20260709 13:04:49.139578 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:49.140298 14020 raft_consensus.cc:2376] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 4. Re-sending same reply.
I20260709 13:04:49.141072 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.141541 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:49.142398 14018 raft_consensus.cc:2376] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4. Re-sending same reply.
I20260709 13:04:49.143446 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.144245 14018 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:49.145367 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.145406 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:49.146201 14022 raft_consensus.cc:2376] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8. Re-sending same reply.
I20260709 13:04:49.146660 14018 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:49.086624 16118 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.100498 16089 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.146736 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.187251 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:49.199177 13944 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:04:49.143151 16061 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.191772 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:49.227671 14016 raft_consensus.cc:2410] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:49.191365 14020 raft_consensus.cc:2376] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 5. Re-sending same reply.
I20260709 13:04:49.229741 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.231288 14020 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:49.232930 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:49.234212 14016 raft_consensus.cc:2410] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:49.235726 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.242020 14016 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:49.237425 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:49.243327 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 99003.1us
I20260709 13:04:49.243667 14020 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:04:49.245299 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:49.249553 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.249486 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.088023 16102 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.113s
I20260709 13:04:49.255120 14024 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:49.258756 14018 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:49.250393 14016 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:04:49.260900 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.269714 14020 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:49.270718 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.271231 14020 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:49.187848 16118 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:49.296379 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52305.7us
I20260709 13:04:49.202494 16089 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:49.238246 16061 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:49.310484 16025 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:49.272107 14020 raft_consensus.cc:2955] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:49.299190 16145 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 13:04:49.300357 16067 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.003s
I20260709 13:04:49.386756 14022 raft_consensus.cc:2376] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 2. Re-sending same reply.
W20260709 13:04:49.435369 16121 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:04:49.436254 16147 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
W20260709 13:04:49.436764 16137 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.226s	user 0.000s	sys 0.000s
W20260709 13:04:49.438336 16136 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
W20260709 13:04:49.438992 16105 consensus_meta.cc:287] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.975s	user 0.014s	sys 0.001s
I20260709 13:04:49.439489 16105 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:49.441315 16109 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:49.441951 16109 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:49.443521 16089 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:49.445330 16118 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:49.467186 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.468117 14023 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:49.470849 14555 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:49.478920 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:49.479916 14023 raft_consensus.cc:2376] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 4. Re-sending same reply.
I20260709 13:04:49.481606 14023 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:49.482868 14023 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:49.389112 16025 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:49.485348 14023 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:04:49.487275 15701 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.659s	user 0.009s	sys 0.002s
I20260709 13:04:49.487762 15701 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:49.489774 15701 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:49.496516 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:49.390517 16067 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:49.501382 15710 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:49.515625 14019 raft_consensus.cc:2368] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4.
I20260709 13:04:49.498163 14021 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:49.524164 14016 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:49.546170 14015 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.008s	sys 0.005s
W20260709 13:04:49.547415 14015 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.009s	sys 0.005s
W20260709 13:04:49.552635 13915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.008s	sys 0.006s
W20260709 13:04:49.553267 13915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.008s	sys 0.006s
W20260709 13:04:49.553615 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 504804us
W20260709 13:04:49.556043 13914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.000s	sys 0.012s
W20260709 13:04:49.556562 13914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.001s	sys 0.012s
I20260709 13:04:49.577759 16129 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:49.578666 16129 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:49.584895 14021 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:49.586032 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:49.588479 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:49.589434 14555 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:49.555649 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:49.602931 16157 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.603677 16157 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:49.624593 14021 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:04:49.597476 16017 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.005s	sys 0.020s
I20260709 13:04:49.586382 16129 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:49.523355 16118 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:04:49.641827 13942 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.085s	sys 0.152s
W20260709 13:04:49.644455 13942 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.086s	sys 0.152s
I20260709 13:04:49.534111 15710 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:49.636173 16017 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.005s	sys 0.021s
I20260709 13:04:49.650878 14021 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:49.640336 16062 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:04:49.672205 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 149645us
W20260709 13:04:49.654600 16153 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:04:49.684448 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 332) took 1208 ms. Trace:
W20260709 13:04:49.684846 16151 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 13:04:49.688221 13947 rpcz_store.cc:276] 0709 13:04:48.476418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.476901 (+   483us) service_pool.cc:224] Handling call
0709 13:04:48.493626 (+ 16725us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:48.502502 (+  8876us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:48.502554 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:48.947708 (+445154us) spinlock_profiling.cc:243] Waited 8.63 ms on lock 0x7b180040fe40. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a
0709 13:04:48.948489 (+   781us) spinlock_profiling.cc:243] Waited 5.6 ms on lock 0x7b2c000e5d98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:04:48.949025 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.082991 (+133966us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:49.083038 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.083116 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.684122 (+601006us) raft_consensus.cc:1685] finished
0709 13:04:49.684153 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.684359 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15380,"raft.queue_time_us":12940,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":12583,"spinlock_wait_cycles":32727296,"thread_start_us":133526,"threads_started":1,"wal-append.queue_time_us":687476}
I20260709 13:04:49.711133 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:49.717259 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:49.684515 15953 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:49.741503 14555 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:49.670678 16067 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:49.775350 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.776475 14018 raft_consensus.cc:2410] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:49.777354 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.777902 14018 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
W20260709 13:04:49.778570 16146 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.196s	user 0.001s	sys 0.001s
I20260709 13:04:49.778805 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:49.780083 16157 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:49.780431 14018 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:04:49.780807 14018 pending_rounds.cc:85] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 1
I20260709 13:04:49.781059 14018 pending_rounds.cc:107] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:04:49.780962 16157 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:49.781417 16155 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:04:49.781533 16157 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:49.782090 16157 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.785089 14018 raft_consensus.cc:2889] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:49.800946 14018 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:49.804085 14018 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:49.684701 16062 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:49.831230 13840 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.874s (SENT)
W20260709 13:04:49.808172 16141 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
W20260709 13:04:49.840541 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 285154us
I20260709 13:04:49.847674 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:49.849395 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:49.849431 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:49.852605 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:49.855490 13873 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:49.820673 16052 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:49.858355 16052 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:49.858830 16052 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:49.808571 15943 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:04:49.874851 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:49.876591 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:49.847780 16118 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:04:49.888794 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 46073.4us
I20260709 13:04:49.890415 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:49.860042 16108 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:49.891819 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 152909us
W20260709 13:04:49.895363 16156 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 13:04:49.895676 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.896044 16174 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.893757 16102 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:04:49.896718 16174 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:49.898967 16140 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:04:49.899142 16174 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:49.899376 16162 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:49.895212 16118 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:49.901723 16108 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:49.893677 16175 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:49.904438 16110 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:49.908020 16108 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:49.913301 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 287) took 1650 ms. Trace:
W20260709 13:04:49.919924 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 202814us
I20260709 13:04:49.923126 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:49.923921 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:49.889554 16025 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:49.937007 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:49.890491 16050 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:49.944104 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:49.911518 16015 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.904560 16175 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:49.920306 16129 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:49.919306 13870 rpcz_store.cc:276] 0709 13:04:48.262685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.262857 (+   172us) service_pool.cc:224] Handling call
0709 13:04:48.263347 (+   490us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:48.263789 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:48.263815 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:49.142670 (+878855us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b2c00000658. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:04:49.189122 (+ 46452us) log.cc:844] Serialized 52 byte log entry
0709 13:04:49.556765 (+367643us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:49.740500 (+183735us) log.cc:844] Serialized 14 byte log entry
0709 13:04:49.912725 (+172225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.912843 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.912923 (+    80us) raft_consensus.cc:1685] finished
0709 13:04:49.913028 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.913224 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10633,"raft.queue_time_us":190213,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":25904,"spinlock_wait_cycles":5714304,"thread_start_us":652076,"threads_started":3,"wal-append.queue_time_us":552239}
I20260709 13:04:49.953418 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:04:49.988163 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 51249us
I20260709 13:04:49.939560 16093 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:49.990577 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:49.952021 16050 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:49.994438 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:49.995849 16050 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:49.995867 13987 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:49.997781 15710 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:49.998164 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:49.998750 15892 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:49.999157 15892 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:50.003584 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 127096us
I20260709 13:04:49.999496 15892 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:49.925473 16009 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:04:49.970135 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:50.013437 16176 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:50.014885 16165 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:04:50.016134 16189 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:50.018445 16183 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:50.018937 16178 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:04:50.019352 16100 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:49.989382 16187 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:50.007766 13840 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:49.990574 15983 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:50.041947 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:50.025673 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 55687.1us
I20260709 13:04:50.027762 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:49.952179 16175 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:50.044601 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:50.044736 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:49.960687 16015 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:50.062228 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 61907us
I20260709 13:04:50.045658 15983 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:50.065771 13840 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:50.067535 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:50.069065 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:50.038398 16100 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:50.067833 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:50.068462 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:50.070680 13944 raft_consensus.cc:3055] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:50.069780 13948 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:50.040042 16093 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:50.072685 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:50.071102 13944 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:50.069756 13941 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:50.073526 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:50.076490 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:50.060526 15953 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:50.064930 16102 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:50.081477 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:50.048357 16181 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:50.066984 16164 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:04:50.048969 16129 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:50.117995 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:50.079455 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:50.078578 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:50.119213 13875 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:50.216019 13875 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:50.140597 13948 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:50.216565 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:50.074990 13944 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:50.217530 13874 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:50.218200 13944 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:50.113306 16170 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:50.117703 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:50.222615 13868 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:50.147828 13869 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:50.026980 16176 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:50.143606 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62240.8us
I20260709 13:04:50.150827 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:50.072283 16100 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:50.113444 16190 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:50.122071 16108 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.077610 16062 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:50.049310 16175 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:50.049479 16118 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:50.251151 14556 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
W20260709 13:04:50.221092 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 141731us
I20260709 13:04:50.072283 15983 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:50.142995 16102 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.222455 15953 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:50.275600 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:50.276553 14018 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:50.237002 16181 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:50.237206 16191 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:50.153008 16120 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:50.287338 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:50.288285 14018 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:50.151930 16165 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:04:50.065579 16183 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 13:04:50.118263 15960 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:50.247906 15710 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:50.248799 16176 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:50.255434 16108 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:50.256421 16062 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:50.330370 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:50.332705 16110 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:50.258404 16169 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.334496 16169 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.334960 16052 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.336951 16169 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:50.338899 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:50.204869 16157 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:50.260746 13941 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:50.262538 16196 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.268623 13944 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:50.216328 16089 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:04:50.278035 16105 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:50.279119 16015 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:50.280827 16102 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.076251 16093 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:50.291797 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47973.7us
I20260709 13:04:50.292127 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:50.245952 16154 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:50.269488 16118 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:50.284497 16187 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:50.326772 16108 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:50.324539 15710 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:50.345302 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:50.322722 16197 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.382261 13941 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:50.346813 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:50.347482 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:50.348991 14018 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:50.386137 13944 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:50.386646 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:50.387287 13944 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 13:04:50.319444 16171 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:04:50.355415 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2108) took 1571 ms. Trace:
I20260709 13:04:50.362802 16093 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:50.363924 14555 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:50.299139 15960 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:50.246364 16170 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.367733 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:50.384188 13943 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:50.389906 14015 rpcz_store.cc:276] 0709 13:04:48.783795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.797282 (+ 13487us) service_pool.cc:224] Handling call
0709 13:04:48.814944 (+ 17662us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:48.815341 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:48.815372 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:48.815856 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.548535 (+732679us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:49.550035 (+  1500us) log.cc:844] Serialized 14 byte log entry
0709 13:04:49.629964 (+ 79929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.630127 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.355179 (+725052us) raft_consensus.cc:1685] finished
0709 13:04:50.355202 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.355348 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":470,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":312886,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":18238,"thread_start_us":810505,"threads_started":2,"wal-append.queue_time_us":1009249}
I20260709 13:04:50.299949 15953 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:50.409201 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:50.418545 16100 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.422430 16192 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:50.423064 16192 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.424129 16100 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:50.425134 16052 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:50.397663 15960 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:50.426635 16186 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:50.399153 16197 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.399339 15892 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.410681 16015 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:50.430522 16090 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:50.379971 16105 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:50.379199 16118 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.420583 16127 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:50.425254 16192 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:50.363608 16157 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:50.381232 16062 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:04:50.450774 16202 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:50.357481 16196 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:50.446619 16186 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.458288 16203 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:50.452041 16108 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:50.398231 15710 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:50.458422 15983 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.460506 16118 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:50.460186 16105 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:50.460933 16127 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.470319 13947 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:50.388329 16154 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:50.453282 16015 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.482249 13946 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:50.458458 16204 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:50.474381 16203 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:50.477999 16110 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:04:50.493971 16076 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:50.495110 16203 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:50.497512 16105 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:50.500511 15710 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:50.477734 16205 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:04:50.494867 16076 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:50.510835 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:50.511229 15983 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:50.512188 16109 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:50.392503 16187 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:50.527508 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 219) took 1559 ms. Trace:
W20260709 13:04:50.530239 13840 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.149s (SENT)
I20260709 13:04:50.528729 14017 rpcz_store.cc:276] 0709 13:04:48.967784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.262896 (+295112us) service_pool.cc:224] Handling call
0709 13:04:49.263210 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.391133 (+127923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.391198 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.391873 (+   675us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.589144 (+197271us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:49.598908 (+  9764us) log.cc:844] Serialized 14 byte log entry
0709 13:04:49.780187 (+181279us) spinlock_profiling.cc:243] Waited 6.54 ms on lock 0x7b1800390100. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287 00007fb2c344e208
0709 13:04:49.838209 (+ 58022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.838305 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.482765 (+644460us) raft_consensus.cc:1685] finished
0709 13:04:50.527263 (+ 44498us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.527443 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6453,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":4197,"raft.queue_time_us":311710,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":10796,"spinlock_wait_cycles":15048064,"thread_start_us":277020,"threads_started":3,"wal-append.queue_time_us":600411}
I20260709 13:04:50.519637 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:50.482002 16069 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:50.526897 16109 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:50.534883 16105 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.474676 16202 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:50.489780 16062 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:04:50.546155 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:51.287511 13916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.004s	sys 0.026s
W20260709 13:04:51.417687 13916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.973s	user 0.007s	sys 0.027s
W20260709 13:04:51.418229 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.418784 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.699467 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:50.585147 14023 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:51.421782 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2248) took 1276 ms. Trace:
I20260709 13:04:50.585053 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2086) took 1808 ms. Trace:
I20260709 13:04:50.530405 16204 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:51.054663 16108 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:04:51.111305 13990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.004s	sys 0.024s
W20260709 13:04:51.048506 14020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.033s	sys 0.035s
W20260709 13:04:51.118848 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 74701us
I20260709 13:04:50.505226 16118 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:50.693140 16120 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:04:51.418031 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 871915us
W20260709 13:04:51.440014 13915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.999s	user 0.000s	sys 0.014s
W20260709 13:04:51.448416 13915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.000s	sys 0.014s
W20260709 13:04:50.990502 16090 consensus_meta.cc:287] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.004s
I20260709 13:04:51.448729 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:50.532385 16129 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:51.431147 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 900844us
W20260709 13:04:51.425475 14018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.037s	user 0.014s	sys 0.023s
W20260709 13:04:51.292115 13988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.007s	sys 0.007s
W20260709 13:04:51.458634 13988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.007s	sys 0.007s
W20260709 13:04:51.448743 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 1.03709e+06us
W20260709 13:04:51.461552 14018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.073s	user 0.014s	sys 0.023s
W20260709 13:04:51.427217 16183 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.051s	user 0.003s	sys 0.014s
I20260709 13:04:51.422304 13871 rpcz_store.cc:276] 0709 13:04:50.145529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.145736 (+   207us) service_pool.cc:224] Handling call
0709 13:04:50.146000 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.183964 (+ 37964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.184015 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:50.184077 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:50.184137 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.421533 (+1237396us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b54000e9a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:51.421583 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.421718 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85525248}
I20260709 13:04:51.423658 14019 rpcz_store.cc:276] 0709 13:04:48.776949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.778220 (+  1271us) service_pool.cc:224] Handling call
0709 13:04:50.584764 (+1806544us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b540023f854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:50.584974 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684295936}
I20260709 13:04:51.417009 16209 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:04:51.441143 13990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.076s	user 0.010s	sys 0.029s
W20260709 13:04:51.442204 14020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.033s	sys 0.045s
I20260709 13:04:51.446825 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:51.456167 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:50.726358 16201 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.259s	user 0.000s	sys 0.000s
I20260709 13:04:51.469497 13944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
W20260709 13:04:51.304059 16093 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.885s	user 0.012s	sys 0.001s
I20260709 13:04:51.542199 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:51.544674 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:51.554526 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 99595.9us
W20260709 13:04:50.715412 16184 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.356s	user 0.000s	sys 0.001s
W20260709 13:04:51.092190 15960 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.647s	user 0.012s	sys 0.000s
I20260709 13:04:50.910295 15953 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:51.300930 16102 consensus_meta.cc:287] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.938s	user 0.009s	sys 0.005s
I20260709 13:04:51.547466 13944 raft_consensus.cc:606] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:51.555894 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:50.931766 16197 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:51.556831 14019 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
W20260709 13:04:50.960243 16207 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.355s	user 0.000s	sys 0.001s
I20260709 13:04:51.057854 16205 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.422423 16221 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:50.732260 16015 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:50.969236 16187 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:51.425959 16186 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:51.439388 16195 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.968s	user 0.000s	sys 0.014s
I20260709 13:04:51.420403 16215 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:50.930886 16154 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:51.434428 16105 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:51.431326 16109 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:51.559036 13944 raft_consensus.cc:614] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:51.790346 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:51.792011 14523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.006s	sys 0.015s
W20260709 13:04:51.792520 14523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.007s	sys 0.015s
I20260709 13:04:51.792631 13944 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:51.782725 16213 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:51.794592 16213 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:51.797432 16213 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:51.798966 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 1980) took 2783 ms. Trace:
W20260709 13:04:51.559612 13913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.006s	sys 0.006s
W20260709 13:04:51.808503 13913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.368s	user 0.011s	sys 0.008s
I20260709 13:04:51.467851 13950 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:51.799407 13942 rpcz_store.cc:276] 0709 13:04:49.015864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.036651 (+ 20787us) service_pool.cc:224] Handling call
0709 13:04:49.037059 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.665906 (+628847us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.665944 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.676033 (+ 10089us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.938688 (+262655us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:49.952120 (+ 13432us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.076663 (+124543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.076794 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.501057 (+424263us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:04:51.798665 (+1297608us) raft_consensus.cc:1685] finished
0709 13:04:51.798765 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.798907 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":692548,"prepare.queue_time_us":9589,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":99225,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":26235,"spinlock_wait_cycles":11888000,"thread_start_us":877885,"threads_started":3,"wal-append.queue_time_us":310445}
W20260709 13:04:51.811204 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 394464us
I20260709 13:04:51.596071 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:51.553910 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:50.694361 15983 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:51.423067 16202 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:51.837363 13941 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
W20260709 13:04:50.929271 16175 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.665s	user 0.009s	sys 0.001s
W20260709 13:04:51.554579 16183 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.192s	user 0.004s	sys 0.016s
I20260709 13:04:51.424172 16204 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.858119 16205 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:50.535508 16191 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:50.694613 16196 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:51.541033 16024 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:50.923697 16170 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:51.735977 16076 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:51.837117 16195 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.383s	user 0.008s	sys 0.015s
W20260709 13:04:51.852064 16111 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.379s	user 0.018s	sys 0.036s
I20260709 13:04:51.782634 16214 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:51.759269 16210 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:51.887293 16015 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:51.704825 15892 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:51.810892 16199 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.834731 16225 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.938133 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 384311us
I20260709 13:04:50.778561 16127 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:51.836448 16169 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.277s	user 0.030s	sys 0.044s
I20260709 13:04:51.737589 16212 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:51.860910 16187 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.597606 16219 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:51.910276 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1352) took 1833 ms. Trace:
I20260709 13:04:51.836752 16213 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:51.858037 16154 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:51.882256 16183 consensus_queue.cc:1243] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:04:51.922955 16231 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.054329 16130 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:51.749552 16208 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.173s
I20260709 13:04:51.601028 15960 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:51.461910 16108 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:51.916759 16111 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.484s	user 0.019s	sys 0.038s
I20260709 13:04:51.906814 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:51.423774 16069 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:51.654238 16211 consensus_queue.cc:1059] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.286s
I20260709 13:04:51.898939 16202 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:51.569026 16093 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:51.472785 16118 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:51.934037 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.934176 16129 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.899236 16215 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:51.930245 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 374585us
I20260709 13:04:51.907457 16196 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:51.853644 16221 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:51.959924 16212 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:51.858762 16102 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:51.919401 16186 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:51.945473 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:51.940331 16015 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.961463 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:51.889348 16175 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:51.889568 15983 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:51.930936 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:51.958916 14020 rpcz_store.cc:276] 0709 13:04:50.076693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.076812 (+   119us) service_pool.cc:224] Handling call
0709 13:04:50.077146 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.077529 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.077560 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.078282 (+   722us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.296544 (+218262us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:50.330265 (+ 33721us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.542318 (+1212053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.542402 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.909927 (+367525us) raft_consensus.cc:1685] finished
0709 13:04:51.909962 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.910196 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24034,"prepare.queue_time_us":40497,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":149944,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":39196,"thread_start_us":1284493,"threads_started":2,"wal-append.queue_time_us":276734}
I20260709 13:04:51.905608 16024 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:51.932917 16228 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:51.947335 16169 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.417s	user 0.030s	sys 0.044s
I20260709 13:04:51.942797 16209 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:52.038569 13948 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:51.935719 16210 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:51.929175 16214 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:52.008744 15960 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:51.981556 16232 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:52.031247 16211 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:52.031042 16216 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:04:52.031634 14553 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:51.978075 16183 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:51.992769 16208 consensus_queue.cc:1243] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:52.036999 16235 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:51.972905 16213 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:51.973208 16154 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.026948 16015 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:52.012841 16127 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:51.945622 16199 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:51.967906 16187 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:52.028934 16186 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:51.946688 16120 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.028360 16129 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.028523 16202 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.028638 16196 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.058431 16235 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:52.049612 16211 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:52.059000 16212 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:51.948364 16225 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:52.057593 16105 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:52.074662 16241 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:04:52.003528 16231 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:51.986380 16191 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.037871 16215 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.047013 16232 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:52.047912 16228 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:52.059087 16183 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.032845 16130 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:51.995875 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:52.022929 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 77597us
W20260709 13:04:52.037848 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.047113 13987 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:52.047816 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2131) took 1773 ms. Trace:
I20260709 13:04:52.044498 14551 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:04:52.059229 16061 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:52.063560 16109 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:52.085754 16187 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:52.088575 16183 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:52.078210 16205 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:52.085914 16210 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:52.081600 16211 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.081832 16214 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:52.083250 16222 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.109s	user 0.001s	sys 0.001s
I20260709 13:04:52.095777 16225 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:52.093726 16226 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:04:52.088595 16231 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:52.085975 16232 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:52.077685 16235 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.063691 16127 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.087484 16209 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:52.097051 16237 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.149093 13868 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:52.630918 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:52.627357 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:52.630743 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:52.132659 13870 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:52.306792 14021 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.136751 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:52.621455 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:52.465499 16204 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.560s	user 0.020s	sys 0.002s
W20260709 13:04:52.131471 13839 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.001s	sys 0.003s
I20260709 13:04:52.095893 16221 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:52.421227 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:52.134547 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:52.263578 14019 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:52.323992 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
W20260709 13:04:52.234078 13950 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 1990) took 2161 ms (client timeout 2774 ms). Trace:
I20260709 13:04:52.193310 16186 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:52.682152 16109 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:52.682701 16109 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:52.683236 16109 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:52.653445 16215 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:52.654100 16118 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:52.452538 16050 raft_consensus.cc:993] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:52.685804 16050 raft_consensus.cc:1081] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:04:52.233126 16233 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:52.215675 16212 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:52.168164 16213 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:52.660818 14552 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:52.662006 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:52.212693 13946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1390) took 1880 ms (client timeout 1649 ms). Trace:
I20260709 13:04:52.214233 16228 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:52.663388 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:52.419494 16202 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:52.709468 16115 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.709712 16212 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.337507 15983 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:52.710410 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:52.666200 13871 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:52.166700 16108 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:52.667006 16205 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:52.728648 14555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 344) took 2778 ms. Trace:
I20260709 13:04:52.728360 16205 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.668241 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:52.657476 16237 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:52.623327 16188 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 2.609s	user 0.016s	sys 0.018s
I20260709 13:04:52.216073 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1381) took 1913 ms. Trace:
I20260709 13:04:52.652942 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:52.184556 16129 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:52.187428 16069 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:52.235044 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1388) took 1903 ms. Trace:
I20260709 13:04:52.214588 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1380) took 1912 ms. Trace:
I20260709 13:04:52.164296 16102 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:52.673027 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:52.440171 16082 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.674340 14558 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
W20260709 13:04:52.674714 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 351121us
I20260709 13:04:52.674804 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:52.672065 13950 rpcz_store.cc:269] 0709 13:04:50.072221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.073271 (+  1050us) service_pool.cc:224] Handling call
0709 13:04:52.233547 (+2160276us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400088754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:52.233828 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3204976512}
I20260709 13:04:52.676363 16204 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.245873 16024 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:52.293530 14022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1326) took 3199 ms (client timeout 2149 ms). Trace:
I20260709 13:04:52.671797 14019 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:52.806627 13949 rpcz_store.cc:276] 0709 13:04:50.302816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.303010 (+   194us) service_pool.cc:224] Handling call
0709 13:04:50.303412 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.441083 (+137671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.441127 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:50.441182 (+    55us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:50.441234 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.215325 (+1774091us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54001c6e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:52.215428 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:52.215788 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":316046720}
W20260709 13:04:52.130987 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 109887us
I20260709 13:04:52.234542 16093 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:52.694940 16233 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:52.706553 16096 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.707967 14560 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:52.708313 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:52.708663 15960 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.189810 13875 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:52.725286 16208 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:52.726826 16191 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.803751 13947 rpcz_store.cc:276] 0709 13:04:50.302325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.302517 (+   192us) service_pool.cc:224] Handling call
0709 13:04:50.329920 (+ 27403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.478689 (+148769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.213721 (+1735032us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54001f4fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:52.214255 (+   534us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322239360}
I20260709 13:04:52.729163 14555 rpcz_store.cc:276] 0709 13:04:49.949944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.950249 (+   305us) service_pool.cc:224] Handling call
0709 13:04:49.950676 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.391571 (+440895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.728068 (+2336497us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54007102d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:52.728423 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949683456}
W20260709 13:04:52.694311 13946 rpcz_store.cc:269] 0709 13:04:50.332377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.338641 (+  6264us) service_pool.cc:224] Handling call
0709 13:04:52.211967 (+1873326us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b5400228b54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:52.212291 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328040704}
I20260709 13:04:52.247852 16175 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:52.822454 16233 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:52.811758 14022 rpcz_store.cc:269] 0709 13:04:49.093347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.186977 (+ 93630us) service_pool.cc:224] Handling call
0709 13:04:52.292846 (+3105869us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b540021cfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:52.293108 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":447230848}
I20260709 13:04:52.804463 13948 rpcz_store.cc:276] 0709 13:04:50.331886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.361510 (+ 29624us) service_pool.cc:224] Handling call
0709 13:04:50.361786 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.039128 (+1677342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.234394 (+195266us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b540022ae54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:52.234828 (+   434us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3855548672}
I20260709 13:04:52.814381 14553 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:52.814520 16082 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:52.815577 16096 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:52.829303 13941 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:52.828785 16096 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:52.809478 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:52.833752 14019 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.819636 13869 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
W20260709 13:04:52.729887 16015 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.666s	user 0.007s	sys 0.007s
W20260709 13:04:52.810595 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 763336us
I20260709 13:04:52.826305 13868 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:52.826598 16067 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:52.836395 14019 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:52.835217 13942 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:04:52.839748 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:52.710352 16115 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.840636 16191 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.834295 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:52.841735 16208 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.845980 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:52.836726 16067 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.835620 16205 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:52.846314 15960 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:52.857646 13839 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.346s	user 0.010s	sys 0.011s
I20260709 13:04:52.856307 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:52.857882 13875 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:52.841732 16220 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:52.855882 16067 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:52.894768 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 186836us
I20260709 13:04:52.868337 16233 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:52.895840 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 1.38477e+06us
I20260709 13:04:52.903198 16115 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:52.903985 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:52.905332 16109 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:52.905771 13872 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:52.907588 15960 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:52.911203 13868 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:52.846683 16015 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.898340 16233 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:52.892103 16211 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:52.838745 14018 rpcz_store.cc:276] 0709 13:04:50.274881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.387539 (+112658us) service_pool.cc:224] Handling call
0709 13:04:50.387810 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:51.462236 (+1074426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.462287 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:51.462989 (+   702us) log.cc:844] Serialized 26 byte log entry
0709 13:04:51.900408 (+437419us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:51.900464 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.900566 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:52.047593 (+147027us) raft_consensus.cc:1685] finished
0709 13:04:52.047618 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:52.047751 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105536,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"thread_start_us":1510861,"threads_started":2,"wal-append.queue_time_us":514821}
I20260709 13:04:52.933135 14022 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:52.925582 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1371) took 4031 ms. Trace:
I20260709 13:04:52.900810 16212 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:52.942216 16241 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.943205 16241 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:52.946897 16082 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:52.947577 16196 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:52.947767 16191 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:52.951153 16127 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:52.935652 16233 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:52.974551 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 306834us
I20260709 13:04:52.934123 13945 rpcz_store.cc:276] 0709 13:04:48.829448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.829564 (+   116us) service_pool.cc:224] Handling call
0709 13:04:48.829859 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:48.831574 (+  1715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:48.831608 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:48.832124 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.016687 (+184563us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:49.016733 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.016833 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:52.841324 (+3824491us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b2400249fc8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c01cc9c4 00007fb2c01caacb 00007fb2c0385dc9 00007fb2c0385a55 00007fb2c0384fb0 00007fb2c2ffb488 00007fb2c2ffbbe8 00007fb2c2fd3398 00007fb2c2fce76a 00007fb2c2fcdbb6
0709 13:04:52.843587 (+  2263us) raft_consensus.cc:1685] finished
0709 13:04:52.843612 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:52.860830 (+ 17218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":770,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":7212,"spinlock_wait_cycles":3579648,"thread_start_us":119232,"threads_started":1,"wal-append.queue_time_us":574249}
I20260709 13:04:52.977136 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1332) took 3861 ms. Trace:
I20260709 13:04:52.974120 16241 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:52.988734 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:52.991060 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:52.991856 16187 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.993062 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:52.993716 14558 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:52.994105 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:52.996390 14551 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:52.997625 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:52.998392 14561 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:52.998807 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:52.978727 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:52.992724 16187 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:52.981211 14021 rpcz_store.cc:276] 0709 13:04:49.094661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.679122 (+584461us) service_pool.cc:224] Handling call
0709 13:04:52.293284 (+2614162us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.649423 (+356139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.949974 (+300551us) spinlock_profiling.cc:243] Waited 2.61 s on lock 0x7b540021cfd0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:52.955939 (+  5965us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5996463488}
I20260709 13:04:53.010144 16174 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:53.011065 16174 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:53.012387 16226 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:04:53.013854 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1356) took 1550 ms. Trace:
I20260709 13:04:52.970407 16204 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:52.961380 16208 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:53.024730 16109 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:53.035669 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 231317us
I20260709 13:04:53.035943 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2126) took 4089 ms. Trace:
I20260709 13:04:53.036448 14016 rpcz_store.cc:276] 0709 13:04:48.946821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.261494 (+314673us) service_pool.cc:224] Handling call
0709 13:04:49.261809 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.524955 (+263146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.035339 (+3510384us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b540023ad54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:53.035731 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":602801024}
W20260709 13:04:53.025061 16194 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 2.879s	user 0.009s	sys 0.017s
I20260709 13:04:53.040964 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.042253 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.042909 16154 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:53.043051 13949 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:53.046746 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2133) took 2760 ms. Trace:
I20260709 13:04:53.047415 14015 rpcz_store.cc:276] 0709 13:04:50.286345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.407507 (+121162us) service_pool.cc:224] Handling call
0709 13:04:50.407931 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.479641 (+ 71710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.479688 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:50.479744 (+    56us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:50.479795 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.046151 (+2566356us) spinlock_profiling.cc:243] Waited 71.4 ms on lock 0x7b5400297cd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:53.046243 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.046495 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164295040}
I20260709 13:04:53.013864 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2134) took 2727 ms. Trace:
I20260709 13:04:53.049028 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.026569 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.050669 13945 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:53.029798 13988 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:53.031117 13875 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:04:53.049510 14017 rpcz_store.cc:276] 0709 13:04:50.286641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.531255 (+244614us) service_pool.cc:224] Handling call
0709 13:04:50.531728 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.535357 (+  3629us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.535391 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:51.432099 (+896708us) log.cc:844] Serialized 26 byte log entry
0709 13:04:51.566398 (+134299us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:51.918877 (+352479us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.919117 (+   240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.919201 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.013417 (+1094216us) raft_consensus.cc:1685] finished
0709 13:04:53.013463 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.013650 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15713,"prepare.queue_time_us":400017,"prepare.run_wall_time_us":4434,"raft.queue_time_us":927318,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":10021,"spinlock_wait_cycles":32233728,"thread_start_us":351770,"threads_started":2,"wal-append.queue_time_us":509912}
I20260709 13:04:53.031960 14558 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:04:53.032269 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1317) took 3947 ms. Trace:
I20260709 13:04:53.052697 14023 rpcz_store.cc:276] 0709 13:04:49.085232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.486004 (+400772us) service_pool.cc:224] Handling call
0709 13:04:50.584938 (+1098934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.421071 (+836133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.031851 (+1610780us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540023f850. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:53.032056 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2512924928}
W20260709 13:04:53.030040 13838 connection.cc:256] RPC call timeout handler was delayed by -1.907754223999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
I20260709 13:04:53.031502 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:53.034257 14556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59630 (request call id 346) took 2883 ms (client timeout 2071 ms). Trace:
W20260709 13:04:53.054316 13838 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.66:43953 timed out after 1.649s (SENT) [suppressed 24 similar messages]
I20260709 13:04:53.051707 13875 pending_rounds.cc:85] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 8
W20260709 13:04:53.054636 14556 rpcz_store.cc:269] 0709 13:04:50.150426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.150578 (+   152us) service_pool.cc:224] Handling call
0709 13:04:53.033822 (+2883244us) spinlock_profiling.cc:243] Waited 8.96 ms on lock 0x7b5400584b54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:53.033980 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20601600}
I20260709 13:04:53.055684 13875 pending_rounds.cc:107] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:04:53.056068 13875 raft_consensus.cc:2889] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:53.057005 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2246) took 2938 ms. Trace:
I20260709 13:04:53.057344 13875 rpcz_store.cc:276] 0709 13:04:50.118689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.118812 (+   123us) service_pool.cc:224] Handling call
0709 13:04:50.119088 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.188197 (+2069109us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b54004a40a0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33b106f 00007fb2c3429f60 00007fb2c342b7de 00007fb2c3435b49 00007fb2c3436416 00007fb2c3432d8f 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62
0709 13:04:53.056633 (+868436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.056918 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11865344}
W20260709 13:04:53.058483 15763 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:04:53.033898 16015 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:53.061367 13866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2243) took 2943 ms (client timeout 1787 ms). Trace:
W20260709 13:04:53.061784 13866 rpcz_store.cc:269] 0709 13:04:50.117591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.117773 (+   182us) service_pool.cc:224] Handling call
0709 13:04:53.060932 (+2943159us) spinlock_profiling.cc:243] Waited 2.78 s on lock 0x7b540026cfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:53.061120 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6388943232}
I20260709 13:04:53.067680 16237 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:53.054901 14018 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:04:53.069886 14022 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:04:53.082527 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 227) took 1027 ms. Trace:
I20260709 13:04:53.083184 14018 rpcz_store.cc:276] 0709 13:04:52.042973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.014952 (+971979us) service_pool.cc:224] Handling call
0709 13:04:53.070547 (+ 55595us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.084347 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 229) took 1035 ms. Trace:
I20260709 13:04:53.081707 16109 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:53.090669 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 129050us
I20260709 13:04:53.091040 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 220) took 4112 ms. Trace:
I20260709 13:04:53.091727 16241 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:53.092357 16076 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:53.093149 16199 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:53.085493 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 81065.3us
W20260709 13:04:53.093166 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 63516.3us
W20260709 13:04:53.099876 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 61836.6us
I20260709 13:04:53.014415 14020 rpcz_store.cc:276] 0709 13:04:51.463040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.029335 (+566295us) service_pool.cc:224] Handling call
0709 13:04:52.029601 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:52.029975 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:52.030003 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:52.038529 (+  8526us) log.cc:844] Serialized 26 byte log entry
0709 13:04:52.693439 (+654910us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:52.955197 (+261758us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.959708 (+  4511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.975856 (+ 16148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:52.975925 (+    69us) raft_consensus.cc:1685] finished
0709 13:04:52.975943 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.013631 (+ 37688us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33920,"raft.queue_time_us":4597,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":20448,"thread_start_us":602670,"threads_started":2,"wal-append.queue_time_us":587490}
I20260709 13:04:53.098474 16082 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:53.114946 16199 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:53.115823 16199 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.125025 14522 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:53.109400 14022 rpcz_store.cc:276] 0709 13:04:52.043505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.845532 (+802027us) service_pool.cc:224] Handling call
0709 13:04:53.078941 (+233409us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.109710 16082 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:53.123136 16076 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.085144 16067 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:53.101276 14024 rpcz_store.cc:276] 0709 13:04:48.978658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.264349 (+285691us) service_pool.cc:224] Handling call
0709 13:04:49.272846 (+  8497us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.318529 (+ 45683us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:49.319555 (+  1026us) log.cc:844] Serialized 14 byte log entry
0709 13:04:49.921144 (+601589us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.921774 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.922309 (+   535us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:49.923781 (+  1472us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.424175 (+2500394us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b48002115d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:04:53.039040 (+614865us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b1800332aa0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c2fe9d70 00007fb2c2fdd293 00007fb2c2fd2cea 00007fb2c2fd323a 00007fb2c343da2a 00007fb2c344d0bf 00007fb2c344d038
0709 13:04:53.090492 (+ 51452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.090614 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.090678 (+    64us) raft_consensus.cc:1685] finished
0709 13:04:53.090692 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.090837 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2799355,"prepare.queue_time_us":3641,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":105,"raft.queue_time_us":286330,"raft.run_cpu_time_us":27316,"raft.run_wall_time_us":3124935,"spinlock_wait_cycles":7147648,"thread_start_us":629492,"threads_started":3,"wal-append.queue_time_us":543799}
I20260709 13:04:53.130661 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:53.110792 16069 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.110122 16174 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:53.142714 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.143708 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:53.141604 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.152781 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:53.156937 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 47666.5us
I20260709 13:04:53.155575 13988 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:53.149114 16082 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:53.163931 14019 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:04:53.144292 16241 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:53.143999 16237 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:53.170800 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.167312 14019 pending_rounds.cc:85] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 3
I20260709 13:04:53.172535 14019 pending_rounds.cc:107] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:04:53.172837 14019 raft_consensus.cc:2889] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:53.168823 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 54367.6us
I20260709 13:04:53.166774 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:53.170841 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:53.176539 14552 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:53.176733 16242 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:53.177297 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.177886 16197 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:53.178258 13874 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:04:53.178354 16197 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.178443 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.178805 16197 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:53.179201 13875 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:53.179279 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.181972 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.183982 16061 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:53.184188 16130 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:53.184572 16061 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:53.185112 16061 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:53.187393 16189 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.177224 16127 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.187822 16187 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:53.188122 16189 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:53.188228 16127 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:53.190187 16180 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:53.190830 13839 connection.cc:256] RPC call timeout handler was delayed by -1.0069249970000556s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20710000000000003s for a response
I20260709 13:04:53.191143 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.192189 13871 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:53.191133 16127 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:53.195350 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:53.195747 13915 connection.cc:256] RPC call timeout handler was delayed by -1.4672591099999863s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1718s for a response
W20260709 13:04:53.196241 13915 connection.cc:256] RPC call timeout handler was delayed by -1.4051591099999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
W20260709 13:04:53.196587 13915 connection.cc:256] RPC call timeout handler was delayed by -1.3034591099999489s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
I20260709 13:04:53.196970 13988 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:53.198329 16174 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:53.198702 16199 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:53.180198 13874 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
W20260709 13:04:53.201596 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 147312us
I20260709 13:04:53.202371 14522 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:53.181140 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.203685 13866 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:04:53.176939 16230 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:53.176594 14553 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:04:53.177466 16242 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.160758 16109 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.215754 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:53.217654 13989 connection.cc:256] RPC call timeout handler was delayed by -1.103871316999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
W20260709 13:04:53.218616 16181 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:04:53.176421 16069 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:53.222568 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 65060.6us
I20260709 13:04:53.236409 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2135) took 2943 ms. Trace:
I20260709 13:04:53.222806 16119 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:53.223559 16235 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:04:53.198758 16174 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:53.190858 16180 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:53.225584 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.214288 16130 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:04:53.227023 16067 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:53.228461 16188 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.397s	user 0.002s	sys 0.000s
I20260709 13:04:53.229420 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 299) took 1095 ms. Trace:
I20260709 13:04:53.236824 14019 rpcz_store.cc:276] 0709 13:04:50.286901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.837882 (+1550981us) service_pool.cc:224] Handling call
0709 13:04:51.839863 (+  1981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.263192 (+423329us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54002bf5a0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33aadc0 00007fb2c34351c0 00007fb2c3436231 00007fb2c3432d8f 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c
0709 13:04:53.203053 (+939861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.230228 (+ 27175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":707209600}
I20260709 13:04:53.238855 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 230) took 1188 ms. Trace:
I20260709 13:04:53.239372 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1360) took 1775 ms. Trace:
I20260709 13:04:53.299453 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.302217 14020 rpcz_store.cc:276] 0709 13:04:51.464362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.123426 (+1659064us) service_pool.cc:224] Handling call
0709 13:04:53.164518 (+ 41092us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.239030 (+ 74512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.239060 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.239109 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:53.239148 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.239210 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.239314 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.302956 13950 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:53.301852 14022 rpcz_store.cc:276] 0709 13:04:52.043758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.161454 (+1117696us) service_pool.cc:224] Handling call
0709 13:04:53.204523 (+ 43069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.231909 (+ 27386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.231951 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.231996 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:53.232038 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.232102 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.232215 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.213387 16230 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.268870 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.313498 13870 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:53.278606 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.242535 14021 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:04:53.321027 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 228) took 1277 ms. Trace:
I20260709 13:04:53.321574 14021 rpcz_store.cc:276] 0709 13:04:52.043263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.086350 (+1043087us) service_pool.cc:224] Handling call
0709 13:04:53.320932 (+234582us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.299613 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.306584 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 226) took 1263 ms. Trace:
I20260709 13:04:53.322958 14016 rpcz_store.cc:276] 0709 13:04:52.042708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.225207 (+1182499us) service_pool.cc:224] Handling call
0709 13:04:53.306522 (+ 81315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.323455 13866 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:53.307344 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.300801 13869 rpcz_store.cc:276] 0709 13:04:52.134292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.136221 (+  1929us) service_pool.cc:224] Handling call
0709 13:04:53.229192 (+1092971us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54002e7cd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:53.229321 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383112448}
I20260709 13:04:53.239156 16119 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.333277 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:53.207767 16237 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.291669 16174 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:53.355199 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48070.3us
I20260709 13:04:53.357997 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:53.360961 13868 raft_consensus.cc:2410] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:53.216588 16212 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:04:53.354688 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.332989 16230 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.265209 16241 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:53.381132 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.386317 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.386552 16212 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:53.386992 13914 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:53.387216 14023 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:04:53.389463 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.352505 16119 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:53.399510 13871 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:53.361119 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:53.399770 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 225) took 1357 ms. Trace:
I20260709 13:04:53.405018 14023 rpcz_store.cc:276] 0709 13:04:52.042408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.303798 (+1261390us) service_pool.cc:224] Handling call
0709 13:04:53.399696 (+ 95898us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21632}
W20260709 13:04:53.405898 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 202362us
I20260709 13:04:53.407886 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:53.410964 16120 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
W20260709 13:04:53.412400 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 148431us
I20260709 13:04:53.412992 16098 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:53.411557 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.414069 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:53.430001 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:53.423480 16098 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.432870 13871 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
W20260709 13:04:53.439002 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 57017.7us
I20260709 13:04:53.451643 14015 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:53.453500 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:53.454640 16242 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:53.455343 16242 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:53.456204 16242 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:53.457203 16208 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:53.413084 14552 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:53.453608 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:53.460759 16180 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:53.401130 16212 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:53.456372 13989 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:53.447046 14019 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:53.467020 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.470124 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.440598 16098 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:53.486440 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1358) took 2008 ms. Trace:
I20260709 13:04:53.486943 14015 rpcz_store.cc:276] 0709 13:04:51.463791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.263200 (+1799409us) service_pool.cc:224] Handling call
0709 13:04:53.381738 (+118538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.468326 (+ 86588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.471765 (+  3439us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.490747 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.350675 16197 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:53.496876 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2252) took 3348 ms. Trace:
I20260709 13:04:53.497437 13873 rpcz_store.cc:276] 0709 13:04:50.148428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.187379 (+ 38951us) service_pool.cc:224] Handling call
0709 13:04:50.221297 (+ 33918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.255552 (+ 34255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.255585 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:50.255628 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:50.255664 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.479748 (+3224084us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b54000da054. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:53.490255 (+ 10507us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.496673 (+  6418us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78193920}
I20260709 13:04:53.500177 16082 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.483997 16067 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:53.503826 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.505257 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.506873 16069 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:53.507932 16109 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:53.508694 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.510000 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.515092 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.505864 14016 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:53.471148 16189 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:53.525609 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 69293.1us
I20260709 13:04:53.524384 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.532990 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.535126 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.535718 16212 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.537165 16212 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.531760 13873 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:53.543062 16082 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:53.543294 16241 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:53.544080 13871 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:53.545145 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.546351 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.546732 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.547128 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.547907 13941 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:53.548795 16197 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:53.549037 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.529450 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:53.549135 16211 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.601s	user 0.011s	sys 0.003s
I20260709 13:04:53.553617 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:53.551489 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 43605.5us
I20260709 13:04:53.559566 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:53.561519 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.562237 13942 raft_consensus.cc:2410] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:53.565229 16120 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.569267 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.521325 16061 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.571485 14015 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:53.573928 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.576671 13873 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:53.576037 14016 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:53.585642 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.586913 16082 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.587055 15875 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:53.587591 16082 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:53.587929 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:53.588289 15875 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:53.591042 15985 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:53.590418 16225 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:53.601737 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1357) took 2138 ms. Trace:
I20260709 13:04:53.586997 13871 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:53.579017 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:53.603514 14019 raft_consensus.cc:2393] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:53.611833 14017 rpcz_store.cc:276] 0709 13:04:51.463481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.525455 (+2061974us) service_pool.cc:224] Handling call
0709 13:04:53.568654 (+ 43199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.579675 (+ 11021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.579729 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.601367 (+ 21638us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:53.601427 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.601491 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.601636 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.617874 13872 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:53.622690 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:53.628074 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.628851 14021 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:04:53.630398 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:53.631086 14019 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:53.629808 14016 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:53.634210 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:53.636125 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.630872 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.638453 14020 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:53.636770 14015 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:04:53.642863 14017 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.611682 16225 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:53.645216 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.644775 16225 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.646513 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:53.647850 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.648011 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.648746 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:53.649034 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.649693 13869 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:04:53.650754 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.649703 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.653105 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.650451 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.642045 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.655160 13872 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:53.669090 13874 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:53.666471 16090 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:53.673013 14021 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
W20260709 13:04:53.635882 16233 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.653s	user 0.010s	sys 0.005s
I20260709 13:04:53.639272 16237 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:53.677376 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.639358 16082 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:53.678308 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:53.684072 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:53.684278 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:53.580595 16120 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.672402 16090 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:53.651512 14551 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:53.669826 16209 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:53.690611 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:53.696213 14023 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:53.695294 14015 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:53.699374 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.701771 13873 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.702561 13868 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 2: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:53.706543 13870 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:53.693919 16090 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:53.730326 14020 raft_consensus.cc:2393] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:53.732187 14020 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:53.735150 16061 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:53.737079 16262 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:04:53.737354 16120 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:53.747031 16098 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:53.751333 16264 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:53.752755 16225 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:53.682442 16233 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:53.764675 14019 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:53.704800 16076 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:53.775105 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.778002 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.779732 14019 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:53.780352 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.781339 13869 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:53.781380 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.779562 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.782434 13874 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:04:53.783582 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:04:53.783896 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.784427 13875 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:53.780232 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.791201 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.792868 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:53.793282 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.794144 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.794394 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:53.794476 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.794936 13869 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:53.733579 14020 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "765348f2471a493d92e998132c0c746b"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49644
I20260709 13:04:53.805995 13869 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:53.809058 13866 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:53.813477 14020 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:53.814235 14020 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:53.819473 16230 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.834723 16230 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:53.829595 13875 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:53.835575 16119 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.837783 16119 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:53.839061 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 253670us
W20260709 13:04:53.844408 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 336802us
I20260709 13:04:53.806908 16098 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:53.845726 14023 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:04:53.846653 13873 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:04:53.853004 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 231) took 1809 ms. Trace:
I20260709 13:04:53.853978 14024 rpcz_store.cc:276] 0709 13:04:52.044038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.157721 (+1113683us) service_pool.cc:224] Handling call
0709 13:04:53.192246 (+ 34525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.192695 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.192729 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:53.193323 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 13:04:53.447581 (+254258us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:53.483705 (+ 36124us) log.cc:844] Serialized 14 byte log entry
0709 13:04:53.484014 (+   309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.484096 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.852681 (+368585us) raft_consensus.cc:1685] finished
0709 13:04:53.852718 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.852926 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35854,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":58,"raft.queue_time_us":1089,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":7071,"spinlock_wait_cycles":31616,"thread_start_us":253778,"threads_started":1,"wal-append.queue_time_us":415812}
I20260709 13:04:53.866504 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.868990 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:53.869755 13945 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:53.870965 13946 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:53.874537 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.887265 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.888163 13943 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:53.903332 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.906447 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.850135 16174 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:53.912348 13945 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:04:53.913034 13867 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:53.915966 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.916814 13875 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:53.851537 16180 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:04:53.915167 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.922449 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:53.930416 14017 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:04:53.878999 16208 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:04:53.954414 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.954262 13989 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:53.956169 16067 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.957039 16067 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:53.966176 13989 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:53.968547 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:53.969733 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:53.939764 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:53.971482 13873 raft_consensus.cc:2393] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:53.988418 14024 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:04:53.999248 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.000473 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:54.001096 14023 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:54.003289 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 381474us
I20260709 13:04:54.003449 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:53.988413 14020 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:53.963245 16242 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:04:54.000280 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 76434.7us
I20260709 13:04:54.010223 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:54.011339 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.979029 16082 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:54.012284 16082 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:54.013551 13945 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:54.013414 16082 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:54.014424 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.015100 13941 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:54.015398 13949 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:54.016029 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.016793 13950 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:54.016832 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.019330 16067 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:54.020561 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 488719us
I20260709 13:04:54.025985 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:54.026818 14024 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:54.033479 16130 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:54.033981 16130 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:54.036351 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1359) took 2572 ms. Trace:
I20260709 13:04:54.038460 16172 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:54.038904 16172 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:54.043354 14024 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:54.036755 14018 rpcz_store.cc:276] 0709 13:04:51.464074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.100468 (+1636394us) service_pool.cc:224] Handling call
0709 13:04:53.130974 (+ 30506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.360815 (+229841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.360846 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:53.416557 (+ 55711us) log.cc:844] Serialized 26 byte log entry
0709 13:04:53.689852 (+273295us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:53.709473 (+ 19621us) log.cc:844] Serialized 14 byte log entry
0709 13:04:53.724031 (+ 14558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.035923 (+311892us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b5400411c54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:54.035970 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.036017 (+    47us) raft_consensus.cc:1685] finished
0709 13:04:54.036120 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.036288 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44821,"prepare.queue_time_us":18944,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":9050,"raft.queue_time_us":12183,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":50208,"spinlock_wait_cycles":466366720,"thread_start_us":172332,"threads_started":1,"wal-append.queue_time_us":397720}
I20260709 13:04:53.986941 16233 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:54.004370 16197 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:54.045301 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.053699 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.010442 16242 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:54.071132 14016 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:04:54.071295 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.071921 14018 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:54.072153 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.047314 16067 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:54.076030 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.076676 13875 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:04:54.081425 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:54.082296 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.088636 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.075417 14024 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 1.
I20260709 13:04:54.069700 14023 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:04:54.091974 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:54.090384 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.096630 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.064633 16197 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:54.100921 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.098462 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.074666 16242 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:54.101763 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.075927 16067 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:54.135635 16067 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:54.043992 16172 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:54.138583 16172 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:54.110740 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:54.139632 14551 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:54.112612 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:54.118645 16199 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:54.118040 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.142946 14024 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:54.143214 16230 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:54.143826 16230 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:54.144194 14024 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:54.122848 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.127470 16224 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:54.143221 16199 raft_consensus.cc:3055] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:54.144841 16230 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:54.145407 16224 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.145506 16199 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:54.145924 16224 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.146443 16174 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:54.146510 16199 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:54.148032 16199 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:54.134315 13946 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:54.148319 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:54.134506 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.148958 13871 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:54.135267 13950 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 3: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:54.150229 16267 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.150825 16267 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:54.151351 14023 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:54.152089 16190 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:54.152482 16190 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:54.153251 16267 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:54.099640 13942 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:54.155761 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.156515 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.138357 15908 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:04:54.110709 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:54.161247 13870 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:04:54.161456 13875 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.162423 16265 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:54.141903 14020 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:54.180085 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:54.153451 16190 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:54.187987 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.187253 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.188943 13867 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:54.189195 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.190909 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.200716 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.202342 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.203162 13873 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:54.204897 14023 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:54.207724 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.217149 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.222633 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.224546 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.226657 13989 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:54.228058 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:54.242257 14024 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:54.243533 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.244127 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:54.244635 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:54.251570 16212 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.714s	user 0.013s	sys 0.001s
I20260709 13:04:54.254459 16076 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:54.255180 16076 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:54.255698 16076 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.258328 16061 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:54.258985 16061 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:54.259491 16061 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:54.264237 13838 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:54.266511 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.267241 13873 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:54.267779 16265 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:54.268167 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.268369 16265 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:54.268920 13873 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:54.268975 16265 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:54.269903 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:54.274807 13873 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 2: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:54.245746 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.275871 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.277100 14024 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:54.194205 16208 raft_consensus.cc:3055] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:54.277249 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:54.280900 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:54.290364 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:54.300150 14015 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.600s	user 0.013s	sys 0.000s
I20260709 13:04:54.300927 14015 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:54.308938 13875 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:04:54.314587 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:54.337196 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 56362.1us
I20260709 13:04:54.219565 16190 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:54.352012 14015 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:54.366849 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 224656us
I20260709 13:04:54.371860 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:54.283599 16208 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:54.385538 14020 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:04:54.395578 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 118440us
W20260709 13:04:54.432749 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55801.6us
I20260709 13:04:54.368602 16209 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:54.461858 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.467006 13949 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:54.463943 14015 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:54.464535 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 354379us
I20260709 13:04:54.375665 16191 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:54.473327 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 101601us
W20260709 13:04:54.456924 16090 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.682s	user 0.007s	sys 0.006s
I20260709 13:04:54.493518 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.500099 14023 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:54.520360 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.552043 14020 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:54.560703 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:54.564165 16278 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:54.564864 16278 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:54.566242 15943 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:54.566779 15943 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:54.566457 16208 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:54.567215 15943 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.568240 16208 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:54.569955 16288 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:54.466598 16209 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:54.601713 16209 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:54.502000 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.606600 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.607055 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.502595 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.521931 16090 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:54.354460 16197 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:54.608074 14015 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 2: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:54.611040 13945 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:54.637349 13871 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:54.478402 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:54.657282 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 179012us
I20260709 13:04:54.577592 16242 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:54.644878 16278 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:54.693037 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 202454us
I20260709 13:04:54.688040 13945 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:54.693897 13838 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:54.660573 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:54.705457 16067 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 13:04:54.705572 16267 raft_consensus.cc:2764] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:04:54.707214 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.708137 13950 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 3: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:54.709175 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:54.710462 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:54.710813 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.710482 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.712541 13941 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
W20260709 13:04:54.663868 16082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.048s	sys 0.092s
W20260709 13:04:54.713471 16082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.048s	sys 0.092s
I20260709 13:04:54.714274 16067 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:54.715687 14016 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:54.717549 16067 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:54.723840 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.538810 16224 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:54.679288 16233 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:54.729815 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:54.732093 16204 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:54.732530 16204 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:54.732692 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:54.733039 16204 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:54.733583 13944 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 4: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:54.734324 13943 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:54.734934 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 220347us
I20260709 13:04:54.737074 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:54.738523 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:54.738914 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:54.739244 14558 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:54.739511 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:54.740360 13947 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.741377 16090 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:54.742185 16174 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:54.742692 16174 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:54.743649 16174 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:54.745863 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2144) took 1341 ms. Trace:
I20260709 13:04:54.738565 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:54.746342 14021 rpcz_store.cc:276] 0709 13:04:53.404455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.673763 (+269308us) service_pool.cc:224] Handling call
0709 13:04:53.686152 (+ 12389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.701671 (+ 15519us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:54.051497 (+349826us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.429666 (+378169us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:54.467632 (+ 37966us) log.cc:844] Serialized 26 byte log entry
0709 13:04:54.467807 (+   175us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:54.468024 (+   217us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.487394 (+ 19370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.487519 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.745607 (+258088us) raft_consensus.cc:1685] finished
0709 13:04:54.745628 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.745788 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36381,"prepare.queue_time_us":40390,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":30381,"raft.run_cpu_time_us":776,"raft.run_wall_time_us":35976,"thread_start_us":625099,"threads_started":2,"wal-append.queue_time_us":641960}
I20260709 13:04:54.739578 14561 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:54.748828 14558 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:54.749086 14021 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 13:04:54.753203 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:54.729899 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 69451.2us
I20260709 13:04:54.761574 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:54.762331 13945 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:54.764390 16225 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:54.765618 16281 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:04:54.730772 16233 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:54.767794 16224 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:54.767747 16061 consensus_meta.cc:287] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.000s
W20260709 13:04:54.768219 16172 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.003s
I20260709 13:04:54.768952 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:54.770498 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:54.770767 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:54.785590 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 88280us
I20260709 13:04:54.789202 16180 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:54.791311 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:54.776010 16211 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:04:54.794757 16211 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:54.794502 16130 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:54.795769 16130 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:54.798468 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:54.795141 16211 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:54.810678 16211 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:54.811977 16211 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:54.812940 16265 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.003s
I20260709 13:04:54.783981 16233 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:54.816428 14024 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:54.823282 16242 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:54.753873 16267 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:04:54.845407 13947 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
W20260709 13:04:54.809845 16275 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.235s	user 0.008s	sys 0.000s
I20260709 13:04:54.851325 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.851918 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.856396 13948 raft_consensus.cc:2410] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:04:54.861536 13945 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:54.855420 13949 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:54.857163 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:04:55.098763 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.151602 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.864530 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:04:54.873011 13839 leader_election.cc:336] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.926s (SENT)
I20260709 13:04:55.154381 13839 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:55.152480 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:54.988834 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:04:54.892109 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.156867 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:55.157272 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.158422 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.160104 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:54.864159 14522 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:54.990800 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:55.164497 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 300425us
W20260709 13:04:55.082792 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 201902us
W20260709 13:04:55.023623 16190 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.654s	user 0.012s	sys 0.000s
I20260709 13:04:55.175452 13950 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:55.175370 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:55.121220 16208 consensus_meta.cc:287] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.005s
W20260709 13:04:55.047006 16076 consensus_meta.cc:287] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.791s	user 0.010s	sys 0.002s
I20260709 13:04:55.187053 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:55.187644 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1367) took 1216 ms. Trace:
I20260709 13:04:55.186896 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:55.188038 14016 rpcz_store.cc:276] 0709 13:04:53.960583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.071914 (+111331us) service_pool.cc:224] Handling call
0709 13:04:55.177110 (+1105196us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b54003255d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:55.177265 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1477924224}
I20260709 13:04:55.189215 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:55.190917 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:55.192342 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:55.193193 14561 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:54.994340 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:54.886189 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:54.788301 16061 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.196825 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:55.168916 16281 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:55.194588 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:55.151238 16180 consensus_queue.cc:1243] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:04:55.200269 14561 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:04:55.201591 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:54.847373 16265 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.180272 16224 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:55.205580 13988 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:55.181974 16288 raft_consensus.cc:2764] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:04:55.208964 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:55.186268 16130 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:55.211620 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:55.220286 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:55.222338 14561 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:55.014566 16204 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.066118 15943 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.224283 16204 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:55.205138 16174 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:55.225450 15943 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:55.226279 14556 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:55.227492 13873 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:55.216949 13941 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:55.228240 13941 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:55.229120 13941 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.225466 13950 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:55.125841 16233 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:55.230511 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 376959us
I20260709 13:04:55.230654 13941 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.048192 16209 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.235257 16209 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:55.235440 13988 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:55.236050 16209 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:55.236708 13988 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:55.236829 16209 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:55.237709 16283 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:55.238303 16283 raft_consensus.cc:697] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:55.238765 16209 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:55.239696 16209 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:55.239346 16283 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.241353 13941 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:55.241940 13866 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:55.244277 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:04:55.254237 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.206773 16082 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:55.247877 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:55.249352 13875 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:04:55.212621 16281 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:55.244019 14555 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:04:55.194453 16076 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.246044 13840 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:55.258569 16279 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.684s	user 0.003s	sys 0.000s
I20260709 13:04:55.258677 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2271) took 1614 ms. Trace:
I20260709 13:04:55.250394 16224 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:55.256461 16288 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:55.248574 16233 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:55.297806 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 51809us
W20260709 13:04:55.287297 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.254693 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 381704us
I20260709 13:04:55.265375 16281 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:55.291296 13870 rpcz_store.cc:276] 0709 13:04:53.644321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.648465 (+  4144us) service_pool.cc:224] Handling call
0709 13:04:55.258503 (+1610038us) spinlock_profiling.cc:243] Waited 57.1 ms on lock 0x7b54000a6754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:55.258618 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131362560}
W20260709 13:04:55.293648 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 49433.8us
I20260709 13:04:55.284180 16082 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:55.296103 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 310) took 1206 ms. Trace:
W20260709 13:04:55.301848 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 57663.7us
I20260709 13:04:55.306119 13869 rpcz_store.cc:276] 0709 13:04:54.089737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.089931 (+   194us) service_pool.cc:224] Handling call
0709 13:04:54.090406 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:54.090652 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.090677 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:54.109936 (+ 19259us) log.cc:844] Serialized 26 byte log entry
0709 13:04:54.460369 (+350433us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:54.460421 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.460517 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:55.257769 (+797252us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:04:55.295577 (+ 37808us) raft_consensus.cc:1685] finished
0709 13:04:55.295611 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.295990 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59789696,"thread_start_us":349965,"threads_started":1,"wal-append.queue_time_us":360438}
I20260709 13:04:55.309278 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:55.296818 16288 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:55.317258 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:55.317873 16287 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.747s	user 0.003s	sys 0.000s
I20260709 13:04:55.315171 16130 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:55.322990 16224 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.326180 16298 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:55.305557 16303 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:55.328178 15763 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:55.332963 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:55.316334 16288 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:55.359411 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:55.354331 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.362423 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 53245.2us
I20260709 13:04:55.346926 16237 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:55.355167 16211 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:55.336010 16283 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:55.375274 13840 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:55.377686 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:55.377826 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:55.379598 16061 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:55.376253 16212 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:55.380728 15763 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:55.376494 16082 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.379904 16298 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:55.417665 16191 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:55.418251 16191 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:55.405318 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:55.405623 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:55.423353 16208 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:55.424638 16278 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:55.368427 16237 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:55.425110 16278 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:55.425601 16278 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:55.425760 16237 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:55.406163 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 249681us
I20260709 13:04:55.407001 16204 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.408473 16242 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:55.408847 16212 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:04:55.409168 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.410552 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1375) took 1363 ms. Trace:
I20260709 13:04:55.436154 14018 rpcz_store.cc:276] 0709 13:04:54.046936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.088797 (+ 41861us) service_pool.cc:224] Handling call
0709 13:04:55.410322 (+1321525us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54002986d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:55.410483 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2605782912}
W20260709 13:04:55.345151 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 02dcf286c6bd48e3a6e548def7d786cc with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 4dd265d61fc141a48ca9d3d5c8f52921 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:55.357280 16130 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.362406 16303 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.443984 13945 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:55.448791 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:55.450213 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:55.450703 13941 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:55.462635 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:04:55.450161 16301 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.234s	user 0.004s	sys 0.000s
I20260709 13:04:55.447275 16283 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:55.465627 13839 leader_election.cc:336] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.649s (SENT)
I20260709 13:04:55.466320 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:04:55.467904 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.492297 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:55.447705 13758 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 102815us
I20260709 13:04:55.496456 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:55.497113 13870 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:55.497557 16130 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:55.461422 16098 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.018s	sys 0.029s
I20260709 13:04:55.499500 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:55.500268 13867 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:55.501236 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:55.501529 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:55.502707 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:55.503278 13874 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:55.503718 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:55.504876 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:55.505424 13871 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:55.441701 16233 consensus_queue.cc:1059] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:04:55.454428 16181 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:55.465168 16237 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.440848 16204 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.445988 16208 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
W20260709 13:04:55.510018 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 86910.8us
I20260709 13:04:55.449572 16265 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:55.499564 16098 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.019s	sys 0.029s
I20260709 13:04:55.511310 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:55.494967 13872 raft_consensus.cc:2376] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5. Re-sending same reply.
I20260709 13:04:55.514895 13868 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:55.516424 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:55.464582 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:55.519315 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:55.529671 13873 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:04:55.523852 13945 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:55.491667 16296 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:55.524686 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 89093.3us
I20260709 13:04:55.526940 13870 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
W20260709 13:04:55.527061 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 61469.6us
I20260709 13:04:55.537814 13839 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:55.542253 15985 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:55.544844 16296 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.522079 16197 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:55.522481 16212 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.550925 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:55.576962 13866 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 1.
I20260709 13:04:55.554664 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2067) took 1353 ms. Trace:
I20260709 13:04:55.559063 13874 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:55.578073 13875 rpcz_store.cc:276] 0709 13:04:54.201587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.371755 (+170168us) service_pool.cc:224] Handling call
0709 13:04:54.501594 (+129839us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:55.255889 (+754295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.554390 (+298501us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b540029e354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:55.554601 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1718963072}
W20260709 13:04:55.561672 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 97164.3us
W20260709 13:04:55.564483 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.524780 16082 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:55.567628 16288 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.469897 16190 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:04:55.573083 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:55.585600 13867 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:55.536240 16061 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:55.581465 13874 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:55.581419 13987 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:55.582492 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:55.587085 13872 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:55.584137 13950 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:55.561842 16180 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.589577 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:55.595067 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:55.599223 13871 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:55.576550 16197 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:55.597020 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:55.602557 13947 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:55.607692 16288 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:55.608381 16288 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:55.587285 16208 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:55.612843 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 75077.5us
I20260709 13:04:55.632078 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:55.633468 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:55.618801 13868 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:04:55.593981 16237 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:55.641886 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:55.639979 16237 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:55.644829 16283 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:55.645975 16237 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.647351 16237 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:55.649194 16209 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.611840 16197 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:55.585186 16082 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.656220 16082 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:55.597995 16180 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.658685 16180 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:55.637614 16174 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:55.658780 16082 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.659471 16174 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.659926 16174 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:55.659696 16082 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.662300 16296 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:55.663434 16191 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.663977 16082 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:55.664573 13875 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:55.665658 16191 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:55.575433 16308 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 13:04:55.583328 16281 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.675921 16082 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:55.679984 16076 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:55.682437 16209 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:55.682730 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.683298 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.687081 16174 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.688725 16130 raft_consensus.cc:3055] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:55.689306 16130 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:55.659001 16204 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:55.690130 16174 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:55.673708 16265 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:55.691774 13989 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:55.692502 16281 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:55.693074 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:55.693696 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:55.694247 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:55.695098 16283 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:55.695698 16283 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:55.699896 16076 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:55.700350 16076 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:55.700671 16172 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:55.700914 16082 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:55.701270 16172 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:55.696768 16283 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.702432 16172 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.703755 16172 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:55.701447 16082 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:55.739601 16082 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:55.717263 16278 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.742918 13943 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:55.743983 16278 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:55.744696 16278 raft_consensus.cc:3055] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:55.745093 16278 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:04:55.745771 13773 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57642 (request call id 271) took 12760 ms (client timeout 15000 ms). Trace:
W20260709 13:04:55.710994 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.713935 16130 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:55.719341 16303 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:55.725579 16212 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:55.700721 16076 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:55.730396 16281 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:55.678691 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:55.782253 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 103639us
I20260709 13:04:55.753266 13772 catalog_manager.cc:5697] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 4, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 4 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:55.784103 13987 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:55.785817 13987 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:55.738787 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 44597.4us
I20260709 13:04:55.786976 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:55.757687 13869 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:55.669817 16197 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.762624 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:55.764312 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:04:55.777391 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 202639us
W20260709 13:04:55.778613 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 184313us
W20260709 13:04:55.793241 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 220142us
I20260709 13:04:55.795003 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:55.792960 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 151954us
W20260709 13:04:55.803761 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.805349 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:55.803294 13873 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
W20260709 13:04:55.817025 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 183639us
I20260709 13:04:55.819275 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:55.820688 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:55.802616 16174 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:55.847790 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:04:55.746248 13773 rpcz_store.cc:269] 0709 13:04:42.985485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.985712 (+   227us) service_pool.cc:224] Handling call
0709 13:04:55.745540 (+12759828us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:48.373568 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:48.516272 (+142704us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:48.523241 (+  6969us) write_op.cc:435] Acquired schema lock
0709 13:04:48.523281 (+    40us) tablet.cc:662] Decoding operations
0709 13:04:48.619131 (+ 95850us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:48.619199 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:48.619240 (+    41us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:04:48.620040 (+   800us) tablet.cc:701] Row locks acquired
0709 13:04:48.620059 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:04:48.620232 (+   173us) write_op.cc:270] Start()
0709 13:04:48.620371 (+   139us) write_op.cc:276] Timestamp: P: 1783602288620178 usec, L: 0
0709 13:04:48.620390 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:04:48.621159 (+   769us) log.cc:844] Serialized 3625 byte log entry
0709 13:04:49.528748 (+907589us) op_driver.cc:464] REPLICATION: finished
0709 13:04:49.858207 (+329459us) write_op.cc:301] APPLY: starting
0709 13:04:49.858315 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:49.858530 (+   215us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:49.858662 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:49.900576 (+ 41914us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:49.900826 (+   250us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:04:49.900873 (+    47us) write_op.cc:312] APPLY: finished
0709 13:04:49.901216 (+   343us) log.cc:844] Serialized 65 byte log entry
0709 13:04:49.901392 (+   176us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:49.901538 (+   146us) write_op.cc:454] Released schema lock
0709 13:04:49.901596 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:49.902038 (+   442us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:49.902086 (+    48us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":23566,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":271840,"apply.run_cpu_time_us":3772,"apply.run_wall_time_us":43810,"num_ops":8,"prepare.queue_time_us":186890,"prepare.run_cpu_time_us":13529,"prepare.run_wall_time_us":278137,"raft.queue_time_us":436834,"raft.run_cpu_time_us":29021,"raft.run_wall_time_us":419175,"replication_time_us":846537,"thread_start_us":460482,"threads_started":4,"wal-append.queue_time_us":30101,"wal-append.run_cpu_time_us":37482,"wal-append.run_wall_time_us":2252330}]]}
I20260709 13:04:55.819756 13772 catalog_manager.cc:5697] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 3, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:55.853791 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:55.805558 16209 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:55.856649 13942 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:55.849323 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:55.822767 16130 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:55.918591 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:55.919518 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:55.758136 16278 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:55.847327 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:04:55.997304 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.930650 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:56.096205 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:04:56.044912 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 224277us
W20260709 13:04:56.154858 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 335512us
I20260709 13:04:56.156153 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:56.156368 13943 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:56.155452 14555 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:55.848940 16265 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.168s
I20260709 13:04:55.824961 16242 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:04:56.120260 13839 leader_election.cc:336] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.071s (SENT)
I20260709 13:04:56.164012 13839 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:04:56.124689 13840 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.672s (SENT)
I20260709 13:04:56.165228 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:56.125624 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:56.167104 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:56.131135 13772 catalog_manager.cc:5697] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:55.827909 16061 consensus_queue.cc:1059] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.153s
I20260709 13:04:56.127434 16283 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:56.194480 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.127000 16174 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:56.185534 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:56.213577 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 88932.2us
W20260709 13:04:56.217245 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.212319 13772 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:56.248134 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 81074.4us
I20260709 13:04:56.209079 16278 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:56.210276 16098 raft_consensus.cc:2764] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:04:56.158959 16308 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:04:56.221439 16283 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:04:56.267344 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 147478us
W20260709 13:04:56.263302 16082 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.523s	user 0.015s	sys 0.006s
I20260709 13:04:56.258847 13772 catalog_manager.cc:5697] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 reported cstate change: term changed from 3 to 4, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 4 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:56.272370 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 486574us
W20260709 13:04:56.268035 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 111995us
I20260709 13:04:56.222424 16242 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:56.128391 16180 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.278864 13772 catalog_manager.cc:5697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:56.276140 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:56.256482 16314 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:56.257174 16315 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:56.234987 16209 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:04:56.306579 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:56.266922 16283 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:56.287765 13943 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:56.234985 16328 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:56.294960 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.267762 16061 raft_consensus.cc:3055] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:56.299314 16174 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:56.300194 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:56.335573 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:56.300884 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 44862.6us
I20260709 13:04:56.276320 16181 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:56.272837 16322 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:56.277715 16197 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:56.277983 16082 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:56.327862 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:56.328986 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:56.326864 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:56.333587 13772 catalog_manager.cc:5697] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 3, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:56.286198 16225 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:04:56.423272 13840 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.875s (SENT)
W20260709 13:04:56.546890 16172 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.843s	user 0.012s	sys 0.004s
I20260709 13:04:56.338922 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:56.347429 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:56.423215 13869 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:56.410015 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:56.362413 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:56.507601 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:56.401278 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.591934 14560 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:56.582937 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:56.592942 14559 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:56.531086 13839 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:56.446906 13870 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:04:56.593773 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.595574 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:56.335045 16061 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:56.338709 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:56.424057 16237 consensus_meta.cc:287] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.776s	user 0.019s	sys 0.004s
W20260709 13:04:56.602365 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 71350.4us
I20260709 13:04:56.603660 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:56.586508 14556 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
W20260709 13:04:56.605691 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 267175us
I20260709 13:04:56.591972 13867 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:56.387575 16313 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 13:04:56.462208 16331 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:56.615195 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.618078 14555 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:56.578299 16211 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.000s	sys 0.010s
I20260709 13:04:56.350234 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:56.499423 16209 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:56.387034 16323 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:04:56.482247 16329 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.482749 16321 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:56.303529 16180 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:56.370421 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:56.349434 16317 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:04:56.625226 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 201981us
I20260709 13:04:56.344143 16174 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:56.317052 16315 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:56.607008 16265 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:56.650477 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 142985us
I20260709 13:04:56.339577 16328 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:56.681286 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 341386us
I20260709 13:04:56.671525 13867 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
W20260709 13:04:56.622088 16211 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.000s	sys 0.011s
I20260709 13:04:56.673753 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:56.611722 16098 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:56.682403 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:56.632436 16172 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:56.624922 16314 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:56.496991 16242 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:56.662292 16209 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:56.691412 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:56.673372 16237 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:04:56.704730 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 98158.8us
I20260709 13:04:56.673434 16321 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:56.676430 13772 catalog_manager.cc:5697] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 reported cstate change: config changed from index 3 to 4, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER e1649107c781428692458f10597c5742 (127.12.109.65), VOTER 3779649036364744823fc035dac892c7 (127.12.109.66), VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67), VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 5 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:04:56.702888 16130 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.545s	user 0.013s	sys 0.000s
I20260709 13:04:56.747834 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1436) took 1245 ms. Trace:
W20260709 13:04:56.751986 13989 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.027s	sys 0.049s
W20260709 13:04:56.759117 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.726836 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:56.720206 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:56.782397 13989 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.034s	sys 0.057s
W20260709 13:04:56.782928 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 597447us
W20260709 13:04:56.784735 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.784116 13989 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:04:56.789700 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 69555.4us
I20260709 13:04:56.752432 13941 rpcz_store.cc:276] 0709 13:04:55.502375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.502751 (+   376us) service_pool.cc:224] Handling call
0709 13:04:55.503075 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:55.503477 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:55.503504 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:55.504011 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:04:55.610752 (+106741us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:55.610789 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.610861 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.747583 (+1136722us) raft_consensus.cc:1685] finished
0709 13:04:56.747606 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.747762 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38813,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":142,"thread_start_us":106303,"threads_started":1,"wal-append.queue_time_us":314532}
I20260709 13:04:56.710947 16098 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:56.715864 16209 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:56.704233 16331 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
W20260709 13:04:56.797770 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 105143us
I20260709 13:04:56.723838 16181 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:56.790933 13772 catalog_manager.cc:5697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 reported cstate change: config changed from index 2 to 3, 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER e1649107c781428692458f10597c5742 (127.12.109.65), VOTER 3779649036364744823fc035dac892c7 (127.12.109.66), VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67), VOTER 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 3 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:04:56.786373 16211 raft_consensus.cc:3055] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:56.819353 13772 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 3 to 4, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 4 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:56.677415 16329 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:56.796650 16288 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:56.836524 16288 raft_consensus.cc:697] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:56.837412 16180 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:56.840255 16329 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:56.829156 14553 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:04:56.844456 16328 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:56.845379 16328 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:56.845503 16212 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.068s	user 0.006s	sys 0.008s
I20260709 13:04:56.846478 16328 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:56.846994 16328 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:56.847528 16328 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:56.850172 16321 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:56.837810 16288 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:56.852130 16321 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:56.853446 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.856535 16315 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:56.866930 16283 consensus_meta.cc:287] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.003s
I20260709 13:04:56.867679 16283 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:56.869903 16181 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:56.821911 16211 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:56.794605 16325 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.919343 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:56.920203 13941 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:04:56.808303 16332 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:56.921748 14522 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:56.922664 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:56.823156 13772 catalog_manager.cc:5697] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 2, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:56.794420 16337 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.331s
W20260709 13:04:56.929621 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 204035us
W20260709 13:04:56.929723 13915 connection.cc:256] RPC call timeout handler was delayed by -3.6573736340000096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 13:04:56.930544 13915 connection.cc:256] RPC call timeout handler was delayed by -3.5367736339999283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
I20260709 13:04:56.930697 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:04:56.881992 16320 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.730s	user 0.001s	sys 0.005s
W20260709 13:04:56.936210 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 152155us
I20260709 13:04:56.809144 16327 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:56.938997 14560 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:56.941170 13988 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:04:56.882923 16082 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:56.942869 13989 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:56.920415 16346 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.926403 16325 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:56.948172 16346 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:56.944921 16332 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:56.992261 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:57.000339 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.024428 13950 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:56.891490 16211 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:57.031500 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 88808.2us
I20260709 13:04:56.962138 16329 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:57.033093 13989 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:56.979550 16174 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:57.034896 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:56.982201 16267 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.017784 16350 raft_consensus.cc:2764] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:04:57.028930 16336 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.049919 16211 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:57.055317 13948 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:57.032157 16082 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:57.056460 16172 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.058212 16340 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:57.058676 16310 raft_consensus.cc:3055] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:57.052289 16335 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.473s
I20260709 13:04:57.036578 16212 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.072790 16316 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:04:57.078451 16344 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
I20260709 13:04:57.050490 16351 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:57.052121 16181 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:57.049932 16296 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.038837 16130 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.079403 16061 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.151571 16061 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:57.080641 14552 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:04:57.155134 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 580) took 1349 ms. Trace:
I20260709 13:04:57.105054 16332 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.082814 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:04:57.156903 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 74185.5us
I20260709 13:04:57.036471 16327 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:04:57.083493 16224 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:57.159238 16338 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:57.159790 16338 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:57.160800 16338 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.039024 16343 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:57.050706 16331 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:57.094667 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 61653.7us
I20260709 13:04:57.162848 16331 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.163302 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:57.056294 13838 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:57.165843 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 109658us
W20260709 13:04:57.166287 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 197294us
I20260709 13:04:57.096544 16346 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:56.959784 16098 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.099220 16336 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.034559 16329 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.122910 16314 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:57.036715 16180 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:57.104208 16278 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.845s	user 0.007s	sys 0.004s
I20260709 13:04:57.183320 16174 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.183404 16329 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:57.112517 16310 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:57.169272 16130 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.183929 16174 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:57.139482 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.116858 13839 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:57.104233 16225 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:04:57.148456 14551 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:57.155575 14552 rpcz_store.cc:276] 0709 13:04:55.805669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.818961 (+ 13292us) service_pool.cc:224] Handling call
0709 13:04:57.154875 (+1335914us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b5400730354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:57.155023 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1069814656}
I20260709 13:04:57.098649 16283 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:04:57.167853 13840 leader_election.cc:336] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.736s (SENT)
I20260709 13:04:57.169221 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:04:57.169721 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 228641us
I20260709 13:04:57.167239 16351 raft_consensus.cc:697] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:57.194063 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:57.195173 13840 leader_election.cc:336] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.751s (SENT)
I20260709 13:04:57.195629 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:57.193651 16130 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:57.196146 16174 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:57.119285 16316 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:57.196689 16265 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:57.197023 16331 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:57.166572 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:57.197413 16316 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.198383 13838 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:57.174293 16082 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:57.199060 16336 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:57.199441 13914 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:57.199545 16082 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.200163 16325 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:57.197445 16265 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:57.201148 16265 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:57.109488 14555 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:57.171461 16322 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:57.182240 16180 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.176651 16339 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.202271 16322 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.202765 16322 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:57.202550 16339 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.203830 16180 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:57.183583 16350 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.204838 16339 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:57.204576 16350 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.181730 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.205279 16327 consensus_queue.cc:1059] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:57.205648 13948 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:04:57.112711 16172 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:57.189713 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 72920us
I20260709 13:04:57.207044 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:04:57.207994 16341 log.cc:927] Time spent T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.412s	user 0.002s	sys 0.000s
I20260709 13:04:57.166917 16296 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.195660 13869 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:04:57.196202 16342 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:57.118867 16298 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.210047 16342 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:57.210479 16342 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:57.210976 16342 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:57.140410 16209 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.212406 16317 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:57.212582 16326 consensus_queue.cc:1059] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:57.213032 16317 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:57.213894 16209 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:57.214665 13869 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:57.215003 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:57.216768 16350 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:57.206300 16327 raft_consensus.cc:3055] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:57.217846 16327 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:57.217911 16331 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:57.218331 16172 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:57.175262 16343 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:57.219426 16343 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.219089 16327 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.219843 16343 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:57.220480 16327 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:57.221280 16349 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:57.221695 16349 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:57.198513 16316 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:57.222683 16349 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.224342 16349 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:57.201903 16265 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:57.190263 16314 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.226527 16265 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.227111 16265 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:57.228591 16314 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:57.228479 16310 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.122025 16212 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.093878 16267 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:57.232695 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.233217 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.144974 16348 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:04:57.235847 14522 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:57.237319 16337 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:57.237694 16337 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.238013 16337 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:57.210263 16298 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.222793 16351 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.213918 16326 raft_consensus.cc:3055] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:57.246263 16326 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:57.247259 16326 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.248661 16326 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:57.246208 16267 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:57.252828 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 45871.1us
I20260709 13:04:57.254613 13867 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:57.255674 13868 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:57.258134 13871 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:57.260259 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:57.262382 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:57.262744 13872 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:04:57.262952 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:57.263291 13875 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:04:57.264052 16076 consensus_meta.cc:287] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.461s	user 0.011s	sys 0.004s
I20260709 13:04:57.264204 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:57.264503 16310 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:57.264943 13875 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:57.265175 16212 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:57.262744 13871 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:57.263939 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:57.267341 13875 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:57.273046 13872 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:57.108275 16242 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:57.279940 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 349332us
I20260709 13:04:57.282357 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:57.291422 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:04:57.300402 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.302040 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.305248 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 354393us
W20260709 13:04:57.306545 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.202273 16082 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:57.312505 13873 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:57.322964 13875 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:57.316339 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 121172us
W20260709 13:04:57.329005 13840 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.821s (SENT)
I20260709 13:04:57.317521 13871 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:57.329557 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:57.331163 13915 connection.cc:256] RPC call timeout handler was delayed by -3.4071736339999461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2135s for a response
W20260709 13:04:57.331935 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.332278 13915 connection.cc:256] RPC call timeout handler was delayed by -2.3991736340000216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3255s for a response
W20260709 13:04:57.332562 13915 connection.cc:256] RPC call timeout handler was delayed by -2.3038382390000152s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
W20260709 13:04:57.332842 13915 connection.cc:256] RPC call timeout handler was delayed by -2.29843823900012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
W20260709 13:04:57.333140 13915 connection.cc:256] RPC call timeout handler was delayed by -2.2696382390000736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
W20260709 13:04:57.333423 13915 connection.cc:256] RPC call timeout handler was delayed by -2.1877382390000548s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
W20260709 13:04:57.333688 13915 connection.cc:256] RPC call timeout handler was delayed by -2.1535382390001132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1685s for a response
W20260709 13:04:57.333954 13915 connection.cc:256] RPC call timeout handler was delayed by -2.1508382390001088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 13:04:57.334307 13915 connection.cc:256] RPC call timeout handler was delayed by -2.0581382390000726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1791s for a response
W20260709 13:04:57.334580 13915 connection.cc:256] RPC call timeout handler was delayed by -1.9969382390000874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
W20260709 13:04:57.334852 13915 connection.cc:256] RPC call timeout handler was delayed by -1.9834382390000656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
W20260709 13:04:57.335110 13915 connection.cc:256] RPC call timeout handler was delayed by -1.937538239000105s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
W20260709 13:04:57.335384 13915 connection.cc:256] RPC call timeout handler was delayed by -1.6468382390000897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22480000000000003s for a response
I20260709 13:04:57.310570 14553 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:04:57.340085 13840 leader_election.cc:336] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.858s (SENT)
I20260709 13:04:57.343612 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.344717 14023 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:57.345702 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:57.346760 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 148449us
I20260709 13:04:57.290275 16298 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:57.353480 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:57.353811 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:57.354161 14022 raft_consensus.cc:2376] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 2. Re-sending same reply.
I20260709 13:04:57.264505 16076 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.354417 14018 raft_consensus.cc:2376] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 9. Re-sending same reply.
I20260709 13:04:57.362484 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:57.363700 14016 raft_consensus.cc:2376] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 4. Re-sending same reply.
I20260709 13:04:57.324965 16350 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:57.365111 14017 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:57.329066 16209 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:57.366555 13868 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:57.355258 16314 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:57.418905 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.407302 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:04:57.411823 13840 leader_election.cc:336] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.006s (SENT)
I20260709 13:04:57.433290 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:57.434298 14019 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:57.435506 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.436107 14016 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:57.436923 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:57.349951 16212 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:57.407984 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.407673 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.361900 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:57.400434 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.375371 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:57.420990 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.443310 14023 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 2: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:57.367776 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:57.408794 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.444144 14024 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:57.444451 14023 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:04:57.428671 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 67537.2us
I20260709 13:04:57.431044 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:57.440511 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:57.442751 14021 raft_consensus.cc:2393] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 2: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:57.456147 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.458251 14020 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
W20260709 13:04:57.457439 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.462471 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 50743.5us
W20260709 13:04:57.463167 13840 leader_election.cc:336] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.162s (SENT)
I20260709 13:04:57.463591 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:57.464811 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:57.465463 13950 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:57.240541 16317 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.468678 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.468950 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.469513 14016 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:57.470202 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:57.471249 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:57.471939 13871 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:04:57.472165 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:57.472640 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:57.472770 13875 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:57.473353 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.474107 14019 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:57.469568 14017 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:57.475350 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:57.476027 13873 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:57.478773 13943 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:57.480165 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:57.481406 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:57.482156 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:57.482347 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.483551 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.484771 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:57.485067 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.485273 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.485900 13942 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:57.486373 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:57.486807 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.487040 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.487435 13941 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:57.487619 13944 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:04:57.488482 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.488978 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:57.490753 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:57.490947 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:57.491686 13871 raft_consensus.cc:2410] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:04:57.491852 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52074.6us
I20260709 13:04:57.434506 16296 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:57.468237 16328 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.620s	user 0.010s	sys 0.005s
I20260709 13:04:57.500972 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.502205 13869 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:57.496151 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.512044 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:57.513279 13873 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:57.431985 16350 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.513968 13871 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:04:57.514479 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.515312 14019 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
W20260709 13:04:57.519934 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 56788.7us
W20260709 13:04:57.520695 13840 leader_election.cc:336] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.240s (SENT)
I20260709 13:04:57.521163 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:57.513275 16324 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:57.531157 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.532136 14020 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
W20260709 13:04:57.533828 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 87988.7us
W20260709 13:04:57.523861 16355 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.379s	user 0.001s	sys 0.000s
I20260709 13:04:57.540992 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.555634 14019 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:57.545781 13838 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:57.557845 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.560437 13875 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:57.565913 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:57.540517 16364 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:04:57.546756 16321 consensus_meta.cc:287] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.004s
I20260709 13:04:57.532321 16212 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:57.504376 16328 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.567906 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.569599 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:04:57.571353 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.581827 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:57.580227 14020 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:57.562660 16172 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:57.587545 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.560674 16325 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.601233 14015 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:57.576648 16298 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.616099 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:57.592020 16209 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:57.594833 16339 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:57.579932 16098 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:04:57.617760 14553 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:57.623376 16209 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.606563 16321 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:57.601608 16180 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:57.610747 16267 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.587694 16347 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:04:57.586270 16360 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
W20260709 13:04:57.651661 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.642308 16283 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:04:57.659539 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.659771 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.609509 16325 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.632660 16076 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:57.642414 16209 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:57.645040 16321 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.645577 16180 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:57.659366 16340 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.517s	user 0.011s	sys 0.000s
I20260709 13:04:57.679972 13987 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:57.680980 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.681939 14019 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:04:57.682964 14559 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:57.626015 16298 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.675457 16180 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:57.675611 16321 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:57.675205 16346 raft_consensus.cc:476] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Not starting pre-election -- already a leader
I20260709 13:04:57.715483 16298 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:57.718662 16328 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:57.721171 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:57.725129 16372 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 13:04:57.725507 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 109946us
I20260709 13:04:57.726373 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:57.729079 16336 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:57.730144 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 58424.1us
I20260709 13:04:57.732501 13942 raft_consensus.cc:3055] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:57.732920 13942 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:57.733320 13944 raft_consensus.cc:3055] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:57.733700 13944 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:57.675104 16283 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.734239 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:57.734560 13944 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.735950 13944 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:57.736863 16283 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:57.737035 16336 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:57.698323 16350 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:57.738113 16362 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.738709 16362 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.739199 13941 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:57.733750 13942 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.740983 16362 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:57.741376 13942 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:57.684329 16340 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:57.655411 16267 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.708559 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:57.749737 13873 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:57.711086 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.750865 14017 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:57.709501 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:57.687139 16076 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:57.708159 13875 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:57.708186 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:57.744560 13872 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:57.685294 16181 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.063s
W20260709 13:04:57.757066 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 163935us
I20260709 13:04:57.696561 16322 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:57.767151 16211 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.671s	user 0.012s	sys 0.000s
I20260709 13:04:57.767657 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:57.711709 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62531us
I20260709 13:04:57.781307 13943 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
W20260709 13:04:57.773990 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.771652 16327 consensus_meta.cc:287] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.000s
I20260709 13:04:57.768963 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:57.749590 16361 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.243s	user 0.000s	sys 0.004s
W20260709 13:04:57.774305 16326 consensus_meta.cc:287] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.525s	user 0.000s	sys 0.014s
I20260709 13:04:57.776304 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:04:57.798511 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 84730.7us
W20260709 13:04:57.780475 16342 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.004s
I20260709 13:04:57.814685 16342 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.815925 13874 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:57.816663 14015 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:57.799186 13947 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:57.817811 13947 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:57.805445 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:57.818926 14023 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:57.818562 13947 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.820102 16342 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:57.805694 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:57.809264 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.805958 16329 consensus_meta.cc:287] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.622s	user 0.011s	sys 0.000s
I20260709 13:04:57.821519 14017 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:57.811578 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:57.821933 16329 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.822739 14017 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:57.811913 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:57.823771 16348 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:57.824119 14017 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:57.824244 16329 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:57.804493 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.809809 14020 raft_consensus.cc:2410] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:04:57.807936 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.805819 16337 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.000s
I20260709 13:04:57.820855 14015 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:57.808871 16349 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.584s	user 0.005s	sys 0.007s
I20260709 13:04:57.826345 16337 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.828020 14015 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:57.829888 16337 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:57.831992 14015 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:57.829977 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 101532us
I20260709 13:04:57.820904 14023 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:57.843007 13870 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:04:57.845201 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:57.845836 13870 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:57.846731 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:57.856456 13870 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
W20260709 13:04:57.839150 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 159228us
W20260709 13:04:57.773916 16343 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.000s
W20260709 13:04:57.775843 16061 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.624s	user 0.011s	sys 0.000s
I20260709 13:04:57.884521 14022 raft_consensus.cc:2410] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:04:57.885586 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52525.2us
I20260709 13:04:57.886430 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:57.887564 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.861387 14019 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:57.889150 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:57.771692 16265 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.002s
I20260709 13:04:57.866722 16329 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:57.867030 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47011.1us
I20260709 13:04:57.869069 16172 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:57.874178 13942 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:57.768867 16347 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:57.875227 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:04:57.878366 16242 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.003s
I20260709 13:04:57.902334 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:57.878620 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.881664 13873 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:57.891215 13944 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:57.908152 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:04:57.908751 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.859658 16332 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.910422 16332 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.913395 16332 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:57.900146 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:57.917279 16362 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:57.924414 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.909677 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.926808 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.928460 13944 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:57.930506 13838 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:57.931767 16342 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:57.932313 16342 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.932765 16342 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:57.934098 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:57.938644 16380 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:57.939200 16380 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.939646 16380 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:57.942365 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 56042.2us
I20260709 13:04:57.942708 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.944139 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.944803 13942 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:57.945613 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:57.946273 13948 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
W20260709 13:04:57.946748 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 71606.2us
I20260709 13:04:57.947790 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:57.949186 13987 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:57.876643 16211 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:57.951045 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.955749 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:57.945827 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:57.956848 13942 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:57.943120 13840 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:57.954510 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:57.969951 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:57.972267 13840 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:57.972275 13987 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:57.975752 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:57.978827 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:57.945390 16242 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:57.984848 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:58.034612 13948 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:57.969326 16061 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:58.032922 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 60717.1us
W20260709 13:04:58.028518 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52859.2us
W20260709 13:04:58.029071 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.038465 13942 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:04:57.899784 16180 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.039605 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:57.902815 16265 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.121130 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:58.131258 13950 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
W20260709 13:04:58.127009 13840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.735s (SENT) [suppressed 46 similar messages]
I20260709 13:04:57.954425 16076 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:57.947451 16385 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:57.954697 16359 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:58.033371 16310 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.767s	user 0.011s	sys 0.004s
I20260709 13:04:58.034482 16212 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:57.892199 16343 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:57.971024 16267 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:58.028395 16332 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:04:58.149178 16365 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:04:57.968135 14553 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:04:57.981009 16174 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:58.153604 14024 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:57.974829 16296 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:58.154352 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 114845us
I20260709 13:04:58.233369 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:57.981918 16327 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:04:58.157184 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 178452us
I20260709 13:04:57.980681 16375 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:58.035287 16340 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:58.139847 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.137261 16361 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:04:58.243650 13987 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:58.159665 16317 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:58.167869 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:58.162448 16351 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:58.178709 14018 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:58.181051 16342 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:57.942405 16347 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.183007 16343 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:57.970963 16211 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.186335 16363 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:58.207275 16383 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.341s	user 0.000s	sys 0.001s
I20260709 13:04:58.153611 16385 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:58.239604 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:58.194350 16332 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:58.247828 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 80075.1us
W20260709 13:04:58.262647 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.236428 16296 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.239218 16375 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.247012 16082 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:58.264420 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:58.265604 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:04:58.266039 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:58.249010 16076 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:58.259110 16363 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.270558 16375 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:58.272248 16363 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:58.272746 16321 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.001s
I20260709 13:04:58.273172 16321 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.274209 16211 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:58.274444 16082 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:58.266829 16296 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:58.255607 16061 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:58.276669 16061 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:58.278595 13941 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:04:58.260155 16386 log.cc:927] Time spent T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:04:58.264820 16332 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:58.263715 16385 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:58.277652 16061 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:58.292059 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52562.8us
I20260709 13:04:58.261845 16181 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:04:58.259282 16242 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:04:58.259603 16351 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:04:58.290475 16317 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:58.312235 16325 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:58.303190 16394 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:58.313259 14017 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:58.313639 16394 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:58.330152 13947 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:58.303474 16342 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:58.336131 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 70597.4us
I20260709 13:04:58.332213 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:04:58.306010 16317 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:58.346422 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:58.349092 16339 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:58.353762 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.356141 13873 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:04:58.343712 16209 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.658s	user 0.011s	sys 0.000s
I20260709 13:04:58.339136 16076 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.329030 16082 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:58.369663 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:58.321691 16359 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:04:58.332278 16180 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:58.404554 13947 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:58.373587 13948 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:04:58.342886 16363 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.348081 16317 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:58.370116 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:58.343559 16347 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:58.375213 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:58.376580 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:58.421404 13866 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:58.376938 16082 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.333102 16380 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.391526 16076 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.355980 16339 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:58.422199 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:04:58.424412 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 472993us
I20260709 13:04:58.442951 13838 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:58.441690 13950 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:58.445142 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:58.412608 16362 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:58.468647 13838 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:58.374825 16209 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:58.475137 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62605us
I20260709 13:04:58.480346 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:58.483129 13946 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:58.477849 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.475581 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:04:58.493264 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.466665 16316 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:58.495769 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:58.498342 14016 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:58.500617 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2044) took 1020 ms. Trace:
I20260709 13:04:58.501186 13943 rpcz_store.cc:276] 0709 13:04:57.479828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.479939 (+   111us) service_pool.cc:224] Handling call
0709 13:04:58.500376 (+1020437us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400091854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:58.500514 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":243394432}
I20260709 13:04:58.473959 16130 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:58.502308 16130 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.502818 16130 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:58.495180 16278 consensus_queue.cc:1059] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:04:58.504057 16278 raft_consensus.cc:3055] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:58.504419 16278 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:58.504863 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:58.423296 16363 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:58.472361 16321 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:58.375368 16394 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.505447 16278 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:58.505919 13941 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:58.797981 13838 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:58.784725 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1415) took 1431 ms. Trace:
I20260709 13:04:58.489404 16397 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.785542 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1414) took 1433 ms. Trace:
I20260709 13:04:58.803196 13838 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:58.804466 14024 rpcz_store.cc:276] 0709 13:04:57.352400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.444887 (+ 92487us) service_pool.cc:224] Handling call
0709 13:04:57.473343 (+ 28456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.232990 (+759647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.785256 (+552266us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b5400412654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:58.785472 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1552728832}
W20260709 13:04:58.615998 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.733198 13946 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
W20260709 13:04:58.794391 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.517319 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:58.813753 13943 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:04:58.795213 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 287546us
I20260709 13:04:58.502278 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:58.474138 16180 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:58.799139 14021 rpcz_store.cc:276] 0709 13:04:57.353183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.860241 (+507058us) service_pool.cc:224] Handling call
0709 13:04:57.873850 (+ 13609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.256993 (+383143us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.257027 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:58.257076 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:58.257119 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.784337 (+527218us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b5400239e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:58.784409 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.784632 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":880608128}
I20260709 13:04:58.815845 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:04:58.818780 16380 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:58.823329 16278 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:58.825197 13943 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:04:58.826375 16342 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:58.826851 16342 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.827210 16342 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:58.828027 13838 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:58.800966 16394 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:58.807175 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.807297 16397 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.569653 16130 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.790586 16362 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.836470 16209 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:58.738229 16316 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:58.734634 16400 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.566495 16076 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:58.844249 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2102) took 1137 ms. Trace:
I20260709 13:04:58.850843 13872 rpcz_store.cc:276] 0709 13:04:57.706768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.706925 (+   157us) service_pool.cc:224] Handling call
0709 13:04:57.707311 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.800721 (+ 93410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.843959 (+1043238us) spinlock_profiling.cc:243] Waited 36.9 ms on lock 0x7b54000da054. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:58.844182 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84844800}
I20260709 13:04:58.837755 16324 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:04:58.839367 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.493520 16375 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.849877 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:04:58.854955 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.571314 16296 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.839764 16321 raft_consensus.cc:2764] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
W20260709 13:04:58.864722 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.844594 16180 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:04:58.869864 16388 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.635s	user 0.008s	sys 0.006s
I20260709 13:04:58.846015 16362 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:58.875663 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.858539 16324 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:58.873329 16180 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:58.867344 16326 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:58.857532 16400 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:58.878600 16324 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:58.884409 16326 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:04:58.894946 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 67005.8us
I20260709 13:04:58.895898 13838 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:58.897663 16326 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.898259 16267 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:58.898989 16267 raft_consensus.cc:697] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:58.899080 16212 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:58.899670 16212 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.898378 16362 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:58.899907 16267 consensus_queue.cc:237] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.900758 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:58.901400 13943 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:58.901751 16212 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:58.903645 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:58.905611 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:58.907719 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2168) took 1324 ms. Trace:
I20260709 13:04:58.906666 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:58.908144 14020 rpcz_store.cc:276] 0709 13:04:57.582901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.615846 (+ 32945us) service_pool.cc:224] Handling call
0709 13:04:58.907512 (+1291666us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b540023ad54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:58.907653 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443136768}
W20260709 13:04:58.911697 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.912209 14522 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:04:58.914322 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:58.915010 13944 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 3: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:04:58.915524 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:04:58.919322 13874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2290) took 1659 ms (client timeout 1918 ms). Trace:
W20260709 13:04:58.919771 13874 rpcz_store.cc:269] 0709 13:04:57.260204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.260333 (+   129us) service_pool.cc:224] Handling call
0709 13:04:58.919123 (+1658790us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54000e9554. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:58.919258 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204114816}
I20260709 13:04:58.920647 16335 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:58.922473 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1474) took 1016 ms. Trace:
W20260709 13:04:58.924566 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.925136 13949 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:58.923290 13944 rpcz_store.cc:276] 0709 13:04:57.906484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.944012 (+ 37528us) service_pool.cc:224] Handling call
0709 13:04:58.914147 (+970135us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b5800202e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:04:58.922392 (+  8245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13684992}
I20260709 13:04:58.925701 13949 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:58.932235 16265 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:58.933316 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:04:58.934010 16393 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.683s	user 0.003s	sys 0.000s
I20260709 13:04:58.934219 14020 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:04:58.934882 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:58.935288 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:58.926919 16335 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:58.936089 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:58.934221 13949 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:58.936635 16335 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:58.937294 13949 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:58.938247 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:58.939440 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:04:58.940009 14024 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:58.940619 16395 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.706s	user 0.014s	sys 0.013s
W20260709 13:04:58.940989 16317 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.507s	user 0.006s	sys 0.003s
I20260709 13:04:58.927091 16362 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.942472 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:04:58.942565 16362 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:58.943156 14018 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:04:58.950564 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:58.954732 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:58.957039 13949 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:04:58.957295 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:58.941538 16317 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.966030 13989 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:58.966827 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:04:58.967350 13989 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:04:58.960402 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 92222us
I20260709 13:04:58.957309 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:58.968753 13944 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:04:58.922006 16212 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:58.884382 16316 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.972172 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:58.975711 14024 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:04:58.981904 13944 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:58.983106 13948 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:58.963932 16082 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.971141 16265 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:58.975164 16061 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.975358 16212 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.976965 16314 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:58.971102 16242 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.964818 16180 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.965703 16359 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:04:58.984256 16360 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:58.980314 16172 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:58.992282 16405 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:04:59.000262 13915 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:04:58.995519 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.997648 16325 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.999800 16212 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:58.995659 16082 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:58.986567 16288 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:04:58.981460 16380 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:04:58.999986 16061 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.001708 16130 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.005221 16397 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.004678 16360 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:59.021209 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 108124us
I20260709 13:04:59.024031 16379 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:59.017244 16337 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:59.020300 16209 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:59.076407 14522 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:59.002821 16314 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:58.992877 16324 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.066110 16360 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:59.002714 16242 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:59.077144 16409 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.187s	user 0.001s	sys 0.001s
W20260709 13:04:59.115190 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 135282us
W20260709 13:04:59.118289 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 118119us
W20260709 13:04:59.120047 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 152798us
I20260709 13:04:59.019073 16325 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.127790 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:59.128242 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1467) took 1242 ms. Trace:
I20260709 13:04:59.132815 13945 rpcz_store.cc:276] 0709 13:04:57.885342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.953538 (+ 68196us) service_pool.cc:224] Handling call
0709 13:04:57.953839 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.954046 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:04:57.981073 (+ 27027us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.933357 (+952284us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:04:59.127559 (+194202us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b18003ff880. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd323a
0709 13:04:59.127810 (+   251us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:59.127867 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:59.127915 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.128012 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.128141 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27342,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":2285,"spinlock_wait_cycles":77184128,"thread_start_us":172090,"threads_started":1,"wal-append.queue_time_us":251577}
I20260709 13:04:59.113171 16400 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:59.130033 16098 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:04:59.129805 16340 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.004s	sys 0.010s
I20260709 13:04:59.143373 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:59.120352 16317 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.085037 16379 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:59.185838 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.143998 16406 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.272s	user 0.000s	sys 0.004s
I20260709 13:04:59.106986 16412 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.033s
W20260709 13:04:59.204187 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.138998 13945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f50263f80db844239948e5e67ef3b987"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:04:59.232460 13945 raft_consensus.cc:606] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:04:59.173349 16380 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:59.180003 16340 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.004s	sys 0.010s
I20260709 13:04:59.241760 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:59.242736 13874 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:59.129712 16172 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
W20260709 13:04:59.235781 16409 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:04:59.117362 16337 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.140811 16209 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.104s
I20260709 13:04:59.251609 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:59.266638 13870 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
W20260709 13:04:59.226784 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 81933.4us
I20260709 13:04:59.203367 16400 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:59.271389 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:59.268865 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 155367us
I20260709 13:04:59.246392 16317 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:59.277894 16380 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:59.262406 16288 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:04:59.277112 16382 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
W20260709 13:04:59.304744 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.313642 16322 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:59.314374 16414 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 13:04:59.303117 16332 consensus_meta.cc:287] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.996s	user 0.017s	sys 0.000s
I20260709 13:04:59.298557 16288 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:59.302946 16419 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.352896 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:04:59.351294 16419 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:59.360119 16415 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:04:59.311173 16428 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.346000 16335 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.375577 16335 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:59.377684 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.379436 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.379212 16335 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:59.454653 13875 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:59.457279 13875 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:59.458806 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:59.383548 16362 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:59.384677 16356 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.388494 16296 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:59.437911 16342 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.000s
I20260709 13:04:59.396979 16343 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:04:59.404937 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.340134 16424 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:59.465399 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:59.346580 16082 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:59.425320 16437 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:04:59.430598 16422 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 13:04:59.431327 16392 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 1.189s	user 0.007s	sys 0.011s
W20260709 13:04:59.432431 16339 consensus_meta.cc:287] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.969s	user 0.005s	sys 0.014s
I20260709 13:04:59.432103 16426 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:59.433507 16242 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:59.434338 16379 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:59.353132 16317 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.441344 16337 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.441998 16314 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.443336 16325 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:59.349628 16265 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:59.371625 16209 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:59.349438 16332 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:59.447665 16385 consensus_meta.cc:287] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.141s	user 0.006s	sys 0.006s
I20260709 13:04:59.371210 16324 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.420673 16428 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.449028 16288 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:59.530735 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 251048us
I20260709 13:04:59.461246 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:59.394953 16076 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:59.464685 13870 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:59.536607 13868 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:04:59.303331 16380 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.443387 16433 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:59.549660 13941 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:59.471663 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.474803 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 122011us
W20260709 13:04:59.488488 14016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1421) took 2041 ms (client timeout 2248 ms). Trace:
I20260709 13:04:59.495986 13866 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:59.467384 16423 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:59.512589 16424 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:04:59.465126 16342 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:59.524470 16394 consensus_meta.cc:287] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.684s	user 0.009s	sys 0.005s
I20260709 13:04:59.539477 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:04:59.551066 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:04:59.550763 14016 rpcz_store.cc:269] 0709 13:04:57.447360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.720879 (+273519us) service_pool.cc:224] Handling call
0709 13:04:59.488073 (+1767194us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b540030cad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:59.488374 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1784927104}
I20260709 13:04:59.470005 16343 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.554100 16082 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:04:59.469229 16356 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.520102 16317 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.574509 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1475) took 1101 ms. Trace:
W20260709 13:04:59.555043 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 283753us
I20260709 13:04:59.575029 13948 rpcz_store.cc:276] 0709 13:04:58.473138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.473375 (+   237us) service_pool.cc:224] Handling call
0709 13:04:58.494037 (+ 20662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.983510 (+489473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.574229 (+590719us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b5400053f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:59.574449 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123805568}
I20260709 13:04:59.552445 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:59.559544 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:04:59.517894 16339 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.546629 16076 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.552232 16426 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.564110 16423 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.530586 16385 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.552953 16432 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:04:59.577536 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:59.578713 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:59.579324 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:59.581884 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:59.585397 13869 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:59.586724 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:59.587875 13872 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:59.515051 16437 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.527594 16209 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.562722 16436 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.554018 16379 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:04:59.643342 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 92380.5us
I20260709 13:04:59.696261 13866 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:59.712947 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:04:59.675385 16360 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.003s
I20260709 13:04:59.696919 13870 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:59.569514 16296 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:59.637029 16278 consensus_meta.cc:287] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.813s	user 0.013s	sys 0.001s
I20260709 13:04:59.726451 13942 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:59.697469 13867 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:59.562462 16335 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:59.700305 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 110498us
I20260709 13:04:59.707449 16359 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:04:59.715569 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 58329us
I20260709 13:04:59.697535 13874 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:04:59.696516 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 119073us
W20260709 13:04:59.636601 16212 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.611s	user 0.012s	sys 0.000s
I20260709 13:04:59.699421 16076 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:59.673779 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 588) took 1928 ms. Trace:
I20260709 13:04:59.597563 16351 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:59.664422 14553 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:59.635229 16317 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.638149 16342 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.735783 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:04:59.734596 13987 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:04:59.621830 16442 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:59.731590 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:04:59.738137 16418 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:04:59.738842 16450 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:04:59.738931 16429 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:59.801437 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:59.740245 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:04:59.717631 16424 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.743599 16437 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.780650 16400 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:59.742834 14560 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:59.637913 16339 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:59.632901 16343 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:59.744248 13867 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:59.744980 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:04:59.782099 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51434.5us
I20260709 13:04:59.826706 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:04:59.783771 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.789904 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.790778 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:59.830633 14554 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:04:59.728044 16420 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:59.791759 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1481) took 1310 ms. Trace:
W20260709 13:04:59.794183 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 59659.3us
I20260709 13:04:59.832718 13950 rpcz_store.cc:276] 0709 13:04:58.481330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.498417 (+ 17087us) service_pool.cc:224] Handling call
0709 13:04:59.791555 (+1293138us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b54001c7354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:04:59.791694 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1604192000}
I20260709 13:04:59.743284 16335 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.589351 16082 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.800233 16454 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:59.804023 16209 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:59.804200 14559 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:59.731192 16296 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.804500 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:04:59.709156 16426 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.741622 16380 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:04:59.815161 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 74996.7us
I20260709 13:04:59.741983 16452 consensus_queue.cc:1059] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:59.721030 16379 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.631826 16265 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:59.813637 16453 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:04:59.783382 16180 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:04:59.720117 16360 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.782820 16428 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.781854 16278 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:04:59.838361 16437 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:59.838615 16098 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:59.839154 16385 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.839619 16442 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:59.891225 16442 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:59.891710 16442 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:59.892784 16442 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.780702 14551 rpcz_store.cc:276] 0709 13:04:57.745586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.750721 (+  5135us) service_pool.cc:224] Handling call
0709 13:04:57.769365 (+ 18644us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:57.769826 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:57.769862 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:57.770622 (+   760us) log.cc:844] Serialized 52 byte log entry
0709 13:04:58.124273 (+353651us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:58.148954 (+ 24681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.157678 (+  8724us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.315386 (+1157708us) spinlock_profiling.cc:243] Waited 45.5 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:04:59.662753 (+347367us) raft_consensus.cc:1685] finished
0709 13:04:59.662787 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.673659 (+ 10872us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":939,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"spinlock_wait_cycles":104578816,"thread_start_us":150063,"threads_started":1,"wal-append.queue_time_us":397851}
I20260709 13:04:59.894502 16442 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:59.872329 16267 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:59.895623 16316 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:59.895691 16267 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:04:59.896202 16316 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:59.896791 16267 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.842514 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:59.780117 16212 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.803944 16342 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.898705 16342 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.803004 16363 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:59.816174 16332 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.868014 16375 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.720906 16436 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.863405 16466 consensus_queue.cc:1059] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:59.866144 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:04:59.870767 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 74866.8us
I20260709 13:04:59.877684 16451 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:59.847821 16449 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:59.860064 16447 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.923767 16466 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:59.924221 16466 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:59.924120 16447 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:04:59.886871 16458 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:04:59.858867 16454 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.888752 16423 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:59.929464 16447 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.897037 16316 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.800128 16417 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:04:59.899717 16426 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:04:59.905448 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 63456.2us
I20260709 13:04:59.930313 16417 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.930891 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:04:59.852873 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1484) took 1113 ms. Trace:
W20260709 13:04:59.914994 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.931604 13941 rpcz_store.cc:276] 0709 13:04:58.739944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.802871 (+ 62927us) service_pool.cc:224] Handling call
0709 13:04:58.803300 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.550241 (+746941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.852513 (+302272us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b54003b01d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:04:59.852781 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1715919104}
I20260709 13:04:59.930316 16316 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:59.833668 16400 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:04:59.877542 16457 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:59.935601 16426 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:59.936097 16426 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:04:59.937662 16400 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:59.923800 16449 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.925269 16466 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:04:59.878154 16440 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.937207 16426 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:59.941009 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.897966 16267 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:59.900013 16212 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:59.876161 16429 raft_consensus.cc:3055] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:59.879827 16278 consensus_queue.cc:1243] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:04:59.872991 16180 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:04:59.969129 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 103097us
I20260709 13:04:59.908227 16418 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:59.958761 16436 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:59.948787 16061 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:59.904861 16242 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.974788 16434 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:59.975250 16340 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:04:59.980199 16454 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:04:59.946491 16130 raft_consensus.cc:3055] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:59.955933 16342 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:59.957199 16082 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:59.959628 16451 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:04:59.959815 16379 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:04:59.960836 16423 consensus_queue.cc:1059] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:04:59.961881 16360 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:59.983923 16449 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:59.989948 16343 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.962327 16339 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:00.017628 16343 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.020213 16343 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:04:59.946398 16363 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:04:59.989687 16337 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:05:00.022861 16363 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.023339 16363 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.023877 16180 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:00.023991 16324 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:00.024469 16324 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:00.025424 16339 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:00.026746 16340 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:00.028497 16339 raft_consensus.cc:3055] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:00.031525 16339 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:00.026392 16375 raft_consensus.cc:3055] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:00.032256 16375 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:00.004206 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1473) took 2098 ms. Trace:
W20260709 13:05:00.035107 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62754.1us
I20260709 13:05:00.035880 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.030424 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:00.030401 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.038477 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:00.035153 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.039320 13941 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:00.042490 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.043401 13949 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:00.043963 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.044312 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:00.044659 13946 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:00.045511 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:00.045832 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:00.046588 13947 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:00.046892 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:00.047616 13941 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:05:00.025364 16324 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.039037 13950 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:05:00.049417 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:00.049932 13950 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:00.051489 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.052085 13946 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:00.053126 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.054028 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:00.055032 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.055032 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.057798 13949 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:00.059173 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.060694 13946 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:05:00.034083 13942 rpcz_store.cc:276] 0709 13:04:57.906187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.136620 (+230433us) service_pool.cc:224] Handling call
0709 13:04:58.136990 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.726934 (+1589944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.003830 (+276896us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b5400091354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:00.004117 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3654897024}
I20260709 13:04:59.972528 16432 consensus_queue.cc:1059] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:05:00.063905 16351 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:00.063941 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.066253 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.066305 13944 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:00.007824 16288 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:00.069967 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:00.070369 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:00.073385 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:00.074211 13868 raft_consensus.cc:2410] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:00.077045 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:00.077721 13866 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:00.029983 16130 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:00.032673 16451 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:00.081341 16335 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:00.082515 16130 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:00.078598 13948 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:00.083666 16130 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:00.071343 16418 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:05:00.065619 16466 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:00.087234 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:00.058300 16061 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:00.034423 16429 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:00.029240 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:00.035270 16434 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.064723 16337 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:59.942006 16076 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.090603 13866 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:05:00.029778 16340 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:00.016017 16296 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:00.098155 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 60094.9us
I20260709 13:05:00.050848 16433 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:00.107400 13941 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:00.089546 13867 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:59.921135 16465 consensus_queue.cc:1059] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.188s
I20260709 13:05:00.096973 16061 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:00.058818 16417 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:00.033176 16375 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:00.165854 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.169466 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:00.048767 16242 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.050041 16428 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:05:00.179615 14561 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
W20260709 13:05:00.183135 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.122443 16434 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:00.206537 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:00.187781 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:00.188876 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:00.124596 16337 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:00.142946 16379 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:05:00.199473 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.203329 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:00.172874 16356 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:00.207484 14556 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
I20260709 13:04:59.999172 16440 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:00.163000 16360 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:00.137431 16339 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:00.520402 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.881315 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:00.887284 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.887492 13941 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:00.535099 13771 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:00.859587 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.888620 16467 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:00.207996 16340 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.886126 16356 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:00.381635 16429 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.503655 16337 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:00.862766 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:00.858668 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:00.859447 16267 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.842s	user 0.012s	sys 0.001s
I20260709 13:05:00.574455 16452 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:00.889953 16467 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:00.846092 16463 raft_consensus.cc:3055] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:00.897221 16463 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:00.860020 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:00.858354 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:00.275362 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.242445 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:00.419230 16449 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.791457 13943 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:00.900095 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:00.611579 16209 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:00.534771 16468 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.455s	user 0.000s	sys 0.000s
I20260709 13:05:00.891582 16356 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:00.901095 13941 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:00.896307 16420 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:00.715806 13950 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.665s	user 0.008s	sys 0.002s
I20260709 13:05:00.426955 16419 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:00.903563 16420 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.900863 13943 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:00.898510 16467 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:00.344013 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 471633us
I20260709 13:05:00.898270 14560 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:00.898780 14554 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:00.178776 16061 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:00.907008 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:00.788630 16242 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:00.905714 16420 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:00.910418 16242 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.911229 13950 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:05:00.911284 16242 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.611635 16076 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:00.890906 16379 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:00.892515 16429 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:00.912628 13947 raft_consensus.cc:2410] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:00.913964 16379 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:00.868676 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.896003 16360 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.896363 16267 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:00.912254 13943 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.884485 16375 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:00.905660 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.918699 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.876039 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:00.919250 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.919476 13946 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:05:00.611876 16359 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.897513 16452 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.920339 16420 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:00.917111 16267 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:00.193868 16343 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.922192 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.921871 16343 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:00.923586 16267 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:00.923734 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.920920 16420 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.925673 13949 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:00.890232 16340 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.913618 16337 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:00.901211 16209 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.918232 16242 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:00.904953 14553 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:05:00.920109 16359 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.929401 13949 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:00.934178 13943 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:00.858866 16324 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:00.921147 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.947286 13941 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:00.960918 13950 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
W20260709 13:05:00.947608 13987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.000s	sys 0.016s
I20260709 13:05:00.951146 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.953728 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.904192 16356 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:00.962457 13947 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:00.925585 16325 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.926502 16363 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:00.962428 13987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.000s	sys 0.017s
I20260709 13:05:00.924597 16449 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:00.983525 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:00.983994 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:00.984969 14018 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:00.990118 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:00.888926 16440 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:00.939154 16379 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:00.904958 16419 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.990900 13948 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:00.992616 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:00.993221 14023 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:00.994297 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:01.011030 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:01.024861 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.026610 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:00.898502 16463 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:01.228222 16316 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.292s	user 0.013s	sys 0.000s
W20260709 13:05:01.516577 13839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.008s	sys 0.023s
W20260709 13:05:01.708464 13839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.014s	sys 0.028s
I20260709 13:05:01.036047 14019 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:05:01.710492 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.186995 16363 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:01.438452 13989 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.005s	sys 0.003s
I20260709 13:05:01.030534 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:01.663761 16466 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.578s	user 0.014s	sys 0.000s
W20260709 13:05:01.283107 16434 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.985s	user 0.013s	sys 0.000s
I20260709 13:05:01.719990 14018 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:01.711890 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:01.720530 16473 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:05:01.720738 13872 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
W20260709 13:05:01.306877 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.721776 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2092) took 1674 ms. Trace:
W20260709 13:05:01.722553 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.722309 13944 rpcz_store.cc:276] 0709 13:05:00.047321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.076169 (+ 28848us) service_pool.cc:224] Handling call
0709 13:05:00.076495 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:00.076865 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:00.076898 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:00.077438 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 13:05:00.304319 (+226881us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:00.304367 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.304448 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.721561 (+1417113us) raft_consensus.cc:1685] finished
0709 13:05:01.721583 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.721698 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3599,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":226435,"threads_started":1,"wal-append.queue_time_us":1306645}
I20260709 13:05:01.093317 16477 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.400804 13943 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:01.721791 13989 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.008s	sys 0.005s
I20260709 13:05:01.100545 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:01.732837 13941 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 3.
I20260709 13:05:01.712476 13868 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:01.679924 16429 consensus_meta.cc:287] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.766s	user 0.004s	sys 0.007s
I20260709 13:05:01.736771 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:01.737748 13868 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:01.739774 13868 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:01.507576 16471 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:05:01.742057 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:01.742923 14525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.011s	sys 0.007s
W20260709 13:05:01.743302 14525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.952s	user 0.012s	sys 0.007s
W20260709 13:05:01.743700 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.725247 16278 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:05:01.724658 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:01.725622 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 841892us
I20260709 13:05:01.726197 16419 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:01.272912 16288 consensus_meta.cc:287] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.194s	user 0.013s	sys 0.000s
I20260709 13:05:00.989086 16325 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:00.987972 16082 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.728675 16463 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:01.706454 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2086) took 1671 ms. Trace:
I20260709 13:05:01.784302 13945 rpcz_store.cc:276] 0709 13:05:00.035484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.036714 (+  1230us) service_pool.cc:224] Handling call
0709 13:05:00.037033 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:00.037381 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:00.037410 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:00.037905 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:05:00.881037 (+843132us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:00.881078 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.881145 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.308583 (+427438us) raft_consensus.cc:1685] finished
0709 13:05:01.308612 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.706366 (+397754us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1782,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":16648,"thread_start_us":177471,"threads_started":1,"wal-append.queue_time_us":1224234}
W20260709 13:05:01.695598 13988 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.006s	sys 0.013s
I20260709 13:05:01.718662 13947 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
W20260709 13:05:01.786911 13988 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.007s	sys 0.014s
I20260709 13:05:01.091665 16440 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:01.788908 14522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.038s	sys 0.016s
I20260709 13:05:01.333359 16061 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:01.034775 16469 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.528s	user 0.000s	sys 0.001s
I20260709 13:05:01.293738 16467 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:01.550402 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2091) took 1507 ms. Trace:
I20260709 13:05:00.924743 16343 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:01.708250 16420 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:01.735371 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:01.744006 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.744745 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:01.794723 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:01.745544 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.314194 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.795725 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.719827 16434 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:01.747478 14022 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:01.748514 13944 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
W20260709 13:05:01.746636 13840 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.011s	sys 0.020s
W20260709 13:05:01.799870 13840 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.011s	sys 0.020s
W20260709 13:05:01.800310 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 884665us
I20260709 13:05:01.746510 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:01.750320 13989 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:01.801188 13840 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.735s (SENT)
I20260709 13:05:01.752190 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2081) took 1724 ms. Trace:
I20260709 13:05:01.727066 16466 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.753902 13875 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:01.756975 13870 pending_rounds.cc:85] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 2
W20260709 13:05:01.805519 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55296.5us
I20260709 13:05:01.805711 13870 pending_rounds.cc:107] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:05:01.806092 13870 raft_consensus.cc:2889] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:01.724397 16477 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.758574 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:01.578347 16130 consensus_meta.cc:287] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.494s	user 0.017s	sys 0.001s
I20260709 13:05:01.728080 16485 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:01.767309 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.240278 16451 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.026s	sys 0.028s
W20260709 13:05:01.673702 16379 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.002s
I20260709 13:05:01.770066 16363 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:01.787403 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 929878us
W20260709 13:05:01.126547 14551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.015s	sys 0.042s
W20260709 13:05:01.789443 14522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.038s	sys 0.016s
W20260709 13:05:01.749403 16375 consensus_meta.cc:287] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.830s	user 0.007s	sys 0.007s
W20260709 13:05:01.754004 16324 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.774s	user 0.013s	sys 0.000s
I20260709 13:05:01.819061 16429 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.819553 16429 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.821224 14024 raft_consensus.cc:2410] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:01.801905 16288 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.186959 16267 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.791427 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:01.748045 16359 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:01.792411 13942 rpcz_store.cc:276] 0709 13:05:00.047069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.071313 (+ 24244us) service_pool.cc:224] Handling call
0709 13:05:00.071665 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:00.072018 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:00.072046 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:00.072629 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:05:00.211758 (+139129us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b180028c540. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1
0709 13:05:00.212008 (+   250us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:00.212067 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.212176 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.549758 (+1337582us) raft_consensus.cc:1685] finished
0709 13:05:01.549802 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.550278 (+   476us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5021,"raft.queue_time_us":134,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":26222,"spinlock_wait_cycles":12539648,"thread_start_us":31447,"threads_started":1,"wal-append.queue_time_us":1247211}
I20260709 13:05:01.793784 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:01.810060 16343 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:01.827601 14015 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:01.796006 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.796517 14020 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:01.796617 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:01.828728 13947 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:01.711642 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:01.829017 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:01.801733 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:01.801752 13943 rpcz_store.cc:276] 0709 13:05:00.027736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.030595 (+  2859us) service_pool.cc:224] Handling call
0709 13:05:00.030906 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:00.791338 (+760432us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54001ec920. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33aadc0 00007fb2c34351c0 00007fb2c3436231 00007fb2c3432d8f 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c
0709 13:05:01.725042 (+933704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.752127 (+ 27085us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725226752}
I20260709 13:05:01.768862 16457 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:01.833364 13944 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:01.803272 14021 raft_consensus.cc:2393] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:01.806869 13987 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:49644 dropped due to backpressure: service queue is full with 50 items
I20260709 13:05:01.837229 16375 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:01.839615 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:01.840281 14020 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:01.840124 16375 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.834484 16324 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:01.842880 16324 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.808142 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:01.809544 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.844472 14022 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:01.812722 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.844992 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.781526 14023 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:01.790198 13874 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:01.804630 16420 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.843429 16324 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.855626 16420 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.856921 16477 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:01.857343 16324 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:01.857681 16420 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:01.824241 16490 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:01.824520 16061 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:01.813545 16451 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.438s	user 0.026s	sys 0.038s
I20260709 13:05:01.861047 16490 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.839143 13987 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:49644 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1441)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1439)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1447)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1469)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1467)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1462)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1455)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2192)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1468)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1470)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2185)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1453)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1451)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1442)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1449)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1452)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1450)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1446)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1463)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1460)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1456)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1448)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2184)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2187)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2189)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2195)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1436)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1437)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1440)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1443)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1457)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1458)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1459)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1461)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1464)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1465)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1466)
I20260709 13:05:01.866994 16325 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:01.867057 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:01.871069 16429 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:01.871695 16363 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:01.873226 16419 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:01.837680 16457 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:01.820720 14551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.353s	user 0.026s	sys 0.058s
I20260709 13:05:01.869755 16434 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:01.794613 16440 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.831182 13871 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:01.831060 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:01.901319 14016 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:01.832078 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.902452 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.903014 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.834915 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:01.821130 16130 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.846690 13838 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:01.848045 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:01.904021 14021 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:01.905061 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:01.854552 13869 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:05:01.789350 16082 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:01.860213 13947 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:01.863370 16490 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:01.810068 16466 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.797894 16467 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.883307 14015 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:01.921264 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:01.888509 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:01.922403 13942 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:05:01.919145 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:01.832609 16359 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.905676 13943 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
W20260709 13:05:01.910009 13915 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.517s (SENT)
I20260709 13:05:01.910549 14020 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:05:01.910676 13875 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:01.930799 13915 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:01.875523 16375 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:01.933506 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:01.828239 16288 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:01.917717 13872 raft_consensus.cc:2410] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:05:01.817350 16485 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:01.893685 16451 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.830456 16267 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.923413 14016 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:01.956264 16440 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:01.956863 16440 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:01.957295 16440 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.957515 16267 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:01.957722 16440 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:01.958292 16498 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:01.919883 16434 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.959510 16434 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:01.961694 16434 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:01.938714 16359 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.962280 16451 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.964171 16359 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:01.964344 16451 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:01.965880 16466 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:01.924170 13873 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:01.969535 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:01.926564 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:01.910714 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:01.935860 16429 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:01.971118 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:01.971747 13873 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:05:01.931368 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:01.932237 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:01.972675 14024 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:01.972757 14020 raft_consensus.cc:2368] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6.
I20260709 13:05:01.973611 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:01.974200 14024 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:01.929003 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:01.975040 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46169.4us
I20260709 13:05:01.976939 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:01.976734 16359 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:01.909435 16130 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.978418 16359 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.978359 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:01.979077 16359 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:01.979826 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:01.978390 16484 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:01.911971 16324 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.981536 16484 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:01.982009 16419 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:01.982493 16324 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:01.914402 16497 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:01.954578 16457 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:01.986756 16497 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.957000 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:01.988051 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:01.988896 14556 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:05:01.988931 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:01.989728 16485 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:01.971450 16429 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:01.991551 16082 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:01.991667 16429 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:01.933110 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:01.909409 16495 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:05:01.983311 16419 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:01.993616 16419 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:01.955425 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 45451.5us
I20260709 13:05:01.989884 14016 raft_consensus.cc:2368] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6.
I20260709 13:05:01.982582 16484 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.918625 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 591) took 1925 ms. Trace:
I20260709 13:05:01.916149 16351 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.018220 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:01.921828 16480 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.921578 16076 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:02.022524 14016 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:01.939764 16375 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.993150 16495 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:01.986222 16288 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:01.981452 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:01.973894 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:01.931238 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:02.004314 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:01.925086 14015 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:02.007910 13914 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:02.004984 13942 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:02.040410 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:02.043295 14024 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:02.042832 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:02.097360 16457 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:02.109660 16457 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:02.110046 16457 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:02.114864 14020 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:05:02.116194 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:02.117066 14020 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:02.117981 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:02.097476 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 89665.3us
I20260709 13:05:02.118873 14020 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:02.097132 13943 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:02.119822 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:02.121552 14020 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:02.119988 16457 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:02.126269 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:02.126823 14556 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:02.129287 14020 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:02.130589 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:02.131172 14020 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:02.134218 14556 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:02.136132 13914 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:02.136907 16466 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:02.137446 16466 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:02.138031 14020 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:02.138514 16466 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:02.139079 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:02.013099 16484 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:02.141247 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:02.141893 14020 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:02.148821 13943 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:02.196147 13947 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 3: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:05:02.211162 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.098330 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:02.231454 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:02.043324 13874 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:02.099429 14016 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:02.237155 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:02.231554 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 68404.4us
I20260709 13:05:02.238300 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:02.239120 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:02.232587 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2185) took 1237 ms. Trace:
I20260709 13:05:02.239639 13942 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:02.239567 14024 rpcz_store.cc:276] 0709 13:05:00.995697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.097912 (+1102215us) service_pool.cc:224] Handling call
0709 13:05:02.232517 (+134605us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40960}
I20260709 13:05:02.233034 13943 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:02.234915 13875 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:02.098330 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:02.241953 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:02.243067 13875 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:02.235651 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 254292us
W20260709 13:05:02.235702 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 389048us
I20260709 13:05:02.244238 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:02.245288 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:02.245230 13838 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:02.234218 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:02.243026 14015 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:02.248431 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:02.247225 14020 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:02.250821 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:02.251151 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:02.251602 14024 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:02.226430 16343 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:02.243192 16337 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.296s	user 0.005s	sys 0.009s
I20260709 13:05:02.227581 16457 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:02.254901 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:02.256678 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:02.257345 14020 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:02.023605 16351 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:02.257258 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:02.253669 13987 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:02.099717 16324 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:02.257483 13947 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:02.246765 16429 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:02.099581 16375 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:02.297976 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.252395 16082 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:02.298689 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.331933 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:02.034927 16076 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:02.334017 14018 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:05:02.336445 14018 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:02.337194 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2184) took 1342 ms. Trace:
I20260709 13:05:02.337617 14018 rpcz_store.cc:276] 0709 13:05:00.995234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.335766 (+1340532us) service_pool.cc:224] Handling call
0709 13:05:02.336182 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.336897 (+   715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.337122 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:02.260039 16428 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.280123 14015 raft_consensus.cc:2410] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:05:02.368624 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.280797 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:02.369606 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.304697 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:02.251000 16363 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:02.232594 16500 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:02.098407 16495 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:02.314064 14016 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:02.390784 14024 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:02.390766 16495 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:02.391896 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2188) took 1393 ms. Trace:
I20260709 13:05:02.344564 14021 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:02.392748 16363 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:02.023777 16480 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:02.393863 14021 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:02.393718 16363 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:02.395537 14021 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:02.396440 16429 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:02.353446 16061 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.397553 16061 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:02.399051 16497 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:02.400219 16061 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:02.359217 16506 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:02.352903 16351 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:02.361219 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 395) took 1357 ms. Trace:
I20260709 13:05:02.360211 16360 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:02.365530 14553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59630 (request call id 380) took 2197 ms. Trace:
I20260709 13:05:02.401480 13946 rpcz_store.cc:276] 0709 13:05:01.004029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.004260 (+   231us) service_pool.cc:224] Handling call
0709 13:05:01.718358 (+714098us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.718779 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.718809 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:01.719377 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:05:01.739683 (+ 20306us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:01.739727 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.739800 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.743218 (+  3418us) raft_consensus.cc:1685] finished
0709 13:05:01.743238 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.360907 (+617669us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54001c6450. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:02.361144 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":729,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":101,"spinlock_wait_cycles":1641267840,"thread_start_us":19916,"threads_started":1,"wal-append.queue_time_us":21280}
I20260709 13:05:02.402560 14553 rpcz_store.cc:276] 0709 13:05:00.168095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.173574 (+  5479us) service_pool.cc:224] Handling call
0709 13:05:02.365281 (+2191707us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b54007148d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:02.365460 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32360832}
I20260709 13:05:02.363317 16452 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:02.370471 14020 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:02.403955 14021 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:02.404577 14024 rpcz_store.cc:276] 0709 13:05:01.238720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.372075 (+1133355us) service_pool.cc:224] Handling call
0709 13:05:02.390523 (+ 18448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.391522 (+   999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.391813 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:02.256237 16451 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:05:02.406018 14024 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:02.014297 14551 rpcz_store.cc:276] 0709 13:04:59.989736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.006187 (+ 16451us) service_pool.cc:224] Handling call
0709 13:05:00.006830 (+   643us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:00.065149 (+ 58319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:00.071811 (+  6662us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:00.208886 (+137075us) log.cc:844] Serialized 26 byte log entry
0709 13:05:01.902219 (+1693333us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:01.902276 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.902356 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.902413 (+    57us) raft_consensus.cc:1685] finished
0709 13:05:01.902428 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.915509 (+ 13081us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43898,"raft.queue_time_us":57175,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":1442103,"threads_started":1,"wal-append.queue_time_us":1374635}
I20260709 13:05:02.409317 16343 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:02.410269 14024 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:02.411513 14024 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:02.411958 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:02.412680 13874 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
W20260709 13:05:02.413341 13839 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.180s (SENT)
I20260709 13:05:02.413827 13839 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:02.382040 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:02.382973 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:02.388103 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:02.392024 16495 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:02.296940 13875 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:02.400329 14017 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:02.433496 14561 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:02.433777 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:02.434733 13875 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
W20260709 13:05:02.434134 14561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59630 (request call id 386) took 2174 ms (client timeout 2496 ms). Trace:
W20260709 13:05:02.436206 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 437984us
W20260709 13:05:02.436913 13915 leader_election.cc:336] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.576s (SENT)
I20260709 13:05:02.437353 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:02.441506 14024 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:02.342777 16082 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:02.527966 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:02.908514 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.453526 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:02.487135 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 64686.7us
W20260709 13:05:02.911808 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:02.913398 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.483636 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:02.919859 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2319) took 1161 ms. Trace:
W20260709 13:05:02.715926 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 435269us
W20260709 13:05:02.922123 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:02.529461 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 92547.1us
W20260709 13:05:02.563125 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.663256 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2190) took 1371 ms. Trace:
I20260709 13:05:02.453197 14016 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:02.452786 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:02.917613 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2308) took 1205 ms. Trace:
I20260709 13:05:02.931397 14015 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:02.721376 14019 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:02.644402 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1436) took 1140 ms. Trace:
I20260709 13:05:02.932976 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2312) took 1189 ms. Trace:
W20260709 13:05:02.657194 14522 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.010s	sys 0.026s
I20260709 13:05:02.815661 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:02.642956 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:02.661291 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:05:02.435513 14561 rpcz_store.cc:269] 0709 13:05:00.259611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.381240 (+2121629us) service_pool.cc:224] Handling call
0709 13:05:02.434073 (+ 52833us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:02.936914 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:02.532657 16209 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:02.942040 16082 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:02.843562 16130 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:02.544854 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:02.838869 16076 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:02.945073 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2098) took 1078 ms. Trace:
I20260709 13:05:02.947011 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:02.833369 16417 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:02.614601 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:02.936983 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:02.965377 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.774883 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:02.836148 16480 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:02.967414 14555 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:02.510023 16497 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:02.967967 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:02.968452 14552 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:05:02.969063 14553 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:02.969509 13875 raft_consensus.cc:2410] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:02.640187 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:02.644953 16340 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:02.613164 16359 consensus_meta.cc:287] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.634s	user 0.013s	sys 0.000s
I20260709 13:05:02.651832 16375 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:02.517668 16498 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:02.538389 16419 consensus_meta.cc:287] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.544s	user 0.008s	sys 0.002s
W20260709 13:05:02.608721 16440 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.000s
I20260709 13:05:02.532704 16467 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:02.547103 16339 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:02.969444 14023 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:02.970037 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:02.373401 16489 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:02.973898 14017 rpcz_store.cc:276] 0709 13:05:01.292275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.386598 (+1094323us) service_pool.cc:224] Handling call
0709 13:05:02.386906 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.430759 (+ 43853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.662928 (+232169us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b540030cad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:02.663177 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30404480}
I20260709 13:05:02.983335 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:02.973318 16484 consensus_meta.cc:287] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.730s	user 0.016s	sys 0.003s
I20260709 13:05:02.989476 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:02.989005 16484 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:02.989061 16489 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:02.976356 16419 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:02.972110 16359 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:02.995379 14017 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:02.997073 16359 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:02.998181 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1466) took 1208 ms. Trace:
I20260709 13:05:02.977218 16440 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:02.969069 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:02.999423 14555 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:05:03.000032 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.001081 13874 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:03.001194 13875 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:03.002343 14522 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.964s	user 0.021s	sys 0.042s
W20260709 13:05:03.003033 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 1.00254e+06us
I20260709 13:05:03.003307 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.004336 13875 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:03.005559 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.001179 16440 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:03.008211 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.009433 13874 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:02.382386 16428 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:03.010140 16489 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:02.735945 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:03.013590 16515 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:03.012020 14017 rpcz_store.cc:276] 0709 13:05:01.789717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.990512 (+1200795us) service_pool.cc:224] Handling call
0709 13:05:02.991133 (+   621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.995855 (+  4722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.997749 (+  1894us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b5400411c54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:02.998014 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9084160}
I20260709 13:05:02.631399 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.016163 14017 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:03.017063 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1473) took 1152 ms. Trace:
I20260709 13:05:02.516191 16436 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:02.487584 16447 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.018795 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:03.019366 14554 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:02.627995 16426 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:02.482263 14024 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:03.022580 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1464) took 1233 ms. Trace:
I20260709 13:05:03.019011 16447 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:03.017526 14017 rpcz_store.cc:276] 0709 13:05:01.864935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.014974 (+1150039us) service_pool.cc:224] Handling call
0709 13:05:03.015548 (+   574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.016660 (+  1112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.016966 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:02.959316 16417 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:02.951483 13872 rpcz_store.cc:276] 0709 13:05:01.743889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.744379 (+   490us) service_pool.cc:224] Handling call
0709 13:05:02.932397 (+1188018us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54000da054. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:02.932725 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280536448}
W20260709 13:05:03.264587 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.955374 13944 rpcz_store.cc:276] 0709 13:05:01.865960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.869671 (+  3711us) service_pool.cc:224] Handling call
0709 13:05:01.870109 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:01.875185 (+  5076us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:02.227705 (+352520us) log.cc:844] Serialized 14 byte log entry
0709 13:05:02.555584 (+327879us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:02.555667 (+    83us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:02.555747 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.930106 (+374359us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b54000f4e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:02.944333 (+ 14227us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.944743 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20806,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":149,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":33282,"spinlock_wait_cycles":11109632,"thread_start_us":679486,"threads_started":2,"wal-append.queue_time_us":435788}
I20260709 13:05:02.950506 13869 rpcz_store.cc:276] 0709 13:05:01.712164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.735021 (+ 22857us) service_pool.cc:224] Handling call
0709 13:05:02.917036 (+1182015us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b540026c5d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:02.917335 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63456128}
W20260709 13:05:03.268683 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.949147 13871 rpcz_store.cc:276] 0709 13:05:01.758224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.758339 (+   115us) service_pool.cc:224] Handling call
0709 13:05:02.919365 (+1161026us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b54002e81d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:02.919606 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55187712}
W20260709 13:05:03.144549 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.023025 14024 rpcz_store.cc:276] 0709 13:05:01.789139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.442709 (+653570us) service_pool.cc:224] Handling call
0709 13:05:02.482005 (+ 39296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.021921 (+539916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.022354 (+   433us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.219054 14023 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:03.028250 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:03.079531 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.078347 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2189) took 1787 ms. Trace:
W20260709 13:05:03.200975 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.257352 14556 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:03.273444 16278 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:03.274364 16278 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:03.277271 16278 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:03.073335 13868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.005s	sys 0.004s
I20260709 13:05:03.246134 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1465) took 1415 ms. Trace:
I20260709 13:05:03.119848 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1446) took 1356 ms. Trace:
W20260709 13:05:03.229205 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 90357.5us
W20260709 13:05:03.183802 14522 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.66:43953 timed out after 2.074s (SENT) [suppressed 30 similar messages]
I20260709 13:05:03.279484 16447 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:03.289635 13870 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 3: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:03.099010 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.073838 13866 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
W20260709 13:05:03.278614 13868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.018s	sys 0.006s
I20260709 13:05:03.055145 13867 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:05:03.078631 14022 raft_consensus.cc:2376] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 3. Re-sending same reply.
I20260709 13:05:03.278998 14019 rpcz_store.cc:276] 0709 13:05:01.789428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.662453 (+873025us) service_pool.cc:224] Handling call
0709 13:05:02.720377 (+ 57924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.024331 (+303954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.204995 (+180664us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.117204 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:03.281524 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 130838us
I20260709 13:05:03.282449 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.284140 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:03.264806 13838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.925s	user 0.011s	sys 0.006s
I20260709 13:05:03.145097 13875 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:05:03.055529 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.279273 14016 rpcz_store.cc:276] 0709 13:05:01.763676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.246531 (+482855us) service_pool.cc:224] Handling call
0709 13:05:03.119314 (+872783us) spinlock_profiling.cc:243] Waited 65 ms on lock 0x7b54002995d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:03.119558 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149511040}
W20260709 13:05:03.293879 14023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1441) took 1766 ms (client timeout 1805 ms). Trace:
I20260709 13:05:03.292367 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2147) took 1036 ms. Trace:
I20260709 13:05:03.272006 14015 rpcz_store.cc:276] 0709 13:05:01.291584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.373323 (+1081739us) service_pool.cc:224] Handling call
0709 13:05:02.383854 (+ 10531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.930886 (+547032us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b54002bafa0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33aadc0 00007fb2c34351c0 00007fb2c3436231 00007fb2c3432d8f 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c
0709 13:05:02.953954 (+ 23068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.078230 (+124276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64610688}
W20260709 13:05:03.296981 14023 rpcz_store.cc:269] 0709 13:05:01.505465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.935991 (+1430526us) service_pool.cc:224] Handling call
0709 13:05:03.271639 (+335648us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:03.298444 13838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.040s	user 0.013s	sys 0.007s
I20260709 13:05:03.299769 14560 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:03.242545 16418 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:03.162997 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.297585 13866 rpcz_store.cc:276] 0709 13:05:02.255679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.946169 (+690490us) service_pool.cc:224] Handling call
0709 13:05:03.292235 (+346066us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.238616 16428 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:03.027371 16484 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:03.223068 14021 rpcz_store.cc:276] 0709 13:05:01.503917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.396457 (+892540us) service_pool.cc:224] Handling call
0709 13:05:02.396808 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:02.404375 (+  7567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.644020 (+239645us) spinlock_profiling.cc:243] Waited 6.9 ms on lock 0x7b54002ea754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:02.644316 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15867776}
I20260709 13:05:03.304695 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2150) took 1025 ms. Trace:
W20260709 13:05:03.309808 14022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1439) took 1805 ms (client timeout 1816 ms). Trace:
W20260709 13:05:03.312197 13840 connection.cc:256] RPC call timeout handler was delayed by -1.6143237879999788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
I20260709 13:05:03.324793 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:03.324558 14022 rpcz_store.cc:269] 0709 13:05:01.504885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.544456 (+1039571us) service_pool.cc:224] Handling call
0709 13:05:03.301944 (+757488us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.323804 13870 rpcz_store.cc:276] 0709 13:05:02.279674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.452466 (+172792us) service_pool.cc:224] Handling call
0709 13:05:03.298568 (+846102us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:03.343572 13840 connection.cc:256] RPC call timeout handler was delayed by -1.599023787999954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1894s for a response
W20260709 13:05:03.339932 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 55991us
W20260709 13:05:03.344115 13840 connection.cc:256] RPC call timeout handler was delayed by -1.5849182780000319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17070000000000002s for a response
I20260709 13:05:03.344923 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.314465 16418 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:03.348994 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 76162.6us
I20260709 13:05:02.975186 16498 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:03.298723 16082 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:03.354311 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:03.340314 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 1.08187e+06us
I20260709 13:05:03.357291 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.351383 16418 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:03.368208 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:03.370790 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.141731 13771 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 4 to 5, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 5 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:03.371794 13987 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:03.368911 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.372990 13874 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:03.377332 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2187) took 2378 ms. Trace:
I20260709 13:05:03.377599 14020 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:03.377449 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.372572 16359 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:03.382184 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2140) took 1148 ms. Trace:
W20260709 13:05:03.382256 13840 connection.cc:256] RPC call timeout handler was delayed by -1.4469237879999355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
W20260709 13:05:03.383865 13840 connection.cc:256] RPC call timeout handler was delayed by -1.4406237879999253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.207s for a response
W20260709 13:05:03.384091 13840 connection.cc:256] RPC call timeout handler was delayed by -1.4154237879999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
I20260709 13:05:03.386986 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.377965 14018 rpcz_store.cc:276] 0709 13:05:00.996330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.338783 (+1342453us) service_pool.cc:224] Handling call
0709 13:05:02.339261 (+   478us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:02.366835 (+ 27574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.366884 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:02.373075 (+  6191us) log.cc:844] Serialized 78 byte log entry
0709 13:05:02.717748 (+344673us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:03.293356 (+575608us) log.cc:844] Serialized 14 byte log entry
0709 13:05:03.363806 (+ 70450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.374725 (+ 10919us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.374803 (+    78us) raft_consensus.cc:1685] finished
0709 13:05:03.374814 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.375094 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43820,"prepare.queue_time_us":255388,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":80,"raft.queue_time_us":45408,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":6799,"thread_start_us":528864,"threads_started":2,"wal-append.queue_time_us":462864}
I20260709 13:05:03.388942 13838 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:03.391357 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:03.391503 16514 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:03.392033 16514 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.393754 16514 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:03.394047 16467 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:03.395152 13869 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:03.395826 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2164) took 1029 ms. Trace:
I20260709 13:05:03.396160 13869 rpcz_store.cc:276] 0709 13:05:02.374928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.295617 (+920689us) service_pool.cc:224] Handling call
0709 13:05:03.395724 (+100107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.396667 16467 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:03.397392 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:03.397480 16316 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.398237 16467 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:03.400318 16209 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:03.400106 16316 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:03.401105 16209 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:03.401458 16467 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:03.402457 16209 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:03.385468 16359 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:03.405784 16359 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:03.409252 13950 raft_consensus.cc:2410] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:05:03.409598 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2652536240000245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
I20260709 13:05:03.404234 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:03.411481 13988 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:03.411511 16076 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:03.412252 16076 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.412582 16515 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:03.412683 16076 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:03.413266 16515 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.412832 16428 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:03.413758 16515 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:03.382635 13868 rpcz_store.cc:276] 0709 13:05:02.233661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.233808 (+   147us) service_pool.cc:224] Handling call
0709 13:05:02.250012 (+ 16204us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:02.254750 (+  4738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.254783 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:02.256676 (+  1893us) log.cc:844] Serialized 26 byte log entry
0709 13:05:03.372744 (+1116068us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:03.377570 (+  4826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.377684 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.381789 (+  4105us) raft_consensus.cc:1685] finished
0709 13:05:03.381807 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.382088 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4309,"raft.queue_time_us":7025,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":45978,"thread_start_us":961579,"threads_started":1,"wal-append.queue_time_us":1025205}
I20260709 13:05:03.415520 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.415789 16428 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:03.416157 13868 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:03.416704 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2162) took 1051 ms. Trace:
I20260709 13:05:03.417042 13868 rpcz_store.cc:276] 0709 13:05:02.365584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.415143 (+1049559us) service_pool.cc:224] Handling call
0709 13:05:03.416616 (+  1473us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:03.418118 13840 connection.cc:256] RPC call timeout handler was delayed by -1.0722640069999443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 13:05:03.418545 13840 connection.cc:256] RPC call timeout handler was delayed by -1.069230270999924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
I20260709 13:05:03.422204 16379 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.423053 16379 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:03.424376 16498 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:03.399140 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48808.7us
I20260709 13:05:03.427172 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:03.428524 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:03.430023 13872 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:03.430882 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 348) took 1046 ms. Trace:
I20260709 13:05:03.430568 16417 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:03.431361 13872 rpcz_store.cc:276] 0709 13:05:02.384692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.317605 (+932913us) service_pool.cc:224] Handling call
0709 13:05:03.430796 (+113191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.393193 14560 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:03.433621 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.429740 16325 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:03.380033 13915 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.602s (SENT)
I20260709 13:05:03.413826 13988 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:03.443735 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.444069 16343 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:03.444597 16343 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.444957 16343 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:03.445446 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.446234 13869 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:05:03.447033 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:03.448006 16440 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:03.448469 16440 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.448167 16515 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:03.448908 16440 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:03.450958 16375 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:03.453323 13870 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:03.453962 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2156) took 1090 ms. Trace:
I20260709 13:05:03.454566 13870 rpcz_store.cc:276] 0709 13:05:02.363998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.357902 (+993904us) service_pool.cc:224] Handling call
0709 13:05:03.453871 (+ 95969us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.459195 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:03.464946 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2160) took 1096 ms. Trace:
I20260709 13:05:03.465477 13868 rpcz_store.cc:276] 0709 13:05:02.365058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.417768 (+1052710us) service_pool.cc:224] Handling call
0709 13:05:03.461033 (+ 43265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.437052 16325 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:03.480372 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.481129 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.485054 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 73575.3us
I20260709 13:05:03.478439 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:03.488080 13869 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:03.488677 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2158) took 1124 ms. Trace:
I20260709 13:05:03.489019 13869 rpcz_store.cc:276] 0709 13:05:02.364538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.396916 (+1032378us) service_pool.cc:224] Handling call
0709 13:05:03.488614 (+ 91698us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.464174 16419 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:03.490728 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:03.494311 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 51404.2us
I20260709 13:05:03.472661 16325 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:03.492656 13838 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:03.501176 16426 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.501410 16498 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:03.505997 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.507673 14561 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:03.510445 16436 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:03.511020 16529 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:05:03.511260 16436 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:03.514720 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.515558 14021 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:03.516628 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.502003 16426 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:03.518836 16082 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:03.520503 16426 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:03.521413 16436 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:03.521538 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.523567 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.524516 16379 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:03.525126 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.525759 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.526118 13945 raft_consensus.cc:2393] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:03.526491 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.526872 13949 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:03.496507 16417 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:03.527730 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.528095 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.528611 13946 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:05:03.528820 13948 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 2.
I20260709 13:05:03.496721 16278 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.530539 16278 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:03.532840 16278 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:03.533200 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 241023us
I20260709 13:05:03.534507 16076 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:03.535312 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2142) took 1300 ms. Trace:
I20260709 13:05:03.536890 16076 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:03.535707 13873 rpcz_store.cc:276] 0709 13:05:02.234458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.235018 (+   560us) service_pool.cc:224] Handling call
0709 13:05:02.960740 (+725722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.966244 (+  5504us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:03.198580 (+232336us) log.cc:844] Serialized 14 byte log entry
0709 13:05:03.350557 (+151977us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:03.350615 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:03.350663 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.361478 (+ 10815us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.533056 (+171578us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b180023a160. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4
0709 13:05:03.534952 (+  1896us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b54000da050. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:03.535107 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1725,"prepare.queue_time_us":382422,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":6026,"raft.run_cpu_time_us":494,"raft.run_wall_time_us":7106,"spinlock_wait_cycles":1641673856,"thread_start_us":200154,"threads_started":2,"wal-append.queue_time_us":127408}
W20260709 13:05:03.538996 13840 leader_election.cc:336] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.569s (SENT)
I20260709 13:05:03.539273 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.539559 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:03.540158 13874 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:03.541092 13872 raft_consensus.cc:2410] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:03.541623 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2159) took 1176 ms. Trace:
I20260709 13:05:03.541980 13872 rpcz_store.cc:276] 0709 13:05:02.364782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.436982 (+1072200us) service_pool.cc:224] Handling call
0709 13:05:03.541555 (+104573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.524534 13950 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 5: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:05:03.555181 13915 leader_election.cc:336] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.669s (SENT)
I20260709 13:05:03.477252 14553 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:03.491809 16209 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:03.560442 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:03.563820 13839 leader_election.cc:336] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.526s (SENT)
I20260709 13:05:03.561460 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:03.558034 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2151) took 1242 ms. Trace:
W20260709 13:05:03.569218 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 92824.3us
I20260709 13:05:03.569823 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.566140 13874 rpcz_store.cc:276] 0709 13:05:02.305446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.541233 (+1235787us) service_pool.cc:224] Handling call
0709 13:05:03.541698 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.542030 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:03.547434 (+  5404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:03.547483 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:03.547524 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.547581 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.547689 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.572136 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.567381 16467 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:03.580545 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.581408 14555 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:05:03.582167 13948 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:05:03.583043 14556 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:03.584445 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:03.585078 13941 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:03.586416 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.586654 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.587478 13943 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:03.587713 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1440) took 2082 ms. Trace:
I20260709 13:05:03.587862 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:03.588500 13944 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:03.588423 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.589169 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.589402 13943 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:03.590087 13947 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:03.590471 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.590971 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.591717 13947 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:03.591073 13950 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:03.593149 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.544866 16316 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:03.593708 13950 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
W20260709 13:05:03.594205 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 180483us
I20260709 13:05:03.588212 14020 rpcz_store.cc:276] 0709 13:05:01.505180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.405053 (+899873us) service_pool.cc:224] Handling call
0709 13:05:02.405311 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.398130 (+992819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.587373 (+189243us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b54000f23d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:03.587635 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2223584128}
I20260709 13:05:03.604574 13946 raft_consensus.cc:2410] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:03.564044 16515 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:03.604348 13914 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:03.610260 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.613518 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:03.614647 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.615319 13950 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:03.615610 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2146) took 1359 ms. Trace:
I20260709 13:05:03.615931 13874 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:03.616060 13869 rpcz_store.cc:276] 0709 13:05:02.248888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.497377 (+1248489us) service_pool.cc:224] Handling call
0709 13:05:03.557752 (+ 60375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.601582 (+ 43830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:03.601615 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:03.607540 (+  5925us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:03.607621 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.607716 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.607870 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.613577 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2144) took 1367 ms. Trace:
I20260709 13:05:03.622530 13870 rpcz_store.cc:276] 0709 13:05:02.240469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.482428 (+1241959us) service_pool.cc:224] Handling call
0709 13:05:03.574865 (+ 92437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.607909 (+ 33044us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:03.607936 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:03.607975 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:03.608016 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.608068 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.608163 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.625020 14560 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:03.627076 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.607133 16359 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:03.639746 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.637513 13947 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:03.618568 16498 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:05:03.648895 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.635584 16359 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.650842 13870 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 6. Ops: []
I20260709 13:05:03.651829 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2161) took 1286 ms. Trace:
I20260709 13:05:03.653164 13873 rpcz_store.cc:276] 0709 13:05:02.365326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.558768 (+1193442us) service_pool.cc:224] Handling call
0709 13:05:03.629502 (+ 70734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.651529 (+ 22027us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:03.651560 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:03.651608 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:03.651633 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.651680 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.651780 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:03.655578 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 116656us
W20260709 13:05:03.662314 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 128307us
W20260709 13:05:03.663148 13915 leader_election.cc:336] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.806s (SENT)
I20260709 13:05:03.668222 16209 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:03.669173 13989 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:03.670044 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 349) took 1270 ms. Trace:
I20260709 13:05:03.670475 13870 rpcz_store.cc:276] 0709 13:05:02.384972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.623503 (+1238531us) service_pool.cc:224] Handling call
0709 13:05:03.642922 (+ 19419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.654944 (+ 12022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.655170 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:03.674861 16532 log.cc:927] Time spent T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:05:03.683449 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:03.684329 13866 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:03.686916 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:03.687779 13875 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:03.688323 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2165) took 1256 ms. Trace:
I20260709 13:05:03.688612 13875 rpcz_store.cc:276] 0709 13:05:02.432503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.686528 (+1254025us) service_pool.cc:224] Handling call
0709 13:05:03.688259 (+  1731us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.690215 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.693281 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.654744 16359 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:03.694856 16467 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.695465 16467 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:03.698477 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:03.699749 16298 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:03.699970 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.700215 16298 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.700685 16298 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:03.700740 13948 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:03.634861 16436 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:05:03.319832 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:03.703430 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.703577 13838 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:03.721460 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:03.683449 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.702685 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 345) took 1335 ms. Trace:
I20260709 13:05:03.728602 13869 rpcz_store.cc:276] 0709 13:05:02.367490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.616967 (+1249477us) service_pool.cc:224] Handling call
0709 13:05:03.648258 (+ 31291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.702291 (+ 54033us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:03.702325 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:03.702393 (+    68us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:03.702426 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.702481 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.702612 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.716244 13946 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
W20260709 13:05:03.732014 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.710598 16298 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:03.713908 13771 catalog_manager.cc:5697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:03.734452 16298 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:03.710204 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.714241 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:03.735051 13771 catalog_manager.cc:5697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:03.681546 16440 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:03.736202 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 84484.2us
I20260709 13:05:03.736375 16428 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:03.736809 16428 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.701897 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:03.738884 13948 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:03.739173 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2157) took 1374 ms. Trace:
W20260709 13:05:03.719663 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.735702 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:03.740111 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.737288 16428 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:03.742997 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:03.740278 13946 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:03.749457 16515 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.750250 16515 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:03.752977 16515 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:03.753765 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.755247 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.756006 13943 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:03.756934 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:03.758493 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 55076.7us
I20260709 13:05:03.706535 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:03.762511 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.762727 13950 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:03.733740 13941 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:05:03.763307 13947 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:05:03.778671 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.779474 13950 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:03.737112 13771 catalog_manager.cc:5697] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 2 to 4, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 4 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:03.764287 13840 leader_election.cc:336] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.591s (SENT)
I20260709 13:05:03.782411 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:03.783895 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 45005.2us
I20260709 13:05:03.786744 13771 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 4 to 5, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 5 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:03.760650 13949 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:03.740841 13872 rpcz_store.cc:276] 0709 13:05:02.364277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.542692 (+1178415us) service_pool.cc:224] Handling call
0709 13:05:03.543092 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.738814 (+195722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:03.738847 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:03.738893 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:03.738933 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.739000 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.739115 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46078,"raft.queue_time_us":92877,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":9126,"thread_start_us":195244,"threads_started":1}
I20260709 13:05:03.766014 14020 raft_consensus.cc:2376] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 7. Re-sending same reply.
I20260709 13:05:03.773003 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:03.794435 13949 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:03.697932 16467 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:03.795753 16467 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:03.796036 13949 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:03.796514 16467 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:03.796943 16467 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:03.797291 16467 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:03.704698 16379 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:03.669543 16514 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:03.702682 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:03.798336 16379 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.789112 13771 catalog_manager.cc:5697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:03.800714 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 58272.4us
I20260709 13:05:03.802433 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.800340 16278 raft_consensus.cc:2764] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:05:03.805958 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 84644us
I20260709 13:05:03.806384 16515 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:03.809583 13988 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:05:03.811590 13772 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57626 (request call id 245) took 14233 ms (client timeout 15000 ms). Trace:
I20260709 13:05:03.798838 16379 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:03.814581 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:03.815047 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:03.815382 14023 raft_consensus.cc:2376] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 4. Re-sending same reply.
I20260709 13:05:03.817413 16417 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:03.813756 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:03.817512 16535 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:03.701843 16418 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:03.825624 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:03.827425 16375 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:03.827561 16514 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:03.828157 16440 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:03.775178 16298 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:03.828902 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:03.831357 13840 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.248s (SENT)
W20260709 13:05:03.833523 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.836344 13944 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:03.838579 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.842216 16515 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:05:03.842482 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 136036us
I20260709 13:05:03.838611 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.813479 16343 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:03.818037 16417 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.847932 13946 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:03.858672 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:03.845906 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 181568us
W20260709 13:05:03.812083 13772 rpcz_store.cc:269] 0709 13:04:49.577939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.578491 (+   552us) service_pool.cc:224] Handling call
0709 13:04:52.163405 (+2584914us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b540012acd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339d090 00007fb2c343bf3a 00007fb2c91f21cc 00007fb2c92f037e 00007fb2c77b12a4 00007fb2c77b1232 00007fb2c77b1161 00007fb2c77b10dc 00007fb2c77b0392 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:03.811286 (+11647881us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:57.713276 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:57.862987 (+149711us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:57.877307 (+ 14320us) write_op.cc:435] Acquired schema lock
0709 13:04:57.883363 (+  6056us) tablet.cc:662] Decoding operations
0709 13:04:58.261591 (+378228us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:58.264061 (+  2470us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:58.267757 (+  3696us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:04:58.370711 (+102954us) tablet.cc:701] Row locks acquired
0709 13:04:58.398786 (+ 28075us) write_op.cc:260] PREPARE: finished
0709 13:04:58.399018 (+   232us) write_op.cc:270] Start()
0709 13:04:58.399204 (+   186us) write_op.cc:276] Timestamp: P: 1783602298398941 usec, L: 0
0709 13:04:58.399226 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:04:58.400060 (+   834us) log.cc:844] Serialized 4528 byte log entry
0709 13:04:58.956540 (+556480us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:04:59.739553 (+783013us) op_driver.cc:464] REPLICATION: finished
0709 13:04:59.877472 (+137919us) write_op.cc:301] APPLY: starting
0709 13:04:59.877606 (+   134us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:59.877897 (+   291us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:59.877940 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:59.925464 (+ 47524us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:59.939897 (+ 14433us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:04:59.939986 (+    89us) write_op.cc:312] APPLY: finished
0709 13:04:59.940638 (+   652us) log.cc:844] Serialized 83 byte log entry
0709 13:04:59.940899 (+   261us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:59.941102 (+   203us) write_op.cc:454] Released schema lock
0709 13:04:59.941215 (+   113us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:59.941803 (+   588us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:59.941844 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":506001,"reported_tablets":33,"spinlock_wait_cycles":491833088,"child_traces":[["op",{"apply.queue_time_us":109682,"apply.run_cpu_time_us":4964,"apply.run_wall_time_us":74225,"num_ops":11,"prepare.queue_time_us":447583,"prepare.run_cpu_time_us":50472,"prepare.run_wall_time_us":1081448,"raft.queue_time_us":367582,"raft.run_cpu_time_us":6324,"raft.run_wall_time_us":66950,"replication_time_us":1340125,"spinlock_wait_cycles":11373440,"thread_start_us":853245,"threads_started":4,"wal-append.queue_time_us":401614,"wal-append.run_cpu_time_us":77111,"wal-append.run_wall_time_us":2124245}]]}
W20260709 13:05:03.855715 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 96411.1us
I20260709 13:05:03.861609 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.852016 16298 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:03.865031 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:03.862151 14022 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:03.867745 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:03.872596 16536 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:03.873126 16536 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:03.873616 16536 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:03.868856 16298 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:03.876071 14016 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:03.870409 16447 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:03.877440 16447 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:03.877810 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:03.872299 13838 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:03.861882 16417 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:03.879571 16375 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:03.880045 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.885795 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.889964 13948 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:03.890244 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:03.890794 13945 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:03.868662 16209 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:03.892508 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:03.891979 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.877853 16447 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:03.905038 14016 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
W20260709 13:05:03.909085 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 99573.3us
I20260709 13:05:03.909688 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.909929 13866 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:03.910714 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2155) took 1547 ms. Trace:
I20260709 13:05:03.911165 13866 rpcz_store.cc:276] 0709 13:05:02.363713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.683128 (+1319415us) service_pool.cc:224] Handling call
0709 13:05:03.910633 (+227505us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:03.927512 13989 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:03.895517 16514 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:03.931583 16436 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:03.934149 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:05:03.922708 13839 leader_election.cc:336] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.798s (SENT)
I20260709 13:05:03.937522 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:03.939698 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 115945us
I20260709 13:05:03.907755 16515 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:03.959697 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.963335 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.936676 16514 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:03.969671 16534 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:03.970278 16534 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:03.970440 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:03.970752 16534 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:03.961218 16498 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:03.974409 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:03.975244 14022 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:03.978997 16375 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:03.956810 16530 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:03.949187 16076 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:03.962775 16440 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:03.984500 16440 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:03.984886 16440 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:03.994913 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.995731 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.996414 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.998229 16547 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:03.998798 16547 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:03.999188 16547 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:04.002002 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:04.002889 14015 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:04.016436 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:04.023130 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 96678.9us
I20260709 13:05:04.021733 16515 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:03.993247 16514 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.002918 16418 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:04.031481 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 163766us
W20260709 13:05:04.032251 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 98285.8us
I20260709 13:05:04.033797 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:04.028700 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:04.035518 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:03.991127 16325 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.049031 16325 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:04.050228 16549 log.cc:927] Time spent T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:05:04.050980 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:04.051548 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.052078 14016 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:04.052469 13943 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:04.028698 16359 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.034513 16379 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:04.054922 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.055415 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.055908 16359 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:04.056217 16379 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:04.056795 16535 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:04.057121 13874 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:04.057358 16535 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:04.056825 14023 raft_consensus.cc:2368] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4.
I20260709 13:05:04.058352 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:04.059268 14015 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:04.063076 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:04.065202 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:03.952139 16436 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:04.039211 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 68860.2us
I20260709 13:05:04.042764 14022 raft_consensus.cc:2376] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 4. Re-sending same reply.
I20260709 13:05:04.052266 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:04.038112 13838 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:04.080106 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48552.6us
W20260709 13:05:04.080334 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.086148 13838 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:04.085835 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:04.099604 13866 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:04.099738 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:04.104303 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:04.116020 13869 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:04.119478 16447 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.122311 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:04.123296 14016 raft_consensus.cc:2368] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7.
I20260709 13:05:04.125355 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:04.126389 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.083314 16316 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:04.079525 16278 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:04.087802 16375 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:05:04.130973 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 49412.7us
I20260709 13:05:04.129441 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:04.132426 14022 raft_consensus.cc:2376] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 4. Re-sending same reply.
W20260709 13:05:04.136756 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 58791.6us
I20260709 13:05:04.111738 16343 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:04.155079 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:04.156366 13945 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:04.154565 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.150297 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:04.161882 14015 raft_consensus.cc:2368] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5.
I20260709 13:05:04.165205 13989 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:04.137531 16447 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:04.167717 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:04.168774 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:04.160422 16375 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.174476 16375 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.085781 16440 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.076205 16534 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.178364 14023 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:05:04.180747 14023 pending_rounds.cc:85] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 5
W20260709 13:05:04.177574 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 103257us
I20260709 13:05:04.181327 14018 raft_consensus.cc:2368] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 3. Current term is 4.
I20260709 13:05:04.182266 14017 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:05:04.186399 16379 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:04.187012 16379 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.187117 16436 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:04.187515 16379 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:04.188223 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:04.188880 13948 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
W20260709 13:05:04.185325 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58272.1us
I20260709 13:05:04.182642 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:04.192364 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.192539 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:04.195509 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.199865 14016 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:04.181031 14023 pending_rounds.cc:107] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:05:04.201715 14023 raft_consensus.cc:2889] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:04.202762 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 137645us
I20260709 13:05:04.206852 13838 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:04.137959 16515 raft_consensus.cc:2764] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:05:04.209506 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:04.216403 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.189647 16538 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:05:04.207682 16550 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:05:04.223598 13948 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:05:04.219966 14019 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:04.189610 16298 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.240280 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.242403 13915 leader_election.cc:400] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6.
I20260709 13:05:04.244506 14022 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:04.236387 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 135709us
I20260709 13:05:04.246333 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.243145 16417 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.253201 16417 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:04.243088 13915 leader_election.cc:403] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:04.225165 16538 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.257015 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:04.257578 13950 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:04.259680 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:04.260766 14552 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:04.261413 13941 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:05:04.272794 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:05:04.275904 16554 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:05:04.276544 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:04.277297 14551 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:04.278789 14561 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:05:04.279460 16343 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:04.275789 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:04.283004 16440 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:04.283542 16534 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:04.283766 16440 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:04.284226 16440 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:04.284675 16440 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.287621 16298 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:04.288847 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:04.290603 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:04.290882 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 48889.8us
I20260709 13:05:04.275722 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:04.291672 14559 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:05:04.291213 16547 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.291770 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:04.292881 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:04.294263 16278 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6.
I20260709 13:05:04.294395 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:04.233660 16467 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.295091 14558 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:04.295168 16278 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:04.296222 16278 consensus_queue.cc:1059] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:04.296729 16467 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:04.294291 16498 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:04.297544 16436 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:04.298367 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.299139 13948 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:05:04.300901 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 55848us
I20260709 13:05:04.298291 16559 consensus_queue.cc:1059] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:05:04.304804 16359 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:04.262777 16538 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.307514 13950 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:05:04.238409 16428 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.293864 16547 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:04.323962 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:04.356521 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:04.342885 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 111299us
I20260709 13:05:04.354689 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.364584 13941 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.366590 13948 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:05:04.367175 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 69560.9us
W20260709 13:05:04.354162 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 63643us
I20260709 13:05:04.304563 16418 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:04.345722 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.384531 13950 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:04.367913 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 183407us
I20260709 13:05:04.366182 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.361006 14554 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:04.348488 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.386014 13949 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:04.384372 14558 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:04.387367 13945 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:04.389484 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:04.391614 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:04.394594 13869 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:04.394922 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:04.395756 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:04.396469 13866 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.399487 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:04.401674 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 45257us
I20260709 13:05:04.404305 14524 leader_election.cc:400] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7.
I20260709 13:05:04.404848 14524 leader_election.cc:403] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:05:04.351096 16379 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.324723 16436 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:05:04.402415 16359 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:04.426158 16379 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:04.434885 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 49550.4us
I20260709 13:05:04.436368 13838 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:04.437237 16572 raft_consensus.cc:3055] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:04.437758 16572 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:04.439244 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.444212 13950 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:04.444125 14559 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
W20260709 13:05:04.446923 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 57591.8us
I20260709 13:05:04.403223 16498 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:04.451887 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:04.449456 13941 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:04.413790 16418 raft_consensus.cc:697] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:04.472234 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:04.475250 13866 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
W20260709 13:05:04.478014 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 83221.2us
I20260709 13:05:04.444708 16316 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:04.497511 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 102301us
W20260709 13:05:04.500062 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.501668 13839 leader_election.cc:336] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.102s (SENT)
W20260709 13:05:04.503715 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 81679us
I20260709 13:05:04.465387 16565 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7.
I20260709 13:05:04.471864 16515 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:05:04.511620 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 110850us
I20260709 13:05:04.450860 16298 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:05:04.519503 16343 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:04.517985 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:04.477500 16547 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:04.499001 16535 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.526800 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:04.520190 16515 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:04.528390 16573 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:04.528231 16418 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.529088 16567 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:04.529456 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:04.516696 16076 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:04.534514 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:04.477378 16566 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:05:04.529695 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:04.523622 16436 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:04.547602 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.548306 13941 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.550356 16316 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.551410 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:04.552240 14561 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:04.535969 16573 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:04.502933 16572 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.554747 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:04.563980 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:04.565042 14559 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:04.536334 16428 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:04.543550 16076 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.587437 13949 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:04.579507 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:04.574190 16573 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.582767 16574 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:04.582424 16515 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.584143 16572 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:04.554309 16419 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:05:04.589192 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 54739.3us
I20260709 13:05:04.544579 16298 consensus_queue.cc:1243] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:04.586529 16559 raft_consensus.cc:2764] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:05:04.591733 16076 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.572700 16343 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:04.599865 14559 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:04.546290 16436 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:04.602365 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 55152.4us
I20260709 13:05:04.611059 13839 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:04.593997 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:04.554028 16538 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:04.605286 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 143092us
I20260709 13:05:04.609134 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:04.592264 16574 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:04.619715 13866 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:04.611909 14561 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:04.606411 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:04.547030 16498 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.631649 16440 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:04.630255 16536 consensus_meta.cc:287] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.756s	user 0.008s	sys 0.006s
W20260709 13:05:04.599869 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 160113us
I20260709 13:05:04.615886 13941 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:04.637468 14522 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:04.611744 16436 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:04.617175 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:04.651939 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.653030 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.653853 13948 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:04.641867 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:04.617105 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:04.655157 13874 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:04.601018 16419 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:04.613606 14558 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:04.620513 16343 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:04.623494 16580 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:05:04.624349 16582 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:04.625751 16576 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:05:04.665910 16592 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:04.667517 16592 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:04.668520 16419 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.672079 16538 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:04.640755 16440 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:04.674584 13942 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:04.674966 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:04.630224 16573 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.630722 16584 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:05:04.679996 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.640542 16535 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:04.655217 13945 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:04.649777 16417 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:04.657274 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.657680 16589 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:04.689419 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.690275 13945 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.658603 13943 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:04.660261 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.660784 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:04.661984 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.661851 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.664330 16343 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.626176 16577 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:05:04.665941 16582 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:04.703307 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.628576 14551 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:04.592648 16547 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:04.635555 16536 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.692780 13950 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:05:04.680698 16514 consensus_meta.cc:287] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.606s	user 0.013s	sys 0.001s
W20260709 13:05:04.675081 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58095.3us
I20260709 13:05:04.663148 16375 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.704793 13941 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:05:04.709369 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.689419 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.729583 16598 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:05:04.716827 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.706267 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 97207.3us
I20260709 13:05:04.731892 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.729754 16588 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:04.735059 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:04.658710 16579 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:05:04.745667 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.746743 13948 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.752677 16600 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:04.703290 16417 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:05:04.757164 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48379.4us
I20260709 13:05:04.762677 14560 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:04.759295 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:04.768157 16577 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:04.768273 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:04.768770 16577 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:04.769945 13838 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:04.769709 16577 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.770879 16601 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:04.771108 16577 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.771339 16601 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.771689 16586 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:04.772624 16535 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:04.771730 16601 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:04.773254 16535 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.773736 16535 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:04.774523 16440 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:04.775508 16582 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:04.776115 16375 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:04.776087 16582 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:04.688601 16574 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.777558 16574 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:04.778681 16076 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.780972 16076 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:04.782797 13950 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:04.783985 16601 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.743556 16592 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:05:04.808483 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 284147us
W20260709 13:05:04.818245 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 52505.1us
W20260709 13:05:04.792498 16498 consensus_queue.cc:1175] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e1649107c781428692458f10597c5742" responder_term: 4 status { last_received { term: 4 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 4 index: 3 } }
I20260709 13:05:04.821364 16579 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:04.822928 16547 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.761853 16598 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:04.825068 16598 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.765702 16538 consensus_queue.cc:1059] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:05:04.820948 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:04.833866 13945 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
W20260709 13:05:04.834483 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 69737.8us
W20260709 13:05:04.836181 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 223634us
I20260709 13:05:04.718640 16514 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.841475 14560 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:04.828331 16538 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:04.840790 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:04.745399 16467 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.084s
W20260709 13:05:04.855473 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 180566us
I20260709 13:05:04.866530 13948 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:04.867314 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:04.842517 16278 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:04.847573 16325 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:05:04.868624 16278 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:04.880731 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 59883.8us
I20260709 13:05:04.882198 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:04.882875 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:04.884694 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:04.886718 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.885903 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:04.880441 16278 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:04.847925 16538 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:04.893591 13989 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:04.894475 16076 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:04.903527 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:04.894411 16588 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:04.894364 16579 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:04.905119 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:04.916738 13839 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:04.919627 16076 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:04.920845 16076 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.921316 16584 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:04.922104 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:04.922933 13947 raft_consensus.cc:2393] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:04.922746 16584 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:04.924090 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.925002 13950 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:04.930338 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:04.901062 16604 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:04.942361 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:04.943048 13944 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:05:04.945457 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 52008.6us
I20260709 13:05:04.906776 16589 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:04.954962 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.933684 16418 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:04.906107 16565 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:05:04.937211 16592 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:04.975973 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.963904 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 75370.6us
I20260709 13:05:04.928689 16601 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:04.944988 16588 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:05.010550 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:04.943571 16467 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:05:04.944785 16566 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:05:04.956810 16562 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:05.016304 16603 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:05.016819 16602 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:05.017258 16564 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:04.914018 16417 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:04.946261 16547 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
W20260709 13:05:05.020599 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55410.9us
I20260709 13:05:04.965165 16604 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:04.967865 16379 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.969728 16592 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:05.024245 13944 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
W20260709 13:05:05.042547 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.047468 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:05.060981 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 60634.7us
I20260709 13:05:05.067513 16436 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:05.070114 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.071974 13949 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:05:05.017657 16594 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.136s
I20260709 13:05:05.035333 16602 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:05.003652 16538 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.048733 16514 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:05.083402 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:05.058480 16536 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:05.092144 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.054280 16564 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:05.094686 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:05.095391 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:05.096035 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:05.069976 16436 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:05.136134 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:05.120092 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 72754.9us
W20260709 13:05:05.126868 16596 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
I20260709 13:05:05.082732 16552 consensus_queue.cc:1059] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.191s
I20260709 13:05:05.059437 16592 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:05.079107 16316 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:05.130129 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:05.084497 16588 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.136088 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:05.077620 16379 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.096774 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:05.138512 14017 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:05.083432 16571 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:05.085264 16515 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:05.159384 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:05.165759 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:05.166970 16575 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.639s	user 0.007s	sys 0.000s
I20260709 13:05:05.167588 16617 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:05.168833 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:05.172256 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:05.175848 13944 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:05:05.093967 16566 consensus_queue.cc:1059] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:05:05.189098 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:05.160765 16316 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:05.034433 16589 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:05.161896 16538 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:05.198638 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:05.199512 14016 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:05.203147 16615 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:05:05.206174 16467 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:05.206574 16298 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:05.207067 16298 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:05.207775 16602 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.178334 16571 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:05.208007 16298 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.209923 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:05.212556 14021 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:05.184300 16436 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:05.208283 13944 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
W20260709 13:05:05.214684 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.196894 16316 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:05.229328 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:05.230198 13946 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:05.210650 16601 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:05.216338 16571 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:05.219738 16573 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.000s
I20260709 13:05:05.217833 16467 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:05.226665 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:05.238166 16608 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
W20260709 13:05:05.239180 16607 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:05:05.239768 16419 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:05.239895 16601 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:05.246311 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 116296us
W20260709 13:05:05.234524 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44458.1us
I20260709 13:05:05.255029 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:05.255904 13945 raft_consensus.cc:2410] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:05.250171 16565 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
W20260709 13:05:05.258268 16616 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.173s	user 0.002s	sys 0.000s
I20260709 13:05:05.260915 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:05.259418 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 163017us
I20260709 13:05:05.262035 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:05.229678 14015 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:05.261560 14016 raft_consensus.cc:2376] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 5. Re-sending same reply.
I20260709 13:05:05.267765 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:05.187049 16564 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:05.193358 16604 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:05.259338 16572 consensus_meta.cc:287] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.660s	user 0.012s	sys 0.002s
I20260709 13:05:05.216459 16617 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.267879 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:05.269273 16603 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:05.296531 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:05.297470 16594 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:05.298045 16594 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:05.299077 16594 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.299242 16601 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.301234 16547 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:05.307520 14021 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:05.354558 14560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 514) took 1087 ms. Trace:
I20260709 13:05:05.354184 16278 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.355839 13948 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:05.358390 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:05.358402 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:05.359786 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 51545.7us
W20260709 13:05:05.358280 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 60395.6us
W20260709 13:05:05.358291 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 90634.5us
W20260709 13:05:05.364809 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 154875us
I20260709 13:05:05.365907 16584 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:05.369067 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:05.370297 16571 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.372359 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:05.372758 16571 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:05.378409 13941 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:05.382709 16467 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.383776 14019 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:05:05.308780 16562 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:05.393625 13874 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:05:05.355002 14560 rpcz_store.cc:276] 0709 13:05:04.267454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.272425 (+  4971us) service_pool.cc:224] Handling call
0709 13:05:05.354321 (+1081896us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54004f1854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:05.354455 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1124168576}
I20260709 13:05:05.398823 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:05.272352 16535 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.408887 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:05.409859 14558 raft_consensus.cc:2376] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5. Re-sending same reply.
I20260709 13:05:05.413446 13866 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:05.421236 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 56413.7us
I20260709 13:05:05.419984 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:05.402665 16515 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.430598 13988 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:05.413069 16584 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:05.490444 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.488526 16580 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:05.491110 16626 raft_consensus.cc:2764] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:05:05.451361 16325 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:05:05.604429 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 178873us
I20260709 13:05:05.503613 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:05.487390 16419 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.502884 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:05.495031 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:05.580753 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.679173 14023 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
W20260709 13:05:05.675582 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 255636us
I20260709 13:05:05.494891 16535 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:05.440128 16515 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.687316 16617 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:05.499426 16580 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:05.723308 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:05.608036 16584 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.776206 16574 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.777348 14558 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:05.778450 16580 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:05.662030 16278 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:05.781775 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:05.799516 16535 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:05.799994 16560 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:05.682711 16325 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:05.800693 16560 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:05.570353 16564 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.209s
I20260709 13:05:05.801990 16325 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:05.804531 16632 log.cc:927] Time spent T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.197s	user 0.000s	sys 0.006s
I20260709 13:05:05.804945 16564 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:05.806962 16561 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:05.807428 16561 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:05.808557 16561 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:05.793403 16515 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:05.797216 16573 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:05.766812 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:05.777221 16574 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.675326 16572 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.652433 16316 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.809329 16562 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:05.699337 16571 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:05.445807 16603 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.682206 16419 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:05.888978 14559 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:05.871440 16573 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.944475 13941 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:05.931630 16379 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:05.887523 16571 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:05.978070 13941 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:05.932216 16578 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:05.978742 16578 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:05.939359 16560 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:05.980950 13941 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:05.890425 13946 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:05.942971 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:05.935088 16574 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:05.982131 16578 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:05.983928 14020 raft_consensus.cc:2376] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 5. Re-sending same reply.
I20260709 13:05:05.977706 16573 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:05.928565 16624 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:05.938954 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:05.974870 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 278590us
I20260709 13:05:05.993108 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:05.986454 16571 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:05.984588 16578 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:05.912379 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 147631us
I20260709 13:05:05.809310 16580 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:06.011569 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.996466 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:06.008927 14522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.039s	sys 0.045s
W20260709 13:05:06.018324 14522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.040s	sys 0.045s
I20260709 13:05:06.018280 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:05.996428 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.019397 14019 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
W20260709 13:05:06.019968 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.995671 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:06.022760 13866 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:06.023592 14558 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:06.023833 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:06.024629 14022 raft_consensus.cc:2368] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6.
I20260709 13:05:06.024914 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:06.019026 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 618350us
W20260709 13:05:05.929452 16628 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:05:05.883216 16572 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.036203 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.037150 13874 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:05:06.037456 14023 raft_consensus.cc:2368] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5.
I20260709 13:05:06.038460 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.039117 14024 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:05:06.040185 14024 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:06.036936 16567 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:06.048439 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:05.988862 16419 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:06.057940 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.108711 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:06.069094 13989 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.008s	sys 0.020s
W20260709 13:05:06.127866 13989 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.011s	sys 0.022s
I20260709 13:05:06.108623 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:06.095093 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.040230 14023 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:06.022234 16573 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.059322 13866 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
W20260709 13:05:06.108533 16633 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:05:06.109550 16629 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.431s	user 0.000s	sys 0.001s
W20260709 13:05:06.146009 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 97712.2us
I20260709 13:05:06.013401 16624 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.146023 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:06.040604 16566 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:06.071854 16535 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:06.108132 16627 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.420s	user 0.000s	sys 0.001s
W20260709 13:05:06.137113 16601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.005s	sys 0.031s
W20260709 13:05:06.142545 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 151225us
I20260709 13:05:06.031478 16594 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:06.119130 16316 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.132320 16343 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.164187 14022 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: [2.2-3.3]
I20260709 13:05:06.055927 16567 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:06.177987 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:06.157668 16601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.005s	sys 0.032s
I20260709 13:05:06.153362 16559 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.150177 16535 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:06.177345 16592 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.011s	user 0.009s	sys 0.005s
W20260709 13:05:06.181005 16538 consensus_meta.cc:287] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.982s	user 0.014s	sys 0.001s
I20260709 13:05:06.195070 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:06.199277 14554 raft_consensus.cc:2410] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:05:06.199689 14561 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:06.148952 16573 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.146131 16419 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:05:06.207523 13840 leader_election.cc:400] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4.
W20260709 13:05:06.209862 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.192287 16535 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:06.218994 16602 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:05:06.223097 16591 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.476s	user 0.002s	sys 0.005s
I20260709 13:05:06.194669 16638 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.224519 16638 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.229202 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:06.230415 16581 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:06.192495 16559 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.232651 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.233606 14015 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:06.180864 16343 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.235738 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.238467 13840 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:06.200376 16594 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:06.244235 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2157) took 1083 ms. Trace:
I20260709 13:05:06.249258 13950 rpcz_store.cc:276] 0709 13:05:05.160800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.162382 (+  1582us) service_pool.cc:224] Handling call
0709 13:05:05.183889 (+ 21507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.191277 (+  7388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:05.191308 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:05.191852 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 13:05:05.429274 (+237422us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:05.429316 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.429408 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.243976 (+814568us) raft_consensus.cc:1685] finished
0709 13:05:06.244008 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.244166 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11074,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":14267,"thread_start_us":219679,"threads_started":1,"wal-append.queue_time_us":410517}
I20260709 13:05:06.259334 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:06.263224 16581 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:06.270025 16316 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.270978 16316 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.273391 16316 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.257051 16601 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.270272 16581 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:06.270255 13866 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:06.234011 16559 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.199725 16592 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:06.283202 16614 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.091s	user 0.002s	sys 0.010s
W20260709 13:05:06.283492 16644 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
W20260709 13:05:06.291299 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.273952 16316 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:06.292997 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 54584.6us
I20260709 13:05:06.297544 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:06.288959 16566 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.316452 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:06.316697 16573 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.319439 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:06.293834 16580 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:06.237202 16567 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.279696 16601 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.332597 16567 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:06.334772 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2149) took 1416 ms. Trace:
I20260709 13:05:06.335268 13948 rpcz_store.cc:276] 0709 13:05:04.917926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.918114 (+   188us) service_pool.cc:224] Handling call
0709 13:05:04.918540 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.356329 (+437789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.334475 (+978146us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b5400085f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:06.334689 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1005172096}
I20260709 13:05:06.321776 16566 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.317971 16572 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:06.309894 16316 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.348404 16594 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:06.346426 16580 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:06.353420 16580 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.353740 16566 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.340667 16559 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.364398 16559 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.357806 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:06.366784 16559 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:06.367074 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.358784 16498 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:06.368369 13838 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:06.357831 16646 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:06.368695 16498 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.369869 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1549) took 1139 ms. Trace:
I20260709 13:05:06.341048 16573 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.371140 16498 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.370961 16573 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.370302 13947 rpcz_store.cc:276] 0709 13:05:05.230840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.230966 (+   126us) service_pool.cc:224] Handling call
0709 13:05:05.231302 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.231668 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:05.231699 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:05.233545 (+  1846us) log.cc:844] Serialized 26 byte log entry
0709 13:05:06.237763 (+1004218us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b180039cfa0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:05:06.280630 (+ 42867us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:06.280660 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.280709 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.369516 (+ 88807us) raft_consensus.cc:1685] finished
0709 13:05:06.369540 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.369798 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2077,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":3887,"spinlock_wait_cycles":2813312,"thread_start_us":42689,"threads_started":1,"wal-append.queue_time_us":1096395}
I20260709 13:05:06.373426 16573 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.375888 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:06.347234 16581 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.361477 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 516) took 1460 ms. Trace:
I20260709 13:05:06.368484 16566 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:06.373284 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:06.361948 16592 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:06.360351 16601 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:06.359566 16561 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:06.369377 16646 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.377859 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:06.454036 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:06.429769 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:06.352792 16664 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:06.477684 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 180319us
I20260709 13:05:06.477840 16668 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:06.423597 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:06.483036 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:06.357316 16624 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:06.384205 16594 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:05:06.451479 16587 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.383s	user 0.007s	sys 0.027s
I20260709 13:05:06.465559 16661 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:06.492278 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 173151us
I20260709 13:05:06.467811 16573 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:06.474345 14551 rpcz_store.cc:276] 0709 13:05:04.901388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.907318 (+  5930us) service_pool.cc:224] Handling call
0709 13:05:04.961757 (+ 54439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.011448 (+ 49691us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:05.076356 (+ 64908us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.360912 (+1284556us) spinlock_profiling.cc:243] Waited 8.3 ms on lock 0x7b180037b8e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287
0709 13:05:06.361041 (+   129us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:06.361100 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:06.361164 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.361243 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.361393 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":135765,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":136754,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":17769,"spinlock_wait_cycles":32352256,"thread_start_us":535321,"threads_started":3,"wal-append.queue_time_us":579172}
W20260709 13:05:06.494982 16587 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.427s	user 0.007s	sys 0.027s
I20260709 13:05:06.507669 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:06.507164 13840 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:06.491197 16566 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.516383 16566 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.516842 13840 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:06.516938 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:06.517975 16536 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:06.518301 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:06.518412 16536 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:06.519492 16536 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.520592 16536 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:06.521591 16316 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.494033 16343 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:06.497206 16580 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.523280 16580 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:06.501217 16573 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.528060 16573 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.528846 14020 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:06.529059 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:06.529968 14017 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:06.530233 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:06.530385 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:06.531553 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.494578 16668 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:06.533128 13950 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:06.534627 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:06.535359 14015 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:06.466317 16535 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.536134 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:06.497030 16592 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:06.499078 16594 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:06.547343 14022 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:06.548012 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:06.548730 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 170974us
I20260709 13:05:06.550864 13838 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:06.516957 16566 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:05:06.553860 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:06.548992 16578 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.521s	user 0.012s	sys 0.001s
I20260709 13:05:06.560838 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:06.563035 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:06.563608 13868 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:06.563962 13867 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:06.560820 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:06.565001 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.568173 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:06.565687 14015 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:06.573808 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:06.567550 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.574791 13875 raft_consensus.cc:2410] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:06.575789 14022 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:06.573840 13950 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:06.499825 16661 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.560707 16668 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:06.583427 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.587389 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:06.588572 14019 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:06.605093 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:06.578017 16592 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:06.660602 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:06.606812 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 88556us
I20260709 13:05:06.650779 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1533) took 1999 ms. Trace:
I20260709 13:05:06.646569 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.671167 14019 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:06.669754 13949 rpcz_store.cc:276] 0709 13:05:04.651670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.651798 (+   128us) service_pool.cc:224] Handling call
0709 13:05:04.652190 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.072404 (+420214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.650471 (+1578067us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b54001f4fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:06.650707 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964716928}
I20260709 13:05:06.633790 14017 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
W20260709 13:05:06.674232 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.675757 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:06.651785 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.677697 13944 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:06.682258 13948 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:06.582950 16667 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:06.667140 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:06.667778 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:06.687543 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:06.694825 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.695468 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.696816 13948 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:06.698654 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.698961 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.699359 14023 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:06.700346 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:06.700629 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:06.701450 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:06.703014 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.698686 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.705288 13947 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:06.705296 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2216) took 1059 ms. Trace:
I20260709 13:05:06.701603 14022 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:06.707247 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.708549 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:06.646744 16578 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.703660 14020 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:06.640707 16638 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:06.710978 16592 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:06.714907 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.710033 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:06.718375 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.720023 13942 raft_consensus.cc:2410] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:06.635578 16514 raft_consensus.cc:3055] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:06.644347 16447 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.732915 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:06.733703 14017 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:06.654086 16573 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.725525 13947 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:06.727363 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:06.730211 16621 consensus_queue.cc:1059] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:06.731092 16669 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.731823 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.742799 14015 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:06.732816 16375 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:06.736364 16592 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.737402 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:06.738031 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:06.737866 16447 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.761287 16668 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.767166 16535 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:06.768090 16606 consensus_queue.cc:1059] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:06.769208 16606 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.770238 16588 raft_consensus.cc:3055] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:06.770718 16588 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:06.702710 16566 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.771654 16588 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.772377 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:06.772923 16588 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:06.704352 16325 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.745787 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.740154 13948 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:06.733531 16514 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:06.754979 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.786856 14023 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:06.787068 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.758968 16375 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.759239 16624 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:06.766929 16617 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:06.768316 16601 raft_consensus.cc:3055] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:06.846421 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:06.776103 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:06.779052 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.705840 14024 rpcz_store.cc:276] 0709 13:05:05.646356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.129517 (+483161us) service_pool.cc:224] Handling call
0709 13:05:06.137340 (+  7823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:06.153398 (+ 16058us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:06.165023 (+ 11625us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.582898 (+417875us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b1800252b80. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd323a 00007fb2c343da2a
0709 13:05:06.704817 (+121919us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:06.704897 (+    80us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:06.704976 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.705069 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.705220 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9988,"prepare.queue_time_us":18194,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":10216,"spinlock_wait_cycles":3596672,"thread_start_us":111380,"threads_started":1,"wal-append.queue_time_us":519863}
I20260709 13:05:06.730099 16661 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.796898 14019 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:06.765406 16621 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:06.801757 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:06.852998 13942 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:06.852932 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:06.846072 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:06.865566 13869 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:06.750447 13947 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:06.849464 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:06.769757 16606 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.753310 16669 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.892305 16669 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.687059 16594 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.774994 16447 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.893946 14019 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:06.894492 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:06.911015 14024 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:05:06.901000 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:06.798484 16611 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.732326 16578 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.915880 14022 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:06.745352 16573 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.853541 16623 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:06.933458 13870 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:06.863071 16552 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.950237 16552 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.952773 16552 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:06.846028 16601 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:06.953366 16606 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:06.799788 16650 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.134s
I20260709 13:05:06.954382 16650 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:06.953898 16601 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.954757 16650 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:06.955340 16601 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:06.955623 16650 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:06.894968 16651 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.582s	user 0.004s	sys 0.000s
I20260709 13:05:06.964891 13875 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:06.772401 16566 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.961781 13869 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:06.963891 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:06.787828 16325 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:06.969684 13868 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:06.894502 16646 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:06.914888 16316 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.895877 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:06.905576 16603 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:06.972898 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 198962us
I20260709 13:05:06.870599 16582 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
W20260709 13:05:06.980188 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 200207us
I20260709 13:05:06.957151 16650 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:07.001291 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.028007 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.029376 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:06.970302 16514 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:06.945996 16611 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:07.198696 14021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1507) took 2043 ms (client timeout 2266 ms). Trace:
W20260709 13:05:07.226042 14021 rpcz_store.cc:269] 0709 13:05:05.155675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.357941 (+202266us) service_pool.cc:224] Handling call
0709 13:05:07.198388 (+1840447us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54000ab754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:07.198575 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2796902016}
I20260709 13:05:07.210675 13949 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:07.216073 13875 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
W20260709 13:05:07.168409 13840 connection.cc:256] RPC call timeout handler was delayed by -1.0896762730000091s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17070000000000002s for a response
I20260709 13:05:07.184662 14559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 519) took 1787 ms. Trace:
W20260709 13:05:07.220135 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.068701 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.230351 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.232266 13950 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:05:07.081995 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.182539 14018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1497) took 2101 ms (client timeout 1707 ms). Trace:
I20260709 13:05:07.003702 16646 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:07.077358 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 365160us
W20260709 13:05:07.234781 14018 rpcz_store.cc:269] 0709 13:05:05.081204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.081343 (+   139us) service_pool.cc:224] Handling call
0709 13:05:07.182197 (+2100854us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540020d0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:07.182416 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2470492928}
I20260709 13:05:07.031424 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.228757 14559 rpcz_store.cc:276] 0709 13:05:05.397010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.397127 (+   117us) service_pool.cc:224] Handling call
0709 13:05:05.417641 (+ 20514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.975762 (+558121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.184330 (+1208568us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b5400710f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:07.184583 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1082619648}
W20260709 13:05:07.238504 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 209257us
W20260709 13:05:07.183445 16676 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:05:07.190008 16631 log.cc:927] Time spent T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 1.331s	user 0.003s	sys 0.007s
I20260709 13:05:07.239627 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:07.030853 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:07.243345 13868 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:07.181747 16660 log.cc:927] Time spent T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.865s	user 0.002s	sys 0.000s
W20260709 13:05:07.233727 16686 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:05:07.003095 16552 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:07.029052 16567 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.696s	user 0.009s	sys 0.004s
I20260709 13:05:07.261601 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:07.268348 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:07.251210 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:07.003369 16316 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:07.062112 16669 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.247967 13873 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:05:07.253261 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:07.346850 13948 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:05:07.027869 16603 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:07.241452 16467 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.211127 16578 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:07.347405 13948 pending_rounds.cc:85] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 2
W20260709 13:05:07.249187 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.278136 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:07.115705 16617 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.137s
I20260709 13:05:07.385838 13948 pending_rounds.cc:107] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:05:07.386622 13948 raft_consensus.cc:2889] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:07.347684 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.247720 16298 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:07.111903 16673 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:05:07.071889 16621 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.182s
I20260709 13:05:07.203531 16325 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:07.381709 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.114957 16359 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.139s
I20260709 13:05:07.394516 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:07.394140 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:07.395114 13875 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:07.397606 13946 raft_consensus.cc:2393] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:07.396137 14018 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:07.266723 16514 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:07.411676 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:07.251484 16649 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:07.353480 13866 raft_consensus.cc:2410] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:07.351377 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:07.267694 16587 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.401031 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.349211 16375 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:07.444631 13873 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
W20260709 13:05:07.350850 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 99727.2us
I20260709 13:05:07.432740 13943 raft_consensus.cc:2376] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 6. Re-sending same reply.
I20260709 13:05:07.398576 16573 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:07.254199 16581 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.181s
I20260709 13:05:07.344305 16626 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:07.438632 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:07.462973 13941 raft_consensus.cc:2410] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:07.458074 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:07.461542 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 108406us
W20260709 13:05:07.421608 16667 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.026s	sys 0.063s
I20260709 13:05:07.389402 16669 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:07.249686 16646 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:07.266685 16538 consensus_queue.cc:1059] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.169s
I20260709 13:05:07.476801 14560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 524) took 1237 ms. Trace:
I20260709 13:05:07.480623 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:07.271729 16552 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:07.483043 14560 rpcz_store.cc:276] 0709 13:05:06.239518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.239693 (+   175us) service_pool.cc:224] Handling call
0709 13:05:06.240156 (+   463us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:06.246287 (+  6131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:06.246362 (+    75us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:06.247340 (+   978us) log.cc:844] Serialized 52 byte log entry
0709 13:05:06.638316 (+390976us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b180026ac40. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1
0709 13:05:06.638526 (+   210us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:06.638569 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.638652 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:07.180868 (+542216us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:07.476546 (+295678us) raft_consensus.cc:1685] finished
0709 13:05:07.476571 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:07.476730 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7476,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":8709,"spinlock_wait_cycles":7315072,"thread_start_us":10546,"threads_started":1,"wal-append.queue_time_us":30259}
W20260709 13:05:07.480195 16667 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.031s	sys 0.064s
I20260709 13:05:07.482322 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.417917 16467 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:07.495015 13941 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:07.500972 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:07.500190 16639 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:07.507086 14561 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:07.482801 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:07.512259 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:07.487042 16560 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:07.523442 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:07.524317 13871 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:07.526546 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.527319 13875 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:05:07.527338 13873 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
W20260709 13:05:07.501011 16678 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:05:07.494182 16667 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:07.391185 16668 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.030s	sys 0.028s
W20260709 13:05:07.554441 13870 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.608s	user 0.013s	sys 0.000s
I20260709 13:05:07.493801 16566 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:07.470846 16573 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:07.551213 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:07.552457 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 70441.9us
I20260709 13:05:07.548319 13914 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:07.544672 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.562386 13875 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:07.562381 13915 leader_election.cc:400] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6.
I20260709 13:05:07.563860 13915 leader_election.cc:403] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:05:07.553666 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:07.555387 13870 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:05:07.566426 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:07.567018 13875 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:07.569533 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.558634 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:07.520262 16679 raft_consensus.cc:3055] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:07.573612 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:07.574230 13871 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:07.513260 16639 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:07.592535 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.598605 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:07.599891 13870 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:07.599686 14018 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:05:07.591444 16690 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:05:07.601514 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:07.554824 16668 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.030s	sys 0.028s
I20260709 13:05:07.453085 16587 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:07.545313 16624 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.613286 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:07.630146 13873 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:07.571532 16278 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:07.572412 16649 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:07.591737 16359 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 13:05:07.605018 16639 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:07.596197 16679 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:07.695941 13875 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
W20260709 13:05:07.756865 14586 heartbeater.cc:646] Failed to heartbeat to 127.12.109.126:43315 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.12.109.126:43315 timed out after 15.000s (SENT)
W20260709 13:05:07.759042 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.739851 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.772114 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.692987 16278 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.760870 16669 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:07.616966 16560 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.143s
W20260709 13:05:07.761824 16693 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
W20260709 13:05:07.762619 16695 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.237s	user 0.000s	sys 0.000s
I20260709 13:05:07.763772 16587 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:07.810925 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:07.769937 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 207769us
I20260709 13:05:07.812325 13915 leader_election.cc:400] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5.
I20260709 13:05:07.812640 13915 leader_election.cc:403] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:07.603014 16613 consensus_queue.cc:1059] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.122s
I20260709 13:05:07.814129 16613 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:07.814610 16613 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:07.815806 16613 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:07.817564 16613 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:07.825477 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:07.826229 13868 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:07.829545 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.801985 16278 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:07.783838 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2338) took 1058 ms. Trace:
W20260709 13:05:07.785254 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.801463 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.629755 16573 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6.
W20260709 13:05:07.749121 16685 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.256s	user 0.000s	sys 0.000s
W20260709 13:05:07.806591 14016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.008s	sys 0.014s
W20260709 13:05:07.845937 14016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.008s	sys 0.014s
I20260709 13:05:07.806965 13838 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:07.765692 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.773510 16316 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:07.636997 16624 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:07.875173 13869 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
W20260709 13:05:07.836812 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.837604 13872 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:07.841126 16467 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:07.842029 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.834683 13867 rpcz_store.cc:276] 0709 13:05:06.734829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.735327 (+   498us) service_pool.cc:224] Handling call
0709 13:05:06.735737 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:06.736195 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:06.736225 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:06.736808 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:05:06.888191 (+151383us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:06.888275 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.888381 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:07.783520 (+895139us) raft_consensus.cc:1685] finished
0709 13:05:07.783556 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:07.783753 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32306,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":128,"thread_start_us":150886,"threads_started":1,"wal-append.queue_time_us":291919}
I20260709 13:05:07.781692 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.890930 13875 raft_consensus.cc:2410] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:05:07.892114 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.893822 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:07.848696 16592 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.010s	sys 0.020s
W20260709 13:05:07.871457 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 297082us
I20260709 13:05:07.898515 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:07.901535 13875 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:07.839617 16691 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:07.828083 16566 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:05:07.907050 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:07.908226 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:07.904049 16592 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.010s	sys 0.020s
I20260709 13:05:07.910177 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.918361 13873 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:07.910509 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:07.896977 16316 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:07.912374 16626 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:07.912640 16552 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:07.897567 16573 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:07.913415 16702 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5.
W20260709 13:05:07.912366 16696 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:05:07.848280 16611 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:07.876667 16672 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.887562 16298 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:07.919870 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.921207 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:07.960960 14561 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:07.804737 16667 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:07.927021 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:07.955780 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 143571us
I20260709 13:05:07.961345 14558 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:05:07.965031 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.960582 13867 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:07.986706 14015 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:07.983166 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:07.978178 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:07.977826 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:07.993362 14019 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:05:07.991261 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1519) took 1285 ms. Trace:
I20260709 13:05:07.997071 14017 rpcz_store.cc:276] 0709 13:05:06.706105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.868817 (+162712us) service_pool.cc:224] Handling call
0709 13:05:06.902512 (+ 33695us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:06.903022 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:06.903051 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:06.903641 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 13:05:07.346924 (+443283us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:07.408587 (+ 61663us) log.cc:844] Serialized 14 byte log entry
0709 13:05:07.437054 (+ 28467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.437256 (+   202us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:07.990989 (+553733us) raft_consensus.cc:1685] finished
0709 13:05:07.991015 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:07.991180 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33991,"prepare.queue_time_us":61426,"prepare.run_wall_time_us":53,"raft.queue_time_us":28453,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":12728,"spinlock_wait_cycles":19726080,"thread_start_us":382847,"threads_started":1,"wal-append.queue_time_us":565601}
I20260709 13:05:07.837653 16679 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:07.932924 16646 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:07.933456 16428 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:07.975802 16698 log.cc:927] Time spent T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:05:08.016047 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.016244 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.078634 13870 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:08.080058 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.080770 13870 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:08.081621 13868 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:07.976230 13942 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:07.933091 16316 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:07.976022 16298 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:07.973951 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:07.932541 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.069854 16679 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:07.967892 16668 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.016556 16552 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.081635 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.106772 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.107384 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:07.980707 16375 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:05:08.110240 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 136370us
I20260709 13:05:08.083755 16428 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.119249 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:07.969914 16672 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.122956 13875 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:08.167893 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:08.102931 13840 leader_election.cc:336] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.707s (SENT)
I20260709 13:05:08.171543 13867 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:07.968819 16603 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:08.120011 13870 raft_consensus.cc:2410] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:05:08.130829 13871 consensus_meta.cc:287] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.531s	user 0.012s	sys 0.000s
I20260709 13:05:07.977264 16584 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 13:05:08.135989 16278 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.190478 16278 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.136757 16707 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:07.965343 16573 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.130216 13942 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:08.142163 16691 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:08.143028 16428 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:08.113925 16716 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:08.144104 13872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2330) took 1602 ms (client timeout 1760 ms). Trace:
I20260709 13:05:08.146680 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:08.147310 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1523) took 1438 ms. Trace:
I20260709 13:05:08.148191 16316 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.150563 16298 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.178274 13868 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:08.240027 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2350) took 1291 ms. Trace:
I20260709 13:05:08.240448 13868 rpcz_store.cc:276] 0709 13:05:06.949028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.119113 (+1170085us) service_pool.cc:224] Handling call
0709 13:05:08.173195 (+ 54082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.239670 (+ 66475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.239958 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.179769 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.128684 16668 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.242233 13941 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:08.179921 13871 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:08.243121 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2174) took 1002 ms. Trace:
I20260709 13:05:08.243541 13871 rpcz_store.cc:276] 0709 13:05:07.240312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.560349 (+320037us) service_pool.cc:224] Handling call
0709 13:05:08.243038 (+682689us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:08.152536 16601 consensus_meta.cc:287] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.197s	user 0.013s	sys 0.001s
I20260709 13:05:08.247991 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:08.176896 16530 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.186008 16566 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:08.186894 16709 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:05:08.120285 16375 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:08.116357 16624 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:08.181473 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:08.192353 13870 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:05:08.198640 16691 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:08.194283 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 91385.4us
I20260709 13:05:08.195884 13875 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:05:08.167582 16560 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:08.257550 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 82121.4us
I20260709 13:05:08.222196 14015 rpcz_store.cc:276] 0709 13:05:06.708864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.860044 (+151180us) service_pool.cc:224] Handling call
0709 13:05:07.182762 (+322718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.986495 (+803733us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54002ba5a0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33aadc0 00007fb2c34351c0 00007fb2c3436231 00007fb2c3432d8f 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c
0709 13:05:07.992455 (+  5960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.147093 (+154638us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b540020d0d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:08.147243 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1081290880}
W20260709 13:05:08.221837 13872 rpcz_store.cc:269] 0709 13:05:06.541713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.541856 (+   143us) service_pool.cc:224] Handling call
0709 13:05:08.143885 (+1602029us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54000a8054. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:08.144028 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2961869568}
I20260709 13:05:08.199764 16712 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:05:08.131337 16552 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.192363 16707 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:08.263823 13867 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:05:08.241719 13873 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
W20260709 13:05:08.245754 16588 consensus_meta.cc:287] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.472s	user 0.011s	sys 0.003s
I20260709 13:05:08.253965 13870 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:08.193342 13988 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:08.255141 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.199066 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:08.260537 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.237531 16716 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:08.255899 13840 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.063s (SENT)
W20260709 13:05:08.268644 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 75399.3us
W20260709 13:05:08.271232 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.270454 13870 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:05:08.274268 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 75288.6us
I20260709 13:05:08.275038 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:08.251768 16592 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:08.279084 13873 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:08.281049 13870 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:08.282196 13875 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:08.300319 13870 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 6 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:08.267050 16316 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:08.323194 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62760.8us
I20260709 13:05:08.255204 16566 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.255185 16530 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.358744 16668 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.359876 16530 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:08.360572 16552 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.277522 16601 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:05:08.335807 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:08.277921 16592 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.367857 13873 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:08.275844 16716 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:08.317689 16316 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.370698 16316 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:08.372452 16639 consensus_meta.cc:287] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.000s
I20260709 13:05:08.337857 13870 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:08.373003 16639 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.374398 14559 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:08.276858 16714 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:08.375420 16573 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:08.375896 13875 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:08.278039 16624 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:08.264078 16560 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.377425 16624 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.377832 16624 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:08.379443 16560 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.318188 16649 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:08.382630 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.350618 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 94754.3us
W20260709 13:05:08.384871 13840 leader_election.cc:336] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.266s (SENT)
I20260709 13:05:08.385269 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.386416 16581 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:08.386224 16707 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.387383 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.349375 16375 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.369796 13873 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:08.368064 16592 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:08.318934 16564 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:08.333465 13872 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:08.365589 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.389550 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.387856 16707 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:08.398660 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.388407 16375 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:08.424126 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 58606.7us
W20260709 13:05:08.427012 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.427273 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:08.444622 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:08.445685 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 56208.2us
I20260709 13:05:08.446606 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.449016 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.391477 16668 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:05:08.458689 13989 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.324s (SENT) [suppressed 40 similar messages]
I20260709 13:05:08.452644 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.424052 16702 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:08.449746 13872 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:05:08.423585 16623 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.460028 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:08.425827 16560 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:05:08.424474 16564 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.388973 16278 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:05:08.428356 16584 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:08.427666 16712 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:08.483826 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:08.447353 16581 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:08.391711 16646 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
W20260709 13:05:08.535862 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52120.1us
I20260709 13:05:08.536111 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2181) took 1101 ms. Trace:
I20260709 13:05:08.536525 13867 rpcz_store.cc:276] 0709 13:05:07.434963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.172611 (+737648us) service_pool.cc:224] Handling call
0709 13:05:08.193515 (+ 20904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.265250 (+ 71735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.535789 (+270539us) spinlock_profiling.cc:243] Waited 69.9 ms on lock 0x7b54002e59d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:08.536016 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160695552}
I20260709 13:05:08.537037 16646 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:08.537559 16646 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.537932 16646 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:08.457826 16667 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:08.537760 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.490280 16564 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:08.489192 16714 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:08.534139 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.465246 16573 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:05:08.508854 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 56305.1us
I20260709 13:05:08.539311 13867 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:08.551611 13871 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 6 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:08.486002 16682 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:08.540082 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:08.541939 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2329) took 2007 ms. Trace:
I20260709 13:05:08.483275 16718 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:08.485426 16702 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.543864 13874 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:05:08.562242 13874 pending_rounds.cc:85] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 5
I20260709 13:05:08.562556 13874 pending_rounds.cc:107] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 3.6
W20260709 13:05:08.554672 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 96030.2us
I20260709 13:05:08.554262 13866 rpcz_store.cc:276] 0709 13:05:06.534175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.534337 (+   162us) service_pool.cc:224] Handling call
0709 13:05:08.541664 (+2007327us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54000e9554. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:08.541869 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1874359680}
I20260709 13:05:08.489704 16623 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.555982 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:08.562839 13874 raft_consensus.cc:2889] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:08.497761 16316 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:08.552361 16650 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.478s	user 0.014s	sys 0.004s
I20260709 13:05:08.579664 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:08.580415 13942 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:08.580651 16691 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.581502 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:08.563776 16702 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:08.571352 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:08.572984 16722 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:05:08.577219 16613 consensus_meta.cc:287] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.728s	user 0.009s	sys 0.001s
I20260709 13:05:08.584584 16639 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.586182 16691 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:08.550784 16584 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.588084 16667 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:08.588613 16667 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.589020 16667 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:08.593191 16697 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.497s	user 0.001s	sys 0.003s
I20260709 13:05:08.594913 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2186) took 1349 ms. Trace:
I20260709 13:05:08.596117 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:08.595353 13947 rpcz_store.cc:276] 0709 13:05:07.245864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.247226 (+  1362us) service_pool.cc:224] Handling call
0709 13:05:07.250110 (+  2884us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:07.250497 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:07.250526 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:07.270362 (+ 19836us) log.cc:844] Serialized 26 byte log entry
0709 13:05:08.151064 (+880702us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b180032a760. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:05:08.315448 (+164384us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:08.315518 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.315619 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.594574 (+278955us) raft_consensus.cc:1685] finished
0709 13:05:08.594596 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.594843 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1948,"raft.queue_time_us":489,"raft.run_cpu_time_us":414,"raft.run_wall_time_us":139688,"spinlock_wait_cycles":2309376,"thread_start_us":164083,"threads_started":1,"wal-append.queue_time_us":1100913}
I20260709 13:05:08.597434 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:08.552703 16573 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.550740 16714 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:08.606529 16714 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:08.607270 16325 raft_consensus.cc:3055] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:08.607686 16325 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:08.608296 16716 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.608675 16325 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.612422 16682 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:08.597543 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:08.597951 14561 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:08.600822 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:08.601680 13942 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:05:08.602778 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2195) took 1207 ms. Trace:
I20260709 13:05:08.603718 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:08.604650 16584 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:08.614137 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:08.615175 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:08.604921 16623 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:08.620404 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 49141.5us
I20260709 13:05:08.609804 16325 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:08.614140 13945 rpcz_store.cc:276] 0709 13:05:07.394927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.923153 (+528226us) service_pool.cc:224] Handling call
0709 13:05:07.923481 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.266394 (+342913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.266444 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:08.267122 (+   678us) log.cc:844] Serialized 26 byte log entry
0709 13:05:08.437916 (+170794us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:08.437961 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.438043 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.602541 (+164498us) raft_consensus.cc:1685] finished
0709 13:05:08.602566 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.602701 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2727,"raft.queue_time_us":149430,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":17716,"thread_start_us":454923,"threads_started":2,"wal-append.queue_time_us":253650}
I20260709 13:05:08.622643 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:08.622931 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.627868 13950 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:08.632650 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 96051.5us
I20260709 13:05:08.623916 13947 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:08.634747 13945 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:08.636571 13945 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:05:08.636822 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.613620 16573 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:08.640161 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.581789 16624 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.643705 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:08.621704 16515 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:08.624110 16682 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:08.634042 16623 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:08.640715 16298 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:08.653503 16467 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:08.660030 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1566) took 1398 ms. Trace:
I20260709 13:05:08.640942 16668 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:08.637761 16584 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:08.663848 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 279) took 1946 ms. Trace:
I20260709 13:05:08.640920 16581 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:05:08.672746 13946 rpcz_store.cc:276] 0709 13:05:07.261199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.398576 (+137377us) service_pool.cc:224] Handling call
0709 13:05:07.399006 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.399241 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:07.673318 (+274077us) log.cc:844] Serialized 14 byte log entry
0709 13:05:07.978236 (+304918us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b18001ba240. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c730758f 00007fb2c0346571 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd
0709 13:05:07.979560 (+  1324us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b18001ba240. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:08.537614 (+558054us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b18001ba240. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd323a
0709 13:05:08.656179 (+118565us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.656511 (+   332us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:08.656571 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:08.657859 (+  1288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.657933 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.659947 (+  2014us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6373,"prepare.queue_time_us":488251,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":50151,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":4595,"spinlock_wait_cycles":27047040,"thread_start_us":583038,"threads_started":3,"wal-append.queue_time_us":228687}
I20260709 13:05:08.673920 14016 rpcz_store.cc:276] 0709 13:05:06.717483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.871291 (+153808us) service_pool.cc:224] Handling call
0709 13:05:06.887419 (+ 16128us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:06.959025 (+ 71606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:06.959070 (+    45us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:06.999391 (+ 40321us) log.cc:844] Serialized 78 byte log entry
0709 13:05:07.846388 (+846997us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:08.023951 (+177563us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.038975 (+ 15024us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.042684 (+  3709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.043057 (+   373us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.592521 (+549464us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:08.663450 (+ 70929us) raft_consensus.cc:1685] finished
0709 13:05:08.663494 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.663749 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16683,"prepare.queue_time_us":169628,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":110,"raft.queue_time_us":32293,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":16625,"spinlock_wait_cycles":56466944,"thread_start_us":880264,"threads_started":2,"wal-append.queue_time_us":897842}
I20260709 13:05:08.693944 16316 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.695313 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2197) took 1099 ms. Trace:
I20260709 13:05:08.667658 16718 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:08.706063 13948 rpcz_store.cc:276] 0709 13:05:07.595814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.987809 (+391995us) service_pool.cc:224] Handling call
0709 13:05:07.988130 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.111761 (+123631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.111809 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:08.111872 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:08.111920 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.695035 (+583115us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54000f5354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:08.695088 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.695240 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283656064}
I20260709 13:05:08.675421 16603 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:08.716015 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2200) took 1045 ms. Trace:
I20260709 13:05:08.716527 13944 rpcz_store.cc:276] 0709 13:05:07.670982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.188997 (+518015us) service_pool.cc:224] Handling call
0709 13:05:08.189338 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.250294 (+ 60956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.250326 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:08.267456 (+ 17130us) log.cc:844] Serialized 26 byte log entry
0709 13:05:08.478958 (+211502us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:08.479010 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.479095 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.715719 (+236624us) raft_consensus.cc:1685] finished
0709 13:05:08.715740 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.715940 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33279,"raft.queue_time_us":50212,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"thread_start_us":145385,"threads_started":1,"wal-append.queue_time_us":330498}
I20260709 13:05:08.718868 16624 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:08.655894 16428 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.686681 16584 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:08.733465 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 80690.2us
W20260709 13:05:08.735069 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 87444.4us
W20260709 13:05:08.734215 13915 connection.cc:256] RPC call timeout handler was delayed by -1.854238693999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 13:05:08.735714 13915 connection.cc:256] RPC call timeout handler was delayed by -1.8506386940000539s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
I20260709 13:05:08.691910 16716 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.675220 16564 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.731132 16603 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.746197 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:08.749430 16624 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:08.750578 16718 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.751051 16716 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:08.751766 16716 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.741503 16581 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:05:08.755662 16667 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.733587 16702 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.760020 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:08.763893 14561 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:08.765517 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:08.766760 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.745914 16603 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:08.766942 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:08.763392 16515 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:05:08.740186 16646 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.737084 16584 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:08.833721 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.813637 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:08.837731 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 70873.6us
I20260709 13:05:08.803756 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:08.840237 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.772809 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:08.842394 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:08.843974 13942 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:05:08.841765 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:08.758045 16667 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.762735 16716 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:08.846984 13915 connection.cc:256] RPC call timeout handler was delayed by -1.6661386939999829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
W20260709 13:05:08.849113 13915 connection.cc:256] RPC call timeout handler was delayed by -1.6355386940000471s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19990000000000002s for a response
W20260709 13:05:08.849583 13915 connection.cc:256] RPC call timeout handler was delayed by -1.6301386940000384s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2005s for a response
W20260709 13:05:08.850008 13915 connection.cc:256] RPC call timeout handler was delayed by -1.4591386939999893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2195s for a response
W20260709 13:05:08.850325 13915 connection.cc:256] RPC call timeout handler was delayed by -1.4425590289999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 13:05:08.850600 13915 connection.cc:256] RPC call timeout handler was delayed by -1.2667873960000406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
W20260709 13:05:08.850966 13915 connection.cc:256] RPC call timeout handler was delayed by -1.2077649750000319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
W20260709 13:05:08.851317 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0852590289999853s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19630000000000003s for a response
I20260709 13:05:08.851953 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:08.854485 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.855283 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:08.857326 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:08.858156 14561 raft_consensus.cc:2368] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8.
I20260709 13:05:08.743896 16592 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.766641 16624 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:08.874310 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.874423 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:08.875186 13866 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:05:08.835474 16418 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.856324 13870 raft_consensus.cc:2376] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3. Re-sending same reply.
I20260709 13:05:08.877075 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:08.843490 16578 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.774271 16428 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.775352 16564 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:08.842150 16650 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:08.861038 16679 consensus_meta.cc:287] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.746s	user 0.014s	sys 0.000s
I20260709 13:05:08.885756 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:08.881714 16578 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.891754 16702 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.893373 16578 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.895358 16573 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.897692 16573 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.871685 16740 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:08.898717 16740 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.899055 16690 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:08.899483 16646 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.878545 16588 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
W20260709 13:05:08.900511 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 46533.8us
I20260709 13:05:08.900875 16740 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:08.901463 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:08.882007 16418 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.902884 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:08.903468 16668 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:08.766420 16623 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.108s
I20260709 13:05:08.904506 16714 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:08.905078 16689 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.290s	user 0.001s	sys 0.004s
I20260709 13:05:08.775310 16375 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 13:05:08.775925 16467 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 13:05:08.849134 16515 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.881139 16667 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.906741 13868 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:08.907123 13946 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:08.869642 16712 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:08.911037 13870 raft_consensus.cc:2410] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:05:08.891655 16737 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:05:08.916991 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:08.921569 13866 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:08.776113 16581 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.135s
I20260709 13:05:08.925983 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:08.928561 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:08.906417 16650 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.910526 16646 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:08.914410 16740 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:05:08.936632 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.964639 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.965718 13866 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:08.964676 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.934794 16738 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:08.974783 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.976696 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.977509 13866 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:08.978693 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:08.979416 13866 raft_consensus.cc:2376] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5. Re-sending same reply.
I20260709 13:05:08.980629 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:08.911334 16578 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:08.933426 16702 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:08.935285 16316 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:08.987365 13869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2347) took 2086 ms (client timeout 1558 ms). Trace:
W20260709 13:05:08.981770 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 133728us
I20260709 13:05:08.985033 13950 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:05:08.954764 16650 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:08.987852 13869 rpcz_store.cc:269] 0709 13:05:06.901369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.976199 (+ 74830us) service_pool.cc:224] Handling call
0709 13:05:08.987162 (+2010963us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b54000a8f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:08.987279 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1941633920}
I20260709 13:05:08.982833 16740 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:08.918206 16515 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:08.998597 13948 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
W20260709 13:05:08.972895 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 102171us
I20260709 13:05:08.972404 16712 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:08.920979 16667 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:08.972927 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.981998 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:08.974880 16601 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:05:09.002480 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.013196 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2189) took 1745 ms. Trace:
I20260709 13:05:09.002341 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:09.016172 13869 raft_consensus.cc:2376] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5. Re-sending same reply.
I20260709 13:05:08.957358 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 534) took 1014 ms. Trace:
I20260709 13:05:09.007411 13866 raft_consensus.cc:2376] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6. Re-sending same reply.
I20260709 13:05:09.019917 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:09.018530 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45696.8us
I20260709 13:05:09.020955 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:08.987551 16316 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:09.022955 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 278) took 2293 ms. Trace:
I20260709 13:05:09.025514 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:09.013797 13943 rpcz_store.cc:276] 0709 13:05:07.267978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.451603 (+183625us) service_pool.cc:224] Handling call
0709 13:05:07.468859 (+ 17256us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:07.519783 (+ 50924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:07.519817 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:07.525691 (+  5874us) log.cc:844] Serialized 52 byte log entry
0709 13:05:07.808732 (+283041us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:07.808779 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.001047 (+1192268us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b5400086954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:09.001094 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.001146 (+    52us) raft_consensus.cc:1685] finished
0709 13:05:09.001166 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.013120 (+ 11954us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":153,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"spinlock_wait_cycles":100940672,"thread_start_us":200483,"threads_started":1,"wal-append.queue_time_us":376448,"wal-append.run_cpu_time_us":2333,"wal-append.run_wall_time_us":1047454}
I20260709 13:05:09.027212 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:09.025202 14023 rpcz_store.cc:276] 0709 13:05:06.717144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.869293 (+152149us) service_pool.cc:224] Handling call
0709 13:05:06.930240 (+ 60947us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:07.028560 (+ 98320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:07.028595 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:07.060782 (+ 32187us) log.cc:844] Serialized 26 byte log entry
0709 13:05:07.470474 (+409692us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:07.484012 (+ 13538us) log.cc:844] Serialized 14 byte log entry
0709 13:05:07.489349 (+  5337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.145937 (+656588us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b1800175300. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:09.010176 (+864239us) spinlock_profiling.cc:243] Waited 75.2 ms on lock 0x7b540023c654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:09.010253 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.010305 (+    52us) raft_consensus.cc:1685] finished
0709 13:05:09.010319 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.010518 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28118,"prepare.queue_time_us":159677,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":5912,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":42043,"spinlock_wait_cycles":196417664,"thread_start_us":382754,"threads_started":1,"wal-append.queue_time_us":641579}
I20260709 13:05:08.999322 16740 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:09.033413 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.001260 16712 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:09.055524 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:09.021541 16714 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:09.026882 16578 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:09.077603 16738 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:09.078228 16578 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:09.078274 16738 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.079353 16578 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.080631 16738 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:09.032758 16716 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:09.084146 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.056735 16751 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.062130 13944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:05:09.056581 16718 raft_consensus.cc:2764] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:05:09.106285 13944 raft_consensus.cc:606] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.106741 13944 raft_consensus.cc:614] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:09.107909 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:09.108274 13867 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:09.109366 13944 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:09.026403 16316 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:09.110329 16718 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:09.110906 16718 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.111920 16748 consensus_queue.cc:1059] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:09.112687 16748 consensus_queue.cc:1243] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:09.113095 16718 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:09.075919 16667 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:09.023249 16467 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:05:09.032984 16573 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:09.054301 16278 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:05:09.030637 16702 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:09.115408 16702 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:09.115842 16702 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:09.116243 16467 consensus_queue.cc:1243] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:09.089710 16620 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:09.093639 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:09.116802 16467 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:09.117177 16467 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:09.117403 14555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 618) took 1186 ms. Trace:
I20260709 13:05:09.023113 16601 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.019858 14552 rpcz_store.cc:276] 0709 13:05:07.942910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.133077 (+190167us) service_pool.cc:224] Handling call
0709 13:05:08.133476 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.133737 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:08.134131 (+   394us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.696443 (+562312us) spinlock_profiling.cc:243] Waited 6.79 ms on lock 0x7b180041b7e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287 00007fb2c344e208
0709 13:05:08.890486 (+194043us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:08.911192 (+ 20706us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:08.917264 (+  6072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.933062 (+ 15798us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.953122 (+ 20060us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30617,"prepare.queue_time_us":9946,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":45,"raft.queue_time_us":36978,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":17056,"spinlock_wait_cycles":46983808,"thread_start_us":318568,"threads_started":2,"wal-append.queue_time_us":304372}
I20260709 13:05:09.117978 16467 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.119547 14552 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:05:09.120281 16601 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:09.114213 14556 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:09.121230 16467 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:09.052609 16740 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:09.124348 13944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:05:09.124974 13944 raft_consensus.cc:606] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.125324 13944 raft_consensus.cc:614] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:09.095911 16515 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:09.105404 16747 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:05:09.117801 14555 rpcz_store.cc:276] 0709 13:05:07.931032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.931214 (+   182us) service_pool.cc:224] Handling call
0709 13:05:09.093430 (+1162216us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b5800004898. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:09.117336 (+ 23906us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46831872}
I20260709 13:05:09.070786 16592 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:09.106159 16751 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.127686 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1575) took 1012 ms. Trace:
I20260709 13:05:09.113159 16748 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:09.129776 16748 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:09.129555 16603 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.130657 13944 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:09.130760 16748 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.131893 16748 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:09.132017 16603 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:09.132241 16592 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:09.097700 13870 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:05:09.133000 13870 pending_rounds.cc:85] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 2
I20260709 13:05:09.133255 13870 pending_rounds.cc:107] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:05:09.133498 13870 raft_consensus.cc:2889] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:09.092190 14553 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:09.056944 16712 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:09.134495 16702 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.128130 13947 rpcz_store.cc:276] 0709 13:05:08.115730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.635472 (+519742us) service_pool.cc:224] Handling call
0709 13:05:08.638365 (+  2893us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.638893 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:08.745820 (+106927us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.914837 (+169017us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:08.915509 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 13:05:08.915723 (+   214us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:08.915772 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.915884 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.127462 (+211578us) raft_consensus.cc:1685] finished
0709 13:05:09.127487 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.127624 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6018,"prepare.queue_time_us":95680,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":9961,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":20440,"thread_start_us":159048,"threads_started":2,"wal-append.queue_time_us":224000}
I20260709 13:05:09.114668 13873 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:09.135288 14553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 531) took 1193 ms. Trace:
I20260709 13:05:09.129438 16751 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:09.139340 16316 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:09.137610 14553 rpcz_store.cc:276] 0709 13:05:07.941673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.946410 (+  4737us) service_pool.cc:224] Handling call
0709 13:05:09.091338 (+1144928us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5800004898. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:05:09.091658 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.134998 (+ 43340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.135223 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24240384}
I20260709 13:05:09.144522 16316 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:09.146026 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:09.147457 14553 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:09.094365 16646 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:09.058458 13869 raft_consensus.cc:2376] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5. Re-sending same reply.
I20260709 13:05:09.166844 14559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 614) took 1681 ms. Trace:
I20260709 13:05:09.167349 14559 rpcz_store.cc:276] 0709 13:05:07.485064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.485204 (+   140us) service_pool.cc:224] Handling call
0709 13:05:07.506321 (+ 21117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.374774 (+868453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.166549 (+791775us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b54007148d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:09.166771 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1995800960}
I20260709 13:05:09.169409 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1571) took 1081 ms. Trace:
I20260709 13:05:09.169898 13941 rpcz_store.cc:276] 0709 13:05:08.087870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.146366 (+ 58496us) service_pool.cc:224] Handling call
0709 13:05:09.169216 (+1022850us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b5400229f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:09.169344 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46284032}
I20260709 13:05:09.170789 16712 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.175148 13867 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:09.149792 16702 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:09.181082 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:09.182156 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2170) took 2500 ms. Trace:
I20260709 13:05:09.182541 13949 rpcz_store.cc:276] 0709 13:05:06.673691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.677846 (+  4155us) service_pool.cc:224] Handling call
0709 13:05:07.210387 (+532541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.227629 (+ 17242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.174024 (+1946395us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b5400054950. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:09.174221 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1223914240}
I20260709 13:05:09.185086 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:09.185109 16566 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:09.187182 16566 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:09.187595 16566 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:09.188010 16566 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:09.197703 13942 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:09.202383 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.203281 14024 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:09.211529 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:09.213631 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 214254us
W20260709 13:05:09.237262 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:09.249749 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.250101 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.250932 14020 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:09.249898 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.253175 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.254495 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.266772 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.267576 14016 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:09.269181 13869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:05:09.269431 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.269991 13869 raft_consensus.cc:606] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.270474 13869 raft_consensus.cc:614] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:09.272676 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.272841 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.273389 14023 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:09.274209 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.210390 16316 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:09.214046 16581 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:09.249725 16638 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.249449 16515 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.256821 16748 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.278079 14019 raft_consensus.cc:2368] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8.
I20260709 13:05:09.278566 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.278627 13989 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:09.282742 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.287739 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.209837 16650 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.291105 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.291412 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.292421 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.294773 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:09.296195 13989 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:09.299062 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.300801 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.302121 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.302672 14023 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:09.289099 16638 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:09.310356 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:09.310662 16751 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:09.311502 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:09.311728 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:09.312402 16702 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:09.312644 13873 raft_consensus.cc:2410] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:09.313072 16702 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:09.313473 16702 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:09.314195 16359 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:09.316347 16672 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:09.279693 16316 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.319718 16316 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:09.317916 16418 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:09.320713 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:09.321105 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2331) took 2777 ms. Trace:
I20260709 13:05:09.279381 16581 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:09.322357 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.323249 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.323555 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.323848 16581 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:09.324512 16581 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:09.324663 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.325321 14019 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:09.325943 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.326121 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:09.321550 13874 rpcz_store.cc:276] 0709 13:05:06.543936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.544119 (+   183us) service_pool.cc:224] Handling call
0709 13:05:08.542397 (+1998278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.568110 (+ 25713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.320873 (+752763us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b54000e9550. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:09.321014 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4576789376}
I20260709 13:05:09.313846 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.327793 13989 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:09.328502 13915 leader_election.cc:400] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8.
I20260709 13:05:09.328902 13915 leader_election.cc:403] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:05:09.329237 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:09.312497 16638 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:09.332819 14020 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
W20260709 13:05:09.333952 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.336160 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.339843 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.339954 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.340159 14023 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:09.340698 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:09.341557 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.342850 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.343611 14016 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:09.349145 14018 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:09.276755 16601 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:05:09.351116 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.276609 16603 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:05:09.352017 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.341951 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:09.352790 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.358633 13873 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:05:09.279889 16624 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:05:09.282416 16419 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:05:09.355964 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.362246 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:09.340308 16672 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:09.353430 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:09.364812 13869 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:09.365152 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.339069 16316 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:09.349109 16667 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:09.365788 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.339627 16623 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:09.339413 16588 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:09.343768 16418 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:09.354995 13988 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:05:09.392215 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.291256 16515 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.349471 16298 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:09.291584 16748 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:09.361169 16325 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:09.354393 16564 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:09.374720 16573 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:09.407210 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.409504 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:09.412323 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 58983.3us
W20260709 13:05:09.411157 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 56244.2us
I20260709 13:05:09.368849 16646 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:09.416851 13941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:05:09.423462 13870 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:09.426853 13869 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:09.419903 13873 raft_consensus.cc:2410] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:09.417850 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:09.401799 14016 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:09.404024 16418 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:09.402119 16588 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.410744 16712 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:09.447988 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.451366 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.409732 16740 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.429416 16573 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:09.457826 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:09.458007 16515 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:09.448596 16638 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:09.459069 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.488190 13873 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:05:09.414870 16582 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:09.417905 16746 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:09.496829 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.473431 13942 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:09.476295 14017 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:09.477980 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.467737 13914 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.451004 16588 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:09.483274 14561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 538) took 1191 ms. Trace:
I20260709 13:05:09.490101 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.499192 13941 raft_consensus.cc:606] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.499594 13941 raft_consensus.cc:614] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:09.499406 13869 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:09.449373 16418 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:09.498804 14561 rpcz_store.cc:276] 0709 13:05:08.291797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.053695 (+761898us) service_pool.cc:224] Handling call
0709 13:05:09.055208 (+  1513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.145777 (+ 90569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.145825 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:09.146830 (+  1005us) log.cc:844] Serialized 26 byte log entry
0709 13:05:09.344945 (+198115us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:09.344995 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.346569 (+  1574us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.483039 (+136470us) raft_consensus.cc:1685] finished
0709 13:05:09.483060 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.483211 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":7263,"spinlock_wait_cycles":53120,"thread_start_us":197686,"threads_started":1,"wal-append.queue_time_us":334251}
I20260709 13:05:09.501274 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:09.468045 14524 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.491436 14024 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:09.418958 16564 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:09.504495 13873 raft_consensus.cc:2368] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8.
I20260709 13:05:09.506956 13914 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.446902 16601 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:05:09.507324 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:09.507314 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:09.509104 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 74960.4us
I20260709 13:05:09.476402 16650 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:09.451994 16278 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:05:09.500478 16746 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.493788 16638 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:09.512773 13874 raft_consensus.cc:2410] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:09.530378 14020 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:09.530494 14017 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:09.532243 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.545351 14017 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:09.512364 13839 leader_election.cc:400] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8.
I20260709 13:05:09.549894 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2187) took 2023 ms. Trace:
W20260709 13:05:09.543177 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 124432us
I20260709 13:05:09.587280 13839 leader_election.cc:403] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:05:09.587435 13871 rpcz_store.cc:276] 0709 13:05:07.526184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.244575 (+718391us) service_pool.cc:224] Handling call
0709 13:05:08.244964 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.552017 (+307053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.549554 (+997537us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54003ac5d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:09.549816 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":704727936}
I20260709 13:05:09.531888 16638 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:09.509431 16756 log.cc:927] Time spent T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:05:09.560652 13873 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:09.510780 16714 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:09.509235 16712 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:09.497359 16566 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:09.562435 16757 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:05:09.566710 16770 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:09.558561 16573 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.568641 16740 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:09.572045 14015 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:09.527119 16746 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:09.510600 16718 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:09.533751 14020 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:09.580468 14554 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:09.616847 14554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 533) took 1674 ms. Trace:
I20260709 13:05:09.581410 16564 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:09.517987 16716 consensus_queue.cc:1059] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:05:09.588559 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:09.589762 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 78243.7us
I20260709 13:05:09.616731 16770 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:09.590723 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:09.640450 16770 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:09.640844 13871 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:09.641042 16770 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:09.646436 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:09.647464 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.648586 16718 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:09.649188 13915 leader_election.cc:400] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8.
I20260709 13:05:09.650542 16740 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:09.651090 16740 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:09.651979 16721 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 1.161s	user 0.003s	sys 0.003s
I20260709 13:05:09.652674 16716 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:09.653128 16716 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:09.653530 16716 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:09.656399 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.660315 13943 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:09.661105 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:09.608206 16712 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.666738 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:09.596500 16755 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:05:09.649525 16718 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:09.709132 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 241140us
I20260709 13:05:09.669047 16712 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:09.715476 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:09.716209 14561 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:09.591002 14019 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:09.553366 16748 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:09.594277 16683 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.597472 16467 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:09.619670 14015 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:09.617449 14554 rpcz_store.cc:276] 0709 13:05:07.942586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.982278 (+ 39692us) service_pool.cc:224] Handling call
0709 13:05:09.579789 (+1597511us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b5800004898. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:05:09.580025 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.616483 (+ 36458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.616764 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31421696}
I20260709 13:05:09.622181 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:09.688918 14021 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:09.691267 16714 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:09.701061 16707 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:09.712555 13871 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:05:09.521207 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:09.727464 13873 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:09.728865 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:09.685253 16667 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:05:09.708047 16765 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:09.712522 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2207) took 1792 ms. Trace:
I20260709 13:05:09.651980 16740 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.745354 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:09.746474 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:09.757439 14015 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:09.759757 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:09.756254 14016 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:09.746435 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.762454 13871 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:09.749143 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:09.760938 14558 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:09.763425 14554 raft_consensus.cc:2410] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:09.753645 13875 rpcz_store.cc:276] 0709 13:05:07.919938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.299361 (+379423us) service_pool.cc:224] Handling call
0709 13:05:08.331460 (+ 32099us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.376290 (+ 44830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.712270 (+1335980us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b54000ea454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:09.712456 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75238784}
I20260709 13:05:09.758529 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:09.756939 14561 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:09.766739 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:09.767637 14558 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:09.770447 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:09.771198 14558 raft_consensus.cc:2376] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 5. Re-sending same reply.
I20260709 13:05:09.772194 14561 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:09.748596 16716 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:09.773010 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.774891 13869 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:09.723580 14019 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:05:09.712517 16702 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:09.791422 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:09.731520 16714 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:09.759756 16765 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:09.820652 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:09.821516 14559 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:09.795363 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.822963 13871 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:09.796388 16566 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:09.800963 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:09.827049 13875 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:09.835693 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:09.837010 13867 raft_consensus.cc:2410] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:09.837271 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:09.838115 14558 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:09.790462 16748 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:09.792317 16738 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:09.755368 16707 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:09.730890 16683 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:10.195829 13869 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:10.351696 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:10.237635 14020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.019s	sys 0.010s
W20260709 13:05:10.352676 14020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.020s	sys 0.011s
W20260709 13:05:10.238121 13916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.017s	sys 0.014s
W20260709 13:05:10.353866 13916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.019s	sys 0.014s
I20260709 13:05:09.877274 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:09.820268 16714 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:10.230566 13949 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.023s	sys 0.020s
W20260709 13:05:10.372599 13949 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.033s	sys 0.021s
I20260709 13:05:10.361899 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2363) took 1095 ms. Trace:
I20260709 13:05:10.373667 13871 rpcz_store.cc:276] 0709 13:05:09.266892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.846878 (+579986us) service_pool.cc:224] Handling call
0709 13:05:10.361803 (+514925us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.375396 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:10.376682 13871 raft_consensus.cc:2410] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:05:10.378028 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:10.379077 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2365) took 1111 ms. Trace:
I20260709 13:05:10.294606 13948 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:10.365272 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 532337us
I20260709 13:05:10.352692 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 386) took 1007 ms. Trace:
I20260709 13:05:10.383641 13869 rpcz_store.cc:276] 0709 13:05:09.345385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.756028 (+410643us) service_pool.cc:224] Handling call
0709 13:05:10.352256 (+596228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.384905 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:10.385632 13869 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:09.905203 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.078862 13989 leader_election.cc:336] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.888s (SENT)
W20260709 13:05:10.362726 14558 consensus_meta.cc:287] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.010s
I20260709 13:05:10.391531 14558 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:10.370429 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:10.371243 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1582) took 1347 ms. Trace:
I20260709 13:05:10.392903 13942 rpcz_store.cc:276] 0709 13:05:09.012209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.012387 (+   178us) service_pool.cc:224] Handling call
0709 13:05:10.359727 (+1347340us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b5400054454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:10.359906 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407214848}
I20260709 13:05:09.824050 16316 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8.
I20260709 13:05:09.821744 16755 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:10.381742 13871 rpcz_store.cc:276] 0709 13:05:09.268181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.377696 (+1109515us) service_pool.cc:224] Handling call
0709 13:05:10.379002 (+  1306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.394358 16755 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:10.395103 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:10.395926 13871 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:05:09.789088 14551 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 2.
W20260709 13:05:10.396270 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 317475us
W20260709 13:05:10.397068 13989 leader_election.cc:336] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.963s (SENT)
I20260709 13:05:10.398070 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2233) took 1053 ms. Trace:
I20260709 13:05:10.398550 13871 rpcz_store.cc:276] 0709 13:05:09.344527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.394714 (+1050187us) service_pool.cc:224] Handling call
0709 13:05:10.397998 (+  3284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.399880 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:10.400898 13871 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:10.299012 16718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.002s	sys 0.026s
W20260709 13:05:10.197880 16428 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.008s	sys 0.028s
I20260709 13:05:10.406172 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2364) took 1139 ms. Trace:
I20260709 13:05:10.420070 13871 rpcz_store.cc:276] 0709 13:05:09.267151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.399536 (+1132385us) service_pool.cc:224] Handling call
0709 13:05:10.406106 (+  6570us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.421417 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:10.422222 13871 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:10.422869 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2362) took 1157 ms. Trace:
W20260709 13:05:10.239632 16758 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:05:09.964965 16761 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 13:05:10.298796 16691 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.014s	sys 0.014s
I20260709 13:05:10.415598 13869 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:10.423295 13871 rpcz_store.cc:276] 0709 13:05:09.265920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.421058 (+1155138us) service_pool.cc:224] Handling call
0709 13:05:10.422790 (+  1732us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.428298 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:10.397138 16755 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:09.873494 16748 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:10.427037 16765 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:10.429304 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2371) took 1157 ms. Trace:
I20260709 13:05:10.442200 13869 rpcz_store.cc:276] 0709 13:05:09.272120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.384578 (+1112458us) service_pool.cc:224] Handling call
0709 13:05:10.429220 (+ 44642us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.443506 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:10.433197 16667 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:10.437263 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 389) took 1076 ms. Trace:
W20260709 13:05:10.440157 13915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.020s	sys 0.049s
I20260709 13:05:10.445217 13871 rpcz_store.cc:276] 0709 13:05:09.360786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.427943 (+1067157us) service_pool.cc:224] Handling call
0709 13:05:10.437184 (+  9241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.452258 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:10.453083 13871 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:05:10.453759 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2381) took 1053 ms. Trace:
I20260709 13:05:10.454181 13871 rpcz_store.cc:276] 0709 13:05:09.400262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.451898 (+1051636us) service_pool.cc:224] Handling call
0709 13:05:10.453679 (+  1781us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.455478 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:10.446451 13914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.015s	sys 0.014s
W20260709 13:05:10.456820 13914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.015s	sys 0.014s
W20260709 13:05:10.449836 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.445366 13915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.020s	sys 0.049s
W20260709 13:05:10.459748 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 793015us
I20260709 13:05:10.444490 13869 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:10.382682 16738 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:10.230890 16325 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.195s
W20260709 13:05:10.408648 16718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.002s	sys 0.027s
W20260709 13:05:10.419970 16428 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.008s	sys 0.033s
I20260709 13:05:10.467640 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2239) took 1119 ms. Trace:
I20260709 13:05:10.472673 13869 rpcz_store.cc:276] 0709 13:05:09.347780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.443157 (+1095377us) service_pool.cc:224] Handling call
0709 13:05:10.467540 (+ 24383us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.474092 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:10.440166 16668 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.475421 13869 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
W20260709 13:05:10.468300 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 328686us
I20260709 13:05:10.483736 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2237) took 1136 ms. Trace:
I20260709 13:05:10.484246 13869 rpcz_store.cc:276] 0709 13:05:09.347161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.473720 (+1126559us) service_pool.cc:224] Handling call
0709 13:05:10.483603 (+  9883us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.485456 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:10.486194 13869 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:10.474602 16770 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:05:10.526044 13870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2367) took 1256 ms (client timeout 1517 ms). Trace:
I20260709 13:05:10.530974 13869 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
W20260709 13:05:10.542301 13870 rpcz_store.cc:269] 0709 13:05:09.269236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.273721 (+  4485us) service_pool.cc:224] Handling call
0709 13:05:10.525801 (+1252080us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54000e9f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:10.525973 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309699456}
I20260709 13:05:10.543540 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:10.544314 13869 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:10.480306 16702 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:10.524250 16746 consensus_meta.cc:287] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.884s	user 0.009s	sys 0.004s
W20260709 13:05:10.532081 16638 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.929s	user 0.013s	sys 0.000s
I20260709 13:05:10.591426 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:10.470340 16667 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:10.598548 13873 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.863s	user 0.012s	sys 0.000s
I20260709 13:05:10.608237 13873 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:10.501245 16668 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:10.609200 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2389) took 1179 ms. Trace:
I20260709 13:05:10.618153 13873 rpcz_store.cc:276] 0709 13:05:09.429907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.595710 (+165803us) service_pool.cc:224] Handling call
0709 13:05:10.609117 (+1013407us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.536522 16770 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:10.620065 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:10.609495 13913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.004s	sys 0.028s
I20260709 13:05:10.622699 13873 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:10.623024 13913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.004s	sys 0.028s
W20260709 13:05:10.622015 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 163300us
I20260709 13:05:10.629077 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:10.607805 16650 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:05:10.637528 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 242719us
W20260709 13:05:10.639008 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 269986us
I20260709 13:05:10.587987 13869 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:10.642831 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2385) took 1236 ms. Trace:
I20260709 13:05:10.643203 13869 rpcz_store.cc:276] 0709 13:05:09.406417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.543170 (+1136753us) service_pool.cc:224] Handling call
0709 13:05:10.642766 (+ 99596us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.629552 14522 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:10.644224 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:10.644982 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2235) took 1299 ms. Trace:
I20260709 13:05:10.645303 13869 rpcz_store.cc:276] 0709 13:05:09.345985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.643943 (+1297958us) service_pool.cc:224] Handling call
0709 13:05:10.644922 (+   979us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:10.646409 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 161305us
I20260709 13:05:10.648162 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:10.649771 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:10.524224 16748 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:10.653043 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.651239 13873 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:10.601920 16746 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:10.661896 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2242) took 1251 ms. Trace:
I20260709 13:05:10.664405 13873 rpcz_store.cc:276] 0709 13:05:09.410215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.619744 (+1209529us) service_pool.cc:224] Handling call
0709 13:05:10.661832 (+ 42088us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.664068 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:10.617157 16638 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:10.668238 14554 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:10.636438 16650 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:10.655378 16683 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:10.714123 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 64444.7us
I20260709 13:05:10.695912 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:10.718665 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:10.763675 13872 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:05:10.700717 16581 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:10.719836 16584 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:10.722093 14553 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:05:10.722652 16785 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:05:10.665294 16707 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:10.789048 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.706293 16552 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:05:10.795954 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 141640us
I20260709 13:05:10.729758 16748 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:10.789206 16672 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:05:10.818277 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 164372us
I20260709 13:05:10.788264 16775 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:10.845834 13872 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:10.802205 16738 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:10.805874 16552 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:10.811542 16707 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:10.860203 16566 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:10.863541 16566 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:10.864845 16428 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.865406 16428 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:10.867784 16428 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:10.800310 16584 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
W20260709 13:05:10.853497 16712 consensus_meta.cc:287] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.135s	user 0.014s	sys 0.000s
I20260709 13:05:10.875690 16808 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.876165 16712 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:10.876345 16808 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:10.878077 16781 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 13:05:10.878260 16797 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:10.762902 16770 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:10.847819 16804 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.874986 16672 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:10.862020 16775 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:05:10.795465 16581 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:10.873842 16787 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.240s	user 0.000s	sys 0.000s
I20260709 13:05:10.883183 16668 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:10.884403 16799 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:05:10.874609 16786 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:05:10.952517 16727 raft_consensus.cc:3055] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:10.886916 16552 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:10.878832 16797 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:10.874567 16746 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:10.900615 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1578) took 2127 ms. Trace:
W20260709 13:05:10.923862 13868 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 383) took 2512 ms (client timeout 1744 ms). Trace:
I20260709 13:05:10.950404 14561 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:05:10.819029 16650 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:10.981940 13867 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:05:11.027926 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1593) took 1369 ms. Trace:
I20260709 13:05:11.028146 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2240) took 1680 ms. Trace:
I20260709 13:05:10.999966 13866 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:05:11.028502 13948 rpcz_store.cc:276] 0709 13:05:09.658565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.663039 (+  4474us) service_pool.cc:224] Handling call
0709 13:05:09.663366 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.381486 (+718120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.999537 (+618051us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b540022a954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:11.027854 (+ 28317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1450988800}
I20260709 13:05:11.029402 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2377) took 1667 ms. Trace:
I20260709 13:05:11.029906 13866 rpcz_store.cc:276] 0709 13:05:09.361578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.999156 (+1637578us) service_pool.cc:224] Handling call
0709 13:05:10.999641 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.029132 (+ 29491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.029346 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.030650 13946 rpcz_store.cc:276] 0709 13:05:08.772696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.839910 (+ 67214us) service_pool.cc:224] Handling call
0709 13:05:08.840258 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.912584 (+ 72326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.900342 (+1987758us) spinlock_profiling.cc:243] Waited 66.7 ms on lock 0x7b54000f5854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:10.900546 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153331584}
I20260709 13:05:11.031571 13866 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:05:11.031669 13868 rpcz_store.cc:269] 0709 13:05:08.389532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.487348 (+ 97816us) service_pool.cc:224] Handling call
0709 13:05:10.901244 (+2413896us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b540026bbd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:10.901387 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":830275840}
I20260709 13:05:11.032373 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2241) took 1633 ms. Trace:
I20260709 13:05:11.034004 13866 rpcz_store.cc:276] 0709 13:05:09.399337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.030893 (+1631556us) service_pool.cc:224] Handling call
0709 13:05:11.031260 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.032137 (+   877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.032321 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.028520 13867 rpcz_store.cc:276] 0709 13:05:09.348098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.981194 (+1633096us) service_pool.cc:224] Handling call
0709 13:05:10.981647 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.027861 (+ 46214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.028092 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.035362 13866 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
W20260709 13:05:11.036127 13989 leader_election.cc:336] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.324s (SENT)
I20260709 13:05:11.036278 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2236) took 1689 ms. Trace:
I20260709 13:05:11.036881 13872 rpcz_store.cc:276] 0709 13:05:09.346829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.811371 (+1464542us) service_pool.cc:224] Handling call
0709 13:05:10.845438 (+ 34067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.846467 (+  1029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.036198 (+189731us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:11.018776 16418 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.508s	user 0.013s	sys 0.001s
W20260709 13:05:11.023484 16714 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.005s
I20260709 13:05:11.054415 13870 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:11.041774 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2378) took 1642 ms. Trace:
I20260709 13:05:11.051836 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1542) took 1744 ms. Trace:
I20260709 13:05:11.056758 14017 rpcz_store.cc:276] 0709 13:05:09.306991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.635918 (+328927us) service_pool.cc:224] Handling call
0709 13:05:09.636277 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.636642 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.636672 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:09.637201 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:05:09.849569 (+212368us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:10.435104 (+585535us) log.cc:844] Serialized 14 byte log entry
0709 13:05:10.622430 (+187326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.623135 (+   705us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.051610 (+428475us) raft_consensus.cc:1685] finished
0709 13:05:11.051632 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.051772 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":846226,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":205606,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":5359,"thread_start_us":817449,"threads_started":3,"wal-append.queue_time_us":1359580}
I20260709 13:05:10.962728 16808 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.020673 16623 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:11.019649 16638 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:11.049393 16467 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.319s	user 0.014s	sys 0.001s
I20260709 13:05:11.056193 13867 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:10.954764 16672 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:10.962785 16727 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:11.038933 13872 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:05:11.061414 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1543) took 1754 ms. Trace:
I20260709 13:05:11.070115 13866 rpcz_store.cc:276] 0709 13:05:09.399448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.034867 (+1635419us) service_pool.cc:224] Handling call
0709 13:05:11.035196 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.035759 (+   563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.041690 (+  5931us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.080147 13872 pending_rounds.cc:85] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 2
I20260709 13:05:11.080845 13872 pending_rounds.cc:107] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:05:11.081501 13872 raft_consensus.cc:2889] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:11.024736 16588 consensus_meta.cc:287] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.516s	user 0.013s	sys 0.000s
I20260709 13:05:10.929318 16804 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.083137 13868 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:11.085981 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 224828us
I20260709 13:05:11.077497 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2384) took 1673 ms. Trace:
W20260709 13:05:11.087113 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 356811us
I20260709 13:05:11.087288 13867 rpcz_store.cc:276] 0709 13:05:09.403766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.035800 (+1632034us) service_pool.cc:224] Handling call
0709 13:05:11.055951 (+ 20151us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.076836 (+ 20885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.077428 (+   592us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.088353 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2387) took 1674 ms. Trace:
I20260709 13:05:11.077250 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:11.078814 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.088790 13872 rpcz_store.cc:276] 0709 13:05:09.413711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.037917 (+1624206us) service_pool.cc:224] Handling call
0709 13:05:11.038481 (+   564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.087832 (+ 49351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.088266 (+   434us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:11.089208 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.082024 14020 rpcz_store.cc:276] 0709 13:05:09.307237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.665893 (+358656us) service_pool.cc:224] Handling call
0709 13:05:09.666230 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:09.666639 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.666669 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:09.685329 (+ 18660us) log.cc:844] Serialized 52 byte log entry
0709 13:05:10.353230 (+667901us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:10.854215 (+500985us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.035812 (+181597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.035915 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.061141 (+ 25226us) raft_consensus.cc:1685] finished
0709 13:05:11.061167 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.061345 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":446698,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":6289,"raft.queue_time_us":37669,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":38827,"thread_start_us":1118435,"threads_started":2,"wal-append.queue_time_us":1216173}
W20260709 13:05:11.073529 16582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.004s	sys 0.009s
I20260709 13:05:11.093372 13866 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:11.057864 16552 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:11.059648 16712 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.107957 16581 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:11.111722 16765 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:11.062573 16623 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.112392 16765 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.115437 16623 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:11.116073 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2234) took 1770 ms. Trace:
I20260709 13:05:11.102623 16807 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:11.083786 16467 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8.
I20260709 13:05:11.105288 16816 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:11.116518 13873 rpcz_store.cc:276] 0709 13:05:09.345224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.665829 (+1320605us) service_pool.cc:224] Handling call
0709 13:05:10.666221 (+   392us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:10.666593 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:10.666622 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:10.704769 (+ 38147us) log.cc:844] Serialized 52 byte log entry
0709 13:05:10.801115 (+ 96346us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:10.811889 (+ 10774us) log.cc:844] Serialized 14 byte log entry
0709 13:05:10.812331 (+   442us) log.cc:844] Serialized 14 byte log entry
0709 13:05:10.812488 (+   157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.812567 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.115797 (+303230us) raft_consensus.cc:1685] finished
0709 13:05:11.115827 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.116003 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17313,"prepare.queue_time_us":49111,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":271371,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":8215,"thread_start_us":84000,"threads_started":1,"wal-append.queue_time_us":374152}
I20260709 13:05:11.086335 16592 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.154439 16592 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.095108 16566 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.093964 16808 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.157161 16808 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.157469 16592 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.159446 16808 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.078550 16325 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
W20260709 13:05:11.093791 16582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.004s	sys 0.009s
I20260709 13:05:11.069619 16755 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:05:11.067641 16418 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.163851 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1574) took 3057 ms. Trace:
I20260709 13:05:11.164304 13950 rpcz_store.cc:276] 0709 13:05:08.106700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.633073 (+526373us) service_pool.cc:224] Handling call
0709 13:05:08.633525 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.988587 (+355062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.163576 (+2174989us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b54001f4fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:11.163774 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":807574272}
I20260709 13:05:11.090015 16819 consensus_queue.cc:1059] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:05:11.166231 14024 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:11.166734 16819 consensus_queue.cc:1243] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:05:11.167246 16819 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:11.167753 16819 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:11.168444 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2250) took 1934 ms. Trace:
I20260709 13:05:11.168586 16819 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.168990 14024 rpcz_store.cc:276] 0709 13:05:09.234418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.511904 (+277486us) service_pool.cc:224] Handling call
0709 13:05:09.512208 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.166774 (+1654566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.168123 (+  1349us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b540023c154. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:11.168367 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3803049856}
I20260709 13:05:11.153251 16797 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.167273 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2245) took 1962 ms. Trace:
I20260709 13:05:11.153559 16816 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:11.059517 16714 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.170830 16816 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.171440 16816 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:11.170452 14022 rpcz_store.cc:276] 0709 13:05:09.205262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.205407 (+   145us) service_pool.cc:224] Handling call
0709 13:05:09.262244 (+ 56837us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.418828 (+156584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.418868 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:09.419386 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 13:05:09.507750 (+ 88364us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:09.507805 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.166866 (+1659061us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400412154. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:11.166914 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.166962 (+    48us) raft_consensus.cc:1685] finished
0709 13:05:11.166976 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.167207 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11559,"raft.run_cpu_time_us":441,"raft.run_wall_time_us":64870,"spinlock_wait_cycles":359201536,"thread_start_us":60534,"threads_started":1,"wal-append.queue_time_us":131956,"wal-append.run_cpu_time_us":1888,"wal-append.run_wall_time_us":1017932}
I20260709 13:05:11.172508 16714 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.174075 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2373) took 1888 ms. Trace:
I20260709 13:05:11.169930 16819 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:11.174535 13870 rpcz_store.cc:276] 0709 13:05:09.287297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.543240 (+1255943us) service_pool.cc:224] Handling call
0709 13:05:11.173823 (+630583us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b54002e77d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:11.173971 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1040403200}
I20260709 13:05:11.094007 16748 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:11.183475 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.185284 13871 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:11.165582 16418 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:11.182528 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 95598.4us
W20260709 13:05:11.189350 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 112210us
I20260709 13:05:11.190917 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:11.082463 16588 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.196725 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:11.189834 16667 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.216202 16679 raft_consensus.cc:3055] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:11.234661 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.189713 16748 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.236706 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.196100 16727 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:11.243774 13839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.005s	sys 0.010s
W20260709 13:05:11.244254 13839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.005s	sys 0.010s
I20260709 13:05:11.248086 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:11.248911 13839 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:11.249851 13839 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:11.244078 16727 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:11.220099 16798 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:11.228220 16807 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:11.259716 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.262728 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.263499 14020 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:11.268007 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.270098 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:11.271981 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.272568 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.251612 16765 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:11.254674 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:11.237080 16679 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:11.253474 16667 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:11.276171 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.253357 16822 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:05:11.279047 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.278971 16418 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:05:11.278213 13839 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:11.280385 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.255776 16798 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:11.281944 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.287070 14023 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:11.253165 16672 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:11.269644 16755 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:05:11.287956 13839 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:11.279829 16667 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.280046 16682 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:11.307874 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:11.300278 14020 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:11.279909 16804 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.298447 16797 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:11.350975 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2247) took 2120 ms. Trace:
I20260709 13:05:11.351608 14018 rpcz_store.cc:276] 0709 13:05:09.230997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.231151 (+   154us) service_pool.cc:224] Handling call
0709 13:05:09.262473 (+ 31322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.351117 (+ 88644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.350608 (+1999491us) spinlock_profiling.cc:243] Waited 80.7 ms on lock 0x7b540021cfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:11.350885 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185703296}
I20260709 13:05:11.298777 16798 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:11.306424 16807 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:11.361917 16807 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.362406 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:11.299397 16822 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:11.363760 16822 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.300146 16672 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.308259 16808 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:11.366624 16755 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:11.367102 16808 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:11.367179 16755 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:11.367616 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.306041 16829 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 13:05:11.342357 16667 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:11.383440 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 139261us
I20260709 13:05:11.366703 16672 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:11.367586 16808 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:11.394712 13870 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
W20260709 13:05:11.397070 13874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2369) took 2126 ms (client timeout 1623 ms). Trace:
W20260709 13:05:11.397506 13874 rpcz_store.cc:269] 0709 13:05:09.270883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.327265 (+ 56382us) service_pool.cc:224] Handling call
0709 13:05:11.396859 (+2069594us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540026b6d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:11.396977 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394999168}
I20260709 13:05:11.370286 16679 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.393394 16588 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:11.403597 16827 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:05:11.385819 16755 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.422602 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36586 (request call id 31) took 2013 ms. Trace:
I20260709 13:05:11.423103 13941 rpcz_store.cc:276] 0709 13:05:09.409595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.409774 (+   179us) service_pool.cc:224] Handling call
0709 13:05:11.422363 (+2012589us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x7b54001c6954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c342d853 00007fb2c83fb60a 00007fb2c19817d4 00007fb2c1981762 00007fb2c1981691 00007fb2c198160c 00007fb2c19808c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:11.422505 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160313728}
W20260709 13:05:11.426566 16831 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:05:11.427964 14017 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:05:11.435374 16788 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.835s	user 0.003s	sys 0.000s
I20260709 13:05:11.438187 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1599) took 1726 ms. Trace:
I20260709 13:05:11.438606 13949 rpcz_store.cc:276] 0709 13:05:09.713102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.713257 (+   155us) service_pool.cc:224] Handling call
0709 13:05:09.713647 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.713882 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:10.381046 (+667164us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.437623 (+1056577us) spinlock_profiling.cc:243] Waited 7.49 ms on lock 0x7b1800298600. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c0348a40 00007fb2c03434e4 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287 00007fb2c344e208
0709 13:05:11.437778 (+   155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:11.437829 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:11.437882 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.437960 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.438123 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34875,"prepare.queue_time_us":687464,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"spinlock_wait_cycles":17233152,"thread_start_us":657430,"threads_started":1}
W20260709 13:05:11.442282 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.442775 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.443863 13946 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:11.445091 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.447119 16718 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:11.447723 16718 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.449268 13839 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:11.450264 16809 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:11.450287 16790 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:11.451732 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:11.452459 16768 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:11.452886 16646 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:11.453440 16646 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:11.456202 16278 raft_consensus.cc:3055] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:11.456612 16278 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:11.459118 14560 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:11.459214 16650 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:11.459811 14561 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:11.460847 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:11.461253 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.461392 16691 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:11.461946 13943 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:05:11.462193 16740 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:11.461997 16691 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.461957 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.462954 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.463119 13947 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:11.463639 13943 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:05:11.463879 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.464357 16691 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.461442 13942 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:05:11.462807 16740 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.452109 16790 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.479882 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.466375 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.481508 13946 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:11.482728 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.483805 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.472301 13944 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:11.453009 16768 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.474390 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:11.476336 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:11.476778 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.485795 13866 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:11.479027 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.488353 13949 raft_consensus.cc:2410] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:05:11.478297 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.488996 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.466019 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:11.490131 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:11.490958 14561 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:11.491338 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.491712 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:11.492120 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:11.493847 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:11.480772 13943 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:11.484345 13946 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:11.496850 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:11.497675 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:11.443967 16820 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:05:11.457434 16278 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.413724 16748 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:11.513516 16278 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:11.515264 16740 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.517863 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:11.519075 16712 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:11.502714 16620 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.522547 16620 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.524971 16620 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:11.526134 16638 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:11.526445 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2231) took 2183 ms. Trace:
I20260709 13:05:11.526648 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:11.460877 16714 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:11.526880 13869 rpcz_store.cc:276] 0709 13:05:09.343889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.645916 (+1302027us) service_pool.cc:224] Handling call
0709 13:05:10.654712 (+  8796us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:10.786611 (+131899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:10.786646 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:10.790358 (+  3712us) log.cc:844] Serialized 26 byte log entry
0709 13:05:11.021734 (+231376us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b48002406d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:05:11.200289 (+178555us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:11.200349 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.200437 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.526131 (+325694us) raft_consensus.cc:1685] finished
0709 13:05:11.526155 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.526383 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64661,"raft.queue_time_us":150772,"raft.run_cpu_time_us":4083,"raft.run_wall_time_us":207667,"spinlock_wait_cycles":8821632,"thread_start_us":461119,"threads_started":2,"wal-append.queue_time_us":418142}
I20260709 13:05:11.528121 16714 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.528735 16638 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:11.461253 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:11.529771 16638 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.530764 14553 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:11.531049 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:11.532593 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.533340 14018 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:05:11.532624 14554 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:11.534973 16790 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:11.535070 13914 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:11.536351 16667 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.536901 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.537649 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:11.433614 16816 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.539760 16672 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.503413 16690 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:05:11.541191 16690 raft_consensus.cc:3055] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:11.541543 16690 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:11.542098 16816 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:11.542493 16690 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.543872 16690 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:11.551736 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.553287 16679 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:11.555307 16714 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:11.555984 16714 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:11.556391 16714 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:11.556634 13871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 387) took 2205 ms (client timeout 2173 ms). Trace:
I20260709 13:05:11.556857 16714 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:11.557372 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.559612 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:11.560397 14559 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:11.561825 13914 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:11.504690 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:11.562675 14561 raft_consensus.cc:2376] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8. Re-sending same reply.
I20260709 13:05:11.562781 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.505976 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:11.557078 13871 rpcz_store.cc:269] 0709 13:05:09.351269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.455029 (+1103760us) service_pool.cc:224] Handling call
0709 13:05:11.556438 (+1101409us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54000a9454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:11.556557 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1627855232}
I20260709 13:05:11.506296 14524 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:11.566217 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.566406 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 60226.5us
I20260709 13:05:11.505597 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:11.511498 13942 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:05:11.504704 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.568193 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.479229 16809 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.569079 13944 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:11.571234 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.573356 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.573683 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.578346 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:11.581558 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.581668 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.586079 14023 raft_consensus.cc:2410] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:11.449926 16718 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:11.594722 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.595662 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:11.599550 13866 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:11.600586 13874 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:05:11.601699 14020 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:11.584777 13872 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:11.602774 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:11.601758 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.603942 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:11.604041 14024 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:11.604638 13872 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:11.605381 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:11.605933 13866 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:11.606515 14554 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
W20260709 13:05:11.607383 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.608494 13987 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:11.561964 16763 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 2.034s	user 0.013s	sys 0.022s
I20260709 13:05:11.517858 16716 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.616531 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:11.611368 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:11.612046 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:11.612389 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:11.614575 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.620944 14018 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:05:11.616057 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:11.622025 13868 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:11.610718 14015 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:11.623410 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:11.617744 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:11.630230 16768 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.630599 16797 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:11.630955 16813 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:11.631536 16813 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:11.631969 16755 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:11.632441 16813 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.588788 16771 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:11.574519 16672 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:11.634994 13988 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:11.635772 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.635874 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:11.650326 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:11.650522 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.651273 13942 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:11.647073 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:11.652209 13868 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:11.633658 14561 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.653635 14522 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:11.588634 16573 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:11.634647 16552 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.533s	user 0.012s	sys 0.000s
I20260709 13:05:11.661907 14522 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:11.619621 16770 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:05:11.660207 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:11.677632 13771 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54072 (request call id 136) took 18922 ms (client timeout 15000 ms). Trace:
W20260709 13:05:11.677917 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.677671 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:11.678318 13866 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:11.660157 16836 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:11.658841 16682 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:05:11.655839 16797 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:05:11.668309 16814 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:11.646354 16771 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:05:11.714866 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.730933 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.732607 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:11.710178 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.706369 16814 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:11.687796 13868 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
W20260709 13:05:11.678195 13771 rpcz_store.cc:269] 0709 13:04:52.725907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.730152 (+  4245us) service_pool.cc:224] Handling call
0709 13:05:11.648337 (+18918185us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:04.020202 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:04.098906 (+ 78704us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:04.115758 (+ 16852us) write_op.cc:435] Acquired schema lock
0709 13:05:04.115803 (+    45us) tablet.cc:662] Decoding operations
0709 13:05:04.117064 (+  1261us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:04.117225 (+   161us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:04.117278 (+    53us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:05:04.197634 (+ 80356us) tablet.cc:701] Row locks acquired
0709 13:05:04.218788 (+ 21154us) write_op.cc:260] PREPARE: finished
0709 13:05:04.237627 (+ 18839us) write_op.cc:270] Start()
0709 13:05:04.270167 (+ 32540us) write_op.cc:276] Timestamp: P: 1783602304231315 usec, L: 0
0709 13:05:04.274005 (+  3838us) op_driver.cc:348] REPLICATION: starting
0709 13:05:04.354470 (+ 80465us) log.cc:844] Serialized 2898 byte log entry
0709 13:05:05.138890 (+784420us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:05.861192 (+722302us) op_driver.cc:464] REPLICATION: finished
0709 13:05:06.290905 (+429713us) write_op.cc:301] APPLY: starting
0709 13:05:06.308478 (+ 17573us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:06.341767 (+ 33289us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:06.351147 (+  9380us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:06.515279 (+164132us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:06.515661 (+   382us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:05:06.515726 (+    65us) write_op.cc:312] APPLY: finished
0709 13:05:06.516104 (+   378us) log.cc:844] Serialized 59 byte log entry
0709 13:05:06.598157 (+ 82053us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:06.634851 (+ 36694us) write_op.cc:454] Released schema lock
0709 13:05:06.640444 (+  5593us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:06.641012 (+   568us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:06.641062 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2468651,"reported_tablets":15,"spinlock_wait_cycles":1888640,"child_traces":[["op",{"apply.queue_time_us":385899,"apply.run_cpu_time_us":30261,"apply.run_wall_time_us":364501,"num_ops":7,"prepare.queue_time_us":121653,"prepare.run_cpu_time_us":20188,"prepare.run_wall_time_us":438999,"raft.queue_time_us":469468,"raft.run_cpu_time_us":35593,"raft.run_wall_time_us":526844,"replication_time_us":1408636,"spinlock_wait_cycles":7762304,"thread_start_us":658254,"threads_started":5,"wal-append.queue_time_us":529129,"wal-append.run_cpu_time_us":30205,"wal-append.run_wall_time_us":3467657}]]}
I20260709 13:05:11.723172 16797 raft_consensus.cc:3055] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:11.735347 16718 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:11.735970 13868 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:05:11.735922 16718 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.737423 13868 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:05:11.738204 16790 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:11.738730 16790 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.630236 16798 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.655606 16573 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:11.625731 16819 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:05:11.734416 13942 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:11.788882 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:11.745738 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:05:11.790421 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 44813.3us
I20260709 13:05:11.791621 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:11.796136 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:11.727314 16716 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.798738 16716 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.799415 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.800376 13942 raft_consensus.cc:2410] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:11.800930 16716 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.801158 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.802467 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.802996 13942 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:11.803843 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:11.804391 13942 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:11.748914 16804 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:05:11.805275 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:11.806339 13942 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:11.753885 16718 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.741850 16740 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.810616 16646 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.811280 16740 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:11.683276 16768 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:11.813093 16768 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.813558 16768 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:11.741209 16790 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:11.682337 16837 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:11.815666 16837 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:11.818732 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:11.655582 16727 consensus_queue.cc:1059] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:05:11.601342 16809 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.710386 16672 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:11.837981 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:11.681666 16623 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:11.742033 16738 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:11.839277 16777 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:11.842512 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:11.870090 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:11.741703 16712 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:11.745086 16771 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:11.852732 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.788590 16748 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.797924 16573 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:11.742851 16797 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:11.743417 16814 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:11.871788 13871 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:05:11.873144 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:11.894733 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.831935 16794 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:11.853979 16825 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:11.840708 16798 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:11.843392 16740 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:11.816901 16837 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:11.924088 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 66983.4us
I20260709 13:05:11.855242 16672 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.925123 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:05:11.932044 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 59037.9us
I20260709 13:05:11.953982 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:11.867265 16775 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:11.868155 16777 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:11.960294 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.922652 16790 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:11.922338 16667 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:11.979040 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.980077 16753 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.988441 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:11.924273 16844 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:11.951653 16738 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:11.927659 16712 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:11.998512 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:11.961244 16672 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:11.974673 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 312498us
I20260709 13:05:11.959573 16775 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:12.013682 14522 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:12.005250 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 325507us
I20260709 13:05:11.965570 16777 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:12.018275 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:12.010821 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:11.998263 16740 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:11.948704 16748 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.024641 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:12.009253 13942 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:12.003726 16790 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:12.004583 16738 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:12.029958 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.031647 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.032639 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.033167 13943 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:12.015993 16650 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:05:12.028843 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.016220 16718 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:12.016191 16775 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:12.036772 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.046365 13943 raft_consensus.cc:2376] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7. Re-sending same reply.
I20260709 13:05:12.047348 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:12.048388 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.027604 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:12.021848 16770 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:12.049401 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:12.025311 16716 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:12.058154 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.060652 13944 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:12.061607 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.064237 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:12.066983 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:12.073935 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.026952 16790 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:12.034818 16738 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:12.035887 16566 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:12.018954 16714 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.079732 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.082046 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.001842 16753 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.088979 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.092947 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.056800 16316 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:11.522656 16691 consensus_queue.cc:1175] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" responder_term: 8 status { last_received { term: 7 index: 7 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:12.093973 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:12.191519 13948 raft_consensus.cc:2376] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8. Re-sending same reply.
W20260709 13:05:12.095395 16849 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:05:12.070874 16278 consensus_meta.cc:287] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.002s
I20260709 13:05:12.044404 16650 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:12.099467 13943 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:12.172500 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:12.100147 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:12.103332 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:12.062758 16770 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:12.069460 16718 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:12.088973 16790 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:12.173653 13946 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:12.020516 16797 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.103557 16836 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:12.192576 13942 raft_consensus.cc:2393] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:12.201481 16740 raft_consensus.cc:697] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:12.184765 16843 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
W20260709 13:05:12.218389 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 115223us
I20260709 13:05:12.179396 16853 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:12.223069 16770 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:12.223229 16601 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:12.249563 13950 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:12.251264 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 394) took 1530 ms. Trace:
I20260709 13:05:12.267294 13867 rpcz_store.cc:276] 0709 13:05:10.721074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.088431 (+367357us) service_pool.cc:224] Handling call
0709 13:05:11.101784 (+ 13353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.259554 (+157770us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:11.259862 (+   308us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.384022 (+124160us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:11.384086 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:11.384144 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.250920 (+866776us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54003ac0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:12.251004 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.251190 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1837,"prepare.queue_time_us":594,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":2735,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":4422,"spinlock_wait_cycles":362112896,"thread_start_us":121896,"threads_started":1,"wal-append.queue_time_us":145854}
W20260709 13:05:12.235822 16808 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.837s	user 0.011s	sys 0.000s
I20260709 13:05:12.246502 16650 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.257565 13987 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:12.271061 16601 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:12.272361 16861 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:05:12.273430 16848 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
W20260709 13:05:12.274610 16851 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:05:12.309340 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:12.276237 16679 consensus_meta.cc:287] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.722s	user 0.011s	sys 0.000s
I20260709 13:05:12.283444 13867 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:05:12.248186 16836 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:12.307019 16690 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:12.323027 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:12.323446 16690 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.323904 16690 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:12.324025 16755 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:12.323639 16601 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.324817 16755 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:12.319770 13872 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:12.297212 16797 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:12.330689 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 320364us
I20260709 13:05:12.331686 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:12.332657 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:12.315773 16650 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:12.314226 14553 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:12.321200 16836 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:12.341933 14555 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
W20260709 13:05:12.347098 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.319007 16712 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:12.304683 16808 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.340651 16584 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:12.331535 16748 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:12.337451 16868 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:12.336527 16755 consensus_queue.cc:1059] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:05:12.400019 13875 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:12.401650 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:12.405443 16808 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:12.402252 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.392904 16714 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:12.338670 16718 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.424664 13946 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:12.400236 16753 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:12.385610 16837 consensus_queue.cc:1059] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:05:12.393016 16712 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:05:12.390113 16584 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.349244 16740 consensus_queue.cc:237] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.444036 16867 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:12.415376 16868 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:12.449162 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 114550us
I20260709 13:05:12.451313 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:12.439826 16858 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:05:12.456967 16878 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:12.459672 16316 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:12.463600 16869 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:05:12.464027 16867 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:12.476564 16751 raft_consensus.cc:3055] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:12.477030 16867 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:12.480798 16872 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:05:12.467265 16768 consensus_meta.cc:287] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.653s	user 0.014s	sys 0.000s
W20260709 13:05:12.494192 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.495879 16841 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.500399 14015 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:12.500747 16714 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:12.501506 16714 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:12.434497 16864 consensus_queue.cc:1059] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:05:12.477061 16751 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:12.511407 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 02dcf286c6bd48e3a6e548def7d786cc with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 4dd265d61fc141a48ca9d3d5c8f52921 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:12.454926 16712 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:12.513041 16712 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:12.512740 16751 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:12.514431 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.493198 16573 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.538s	user 0.012s	sys 0.000s
I20260709 13:05:12.502739 16714 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:12.535302 16873 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:05:12.519276 16751 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:12.540948 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 89688.2us
W20260709 13:05:12.538123 16844 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.001s
I20260709 13:05:12.500768 16841 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.519454 16768 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.521929 16775 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.557806 16844 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:12.514194 16712 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.529605 16573 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.522622 16852 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:05:12.576251 13915 connection.cc:256] RPC call timeout handler was delayed by -2.071509954000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
I20260709 13:05:12.580235 14558 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:05:12.580188 13915 connection.cc:256] RPC call timeout handler was delayed by -1.8150217830000202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
W20260709 13:05:12.580842 13915 connection.cc:256] RPC call timeout handler was delayed by -1.7043217829999548s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1978s for a response
W20260709 13:05:12.581269 13915 connection.cc:256] RPC call timeout handler was delayed by -1.6818217830000322s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
W20260709 13:05:12.582599 13915 connection.cc:256] RPC call timeout handler was delayed by -1.6170217830000411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
W20260709 13:05:12.582981 13915 connection.cc:256] RPC call timeout handler was delayed by -1.564395617999935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1748s for a response
W20260709 13:05:12.583280 13915 connection.cc:256] RPC call timeout handler was delayed by -1.4300260360000721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
W20260709 13:05:12.583576 13915 connection.cc:256] RPC call timeout handler was delayed by -1.2489492729999938s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19140000000000001s for a response
I20260709 13:05:12.584379 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:05:12.570029 16716 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.002s
I20260709 13:05:12.599351 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:12.577041 16853 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.602030 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:05:12.603258 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 92134.3us
I20260709 13:05:12.605167 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:12.605099 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:12.605554 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.605966 14015 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:12.609412 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:12.546694 16868 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.612413 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:12.607947 16884 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.614621 16884 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.616907 16884 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:12.617437 16884 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.618218 16584 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:12.620245 16877 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:12.620817 16877 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:12.621717 16877 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.624317 14018 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:12.625483 16879 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:12.626735 14018 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:12.626996 16879 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:12.628399 16879 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:12.628490 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:12.629498 16878 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:12.629912 16878 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:12.631310 14018 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:12.598655 16716 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:12.600634 16844 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:12.636471 16886 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.636487 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:12.638010 16573 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:12.658614 16836 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.637938 16646 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:12.641012 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:12.641345 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:12.662725 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:12.663152 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 450) took 1203 ms. Trace:
I20260709 13:05:12.663427 14022 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:12.663767 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:12.617889 16884 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.642575 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:12.664649 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2243) took 1097 ms. Trace:
I20260709 13:05:12.665006 13871 rpcz_store.cc:276] 0709 13:05:11.567080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.567209 (+   129us) service_pool.cc:224] Handling call
0709 13:05:11.567572 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.930548 (+362976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.664407 (+733859us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b540026cad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:12.664585 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":699057920}
I20260709 13:05:12.665952 14024 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 3: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:12.666586 16794 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.707s	user 0.009s	sys 0.004s
I20260709 13:05:12.667025 16794 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:12.644997 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:12.667801 13947 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:12.668022 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:12.643348 16768 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:12.647413 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:12.671227 16775 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:12.672536 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:12.673192 14015 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:12.667795 14016 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
W20260709 13:05:12.674086 16777 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.649s	user 0.011s	sys 0.001s
I20260709 13:05:12.647471 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 443) took 4404 ms. Trace:
I20260709 13:05:12.663550 13950 rpcz_store.cc:276] 0709 13:05:11.459684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.465774 (+  6090us) service_pool.cc:224] Handling call
0709 13:05:12.662984 (+1197210us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b5400228654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:12.663091 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1672407296}
I20260709 13:05:12.676240 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.676182 16777 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.677170 13950 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:12.677481 13948 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:12.604666 16864 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:12.674660 13945 rpcz_store.cc:276] 0709 13:05:08.243256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.643061 (+399805us) service_pool.cc:224] Handling call
0709 13:05:09.171189 (+528128us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.171641 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.171700 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:09.172307 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 13:05:09.435625 (+263318us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:09.435665 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.435735 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.561329 (+2125594us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:11.743598 (+182269us) raft_consensus.cc:1685] finished
0709 13:05:11.743627 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.647267 (+903640us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400229f50. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:12.647395 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1226775168,"thread_start_us":262872,"threads_started":1,"wal-append.queue_time_us":322822}
I20260709 13:05:12.651072 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2248) took 1075 ms. Trace:
I20260709 13:05:12.652345 16841 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:12.600601 16712 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:12.655570 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.656800 16552 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:12.637943 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:12.665894 16884 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:12.715651 16552 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:12.715955 14018 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:12.716699 16552 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.647024 14556 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:05:12.650771 16862 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.323s	user 0.003s	sys 0.000s
I20260709 13:05:12.682909 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:12.718521 13945 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:12.675444 16814 consensus_meta.cc:287] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.714s	user 0.014s	sys 0.000s
I20260709 13:05:12.608646 16853 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.710481 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:12.734408 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:12.735997 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.712169 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2262) took 1131 ms. Trace:
I20260709 13:05:12.736917 13872 rpcz_store.cc:276] 0709 13:05:11.580697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.659928 (+ 79231us) service_pool.cc:224] Handling call
0709 13:05:12.694403 (+1034475us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b54004619d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:12.712095 (+ 17692us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1494824832}
I20260709 13:05:12.682718 13875 rpcz_store.cc:276] 0709 13:05:11.576076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.576235 (+   159us) service_pool.cc:224] Handling call
0709 13:05:11.612093 (+ 35858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.403068 (+790975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.650829 (+247761us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54004510d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:12.651009 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811537152}
I20260709 13:05:12.720520 16770 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.722134 16777 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:12.722599 13941 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:12.722595 16844 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:12.725118 16771 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.775s	user 0.011s	sys 0.000s
I20260709 13:05:12.725561 16864 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:12.726743 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2225) took 1182 ms. Trace:
I20260709 13:05:12.769549 13948 rpcz_store.cc:276] 0709 13:05:11.544051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.225208 (+681157us) service_pool.cc:224] Handling call
0709 13:05:12.311198 (+ 85990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.677869 (+366671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.726493 (+ 48624us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b5400090454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:12.726674 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":843294464}
I20260709 13:05:12.727674 14554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 629) took 1191 ms. Trace:
I20260709 13:05:12.729535 14522 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:12.770094 16864 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.770736 14554 rpcz_store.cc:276] 0709 13:05:11.536749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.537381 (+   632us) service_pool.cc:224] Handling call
0709 13:05:12.727478 (+1190097us) spinlock_profiling.cc:243] Waited 68.2 ms on lock 0x7b54005609d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:12.727610 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156852608}
I20260709 13:05:12.772188 16865 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:12.773032 13914 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:12.772789 16844 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.774250 16777 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:12.774657 16777 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:12.775038 16777 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:12.735910 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.714941 16316 raft_consensus.cc:3055] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:12.663198 16836 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:12.713475 16672 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.693s	user 0.003s	sys 0.009s
I20260709 13:05:12.735949 13945 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:12.783447 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.783607 14522 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:12.710487 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.767740 16770 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:12.773321 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.777050 13944 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:05:12.718626 16798 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:12.814904 16798 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:12.815070 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:12.815335 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:12.815794 13941 raft_consensus.cc:2376] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8. Re-sending same reply.
I20260709 13:05:12.816236 16798 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.781559 16316 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:12.818553 16798 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:12.819157 16316 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.820681 16316 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:12.821334 14522 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:12.816253 13944 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 5: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:12.824909 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:12.740691 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2215) took 1270 ms. Trace:
I20260709 13:05:12.804531 16865 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:12.682107 16886 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.740461 16690 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.665948 16790 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:12.807699 16738 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.703s	user 0.013s	sys 0.000s
I20260709 13:05:12.816840 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:12.839610 13941 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:12.768208 16771 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.826300 13949 rpcz_store.cc:276] 0709 13:05:11.470241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.538194 (+ 67953us) service_pool.cc:224] Handling call
0709 13:05:12.690686 (+1152492us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b5800202e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:12.740620 (+ 49934us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45460352}
W20260709 13:05:12.828963 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.845264 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:12.740271 16841 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:05:12.744226 16814 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.849588 14020 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:12.844592 16865 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:12.853897 14555 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:12.844012 16770 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:05:12.866647 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 243862us
I20260709 13:05:12.827224 16672 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.884513 14559 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
W20260709 13:05:12.856060 16847 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.632s	user 0.001s	sys 0.001s
W20260709 13:05:12.890723 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46330.6us
I20260709 13:05:12.891742 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:12.892401 13942 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:12.856098 16690 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:12.846443 16829 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:05:12.908004 16814 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:12.765687 16878 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.910840 16878 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:12.911499 16672 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:12.855211 16738 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.913341 16790 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:12.913650 16738 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:12.914319 13946 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:12.884398 16821 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.922622 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:12.887569 16884 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.898573 16852 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.896469 16837 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:12.921861 16714 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:12.929095 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:12.929981 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:12.940781 14018 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:05:12.930380 14016 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:05:12.930948 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:12.941458 14016 pending_rounds.cc:85] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 6
I20260709 13:05:12.941713 14016 pending_rounds.cc:107] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:05:12.941937 14016 raft_consensus.cc:2889] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:12.942431 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.894256 16770 raft_consensus.cc:697] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:12.932776 14015 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:12.931870 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:12.944365 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:12.889178 16646 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:05:12.888975 16883 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:05:12.948186 13944 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:12.949142 16868 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:12.898468 16278 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:05:12.954073 16868 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.956904 16853 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:12.958420 16868 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:12.951766 16889 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:12.964154 16584 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.964735 16584 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.946112 16790 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:12.969696 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:12.970762 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:12.972167 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:12.972880 14017 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:12.971686 14023 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:12.972221 14019 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:12.975615 14015 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:12.950976 16891 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:12.939759 16884 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.982366 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:12.988072 14017 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:12.925194 16821 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:12.959600 16889 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.965386 16770 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.966958 16844 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:12.996534 14019 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:12.979389 16790 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:12.979938 16891 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:13.011533 16891 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:12.856062 16865 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:12.938975 16852 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.008380 16771 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:13.029487 14018 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:13.000864 16886 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:13.046393 16790 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.048264 16821 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:13.053610 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2247) took 1478 ms. Trace:
I20260709 13:05:13.054039 13873 rpcz_store.cc:276] 0709 13:05:11.574842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.575000 (+   158us) service_pool.cc:224] Handling call
0709 13:05:11.575397 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.584314 (+  8917us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:11.616766 (+ 32452us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.012374 (+395608us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:12.018895 (+  6521us) log.cc:844] Serialized 26 byte log entry
0709 13:05:12.019486 (+   591us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:12.022007 (+  2521us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.611474 (+589467us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b48002406d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:05:13.052904 (+441430us) spinlock_profiling.cc:243] Waited 6.81 ms on lock 0x7b1800243640. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c730758f 00007fb2c0346571 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd
0709 13:05:13.053244 (+   340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.053338 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:13.053391 (+    53us) raft_consensus.cc:1685] finished
0709 13:05:13.053406 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.053545 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1046077,"prepare.queue_time_us":33329,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":81,"raft.queue_time_us":14221,"raft.run_cpu_time_us":14164,"raft.run_wall_time_us":1027993,"spinlock_wait_cycles":125998208,"thread_start_us":372102,"threads_started":1,"wal-append.queue_time_us":470917}
W20260709 13:05:13.056582 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 1.02895e+06us
I20260709 13:05:13.057632 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:13.000468 16844 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:13.065333 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.066951 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.065793 13989 leader_election.cc:336] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.563s (SENT)
I20260709 13:05:13.067849 13989 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:13.068944 13989 leader_election.cc:336] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.608s (SENT)
I20260709 13:05:13.069156 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:13.069887 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.063781 16806 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:13.059039 16667 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:13.072141 13871 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:13.076856 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:13.075500 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.078334 13871 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:13.080526 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:13.081346 16777 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.067068 16584 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:13.085057 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2224) took 1541 ms. Trace:
I20260709 13:05:13.086885 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:13.103160 13947 rpcz_store.cc:276] 0709 13:05:11.543748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.223861 (+680113us) service_pool.cc:224] Handling call
0709 13:05:12.228639 (+  4778us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.668179 (+439540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.084806 (+416627us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b5400090e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:13.084991 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009320320}
I20260709 13:05:13.104290 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:13.105036 13941 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:05:13.087317 13838 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:13.088619 16850 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.986s	user 0.000s	sys 0.013s
I20260709 13:05:13.053069 16852 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:13.111195 13872 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:13.072731 16821 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:13.089795 16856 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.094337 16753 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.062175 16844 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:13.072642 16679 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:13.123345 16884 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:13.116115 16852 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:13.123544 16679 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.124264 16852 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:13.104557 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.087880 16672 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.105930 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:13.130939 16809 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:13.119692 16751 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.573s	user 0.006s	sys 0.005s
I20260709 13:05:13.111755 16801 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:13.122838 16856 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.140589 16856 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:13.143493 16859 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.827s	user 0.003s	sys 0.000s
I20260709 13:05:13.133900 16672 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.146845 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2232) took 1353 ms. Trace:
I20260709 13:05:13.147714 16672 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:13.113224 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:13.123488 16889 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:13.147322 13946 rpcz_store.cc:276] 0709 13:05:11.793800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.431375 (+637575us) service_pool.cc:224] Handling call
0709 13:05:12.431621 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.917857 (+486236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.146569 (+228712us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b54000f5354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:13.146778 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1109450112}
I20260709 13:05:13.130250 16679 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:13.121519 16316 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:05:13.156565 16852 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:13.126829 16753 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.159237 16753 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:13.161696 16891 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.165652 16891 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:13.169108 16900 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:05:13.175396 16714 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:13.178823 16714 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:13.181583 16777 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:13.176730 16844 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.191532 16844 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:13.195583 16775 consensus_queue.cc:1175] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3779649036364744823fc035dac892c7" responder_term: 6 status { last_received { term: 6 index: 7 } last_committed_idx: 6 last_received_current_leader { term: 6 index: 7 } }
I20260709 13:05:13.198189 16601 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:13.198680 16601 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:13.239337 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 151830us
I20260709 13:05:13.096045 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:13.243335 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 147179us
I20260709 13:05:13.251399 13773 catalog_manager.cc:5697] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:13.257252 13773 catalog_manager.cc:5697] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 3 to 5, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 5 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.258189 13838 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:13.262269 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:13.261389 13773 catalog_manager.cc:5697] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 3, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.264251 13773 catalog_manager.cc:5697] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 5, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 5 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.269538 13773 catalog_manager.cc:5697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 1 to 4, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 4 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.271661 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.272537 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.274309 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.272650 13773 catalog_manager.cc:5697] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:13.275503 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:13.276039 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.276203 13773 catalog_manager.cc:5697] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 4, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 4 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:13.277275 13867 raft_consensus.cc:2410] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:13.277459 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.278201 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.271682 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2259) took 1691 ms. Trace:
I20260709 13:05:13.274380 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.279172 13867 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:05:13.279168 13866 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:13.280563 13869 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:13.282855 13773 catalog_manager.cc:5697] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 3, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.283775 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.285620 13773 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 3, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.279263 13868 rpcz_store.cc:276] 0709 13:05:11.579828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.738277 (+158449us) service_pool.cc:224] Handling call
0709 13:05:11.738568 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.746401 (+  7833us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:11.809228 (+ 62827us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.649621 (+840393us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:13.058850 (+409229us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.059415 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 13:05:13.059638 (+   223us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:13.059691 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.059786 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:13.270787 (+211001us) raft_consensus.cc:1685] finished
0709 13:05:13.270999 (+   212us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.271384 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":996157,"prepare.queue_time_us":62917,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":1002057,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":50109,"spinlock_wait_cycles":7822208,"thread_start_us":259806,"threads_started":1,"wal-append.queue_time_us":414792}
I20260709 13:05:13.289708 13872 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:13.097365 13873 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:13.095852 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:13.307072 13875 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:13.288955 13773 catalog_manager.cc:5697] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 5, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 5 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.132176 13941 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 9 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:05:13.332731 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2247) took 1268 ms. Trace:
I20260709 13:05:13.199781 16601 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.336052 16601 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:13.336580 13950 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:13.333323 13944 rpcz_store.cc:276] 0709 13:05:12.064082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.835877 (+771795us) service_pool.cc:224] Handling call
0709 13:05:12.836298 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.952447 (+116149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.332058 (+379611us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54001c6954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:13.332487 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256859008}
I20260709 13:05:13.342495 16806 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:13.345273 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2249) took 1280 ms. Trace:
I20260709 13:05:13.346385 16801 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:13.347380 14559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 561) took 1254 ms. Trace:
I20260709 13:05:13.347958 14559 rpcz_store.cc:276] 0709 13:05:12.092959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.093141 (+   182us) service_pool.cc:224] Handling call
0709 13:05:12.093476 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.885085 (+791609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.346673 (+461588us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b54003372d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:13.347101 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1818632576}
I20260709 13:05:13.342581 13950 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:13.351197 13948 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:13.351209 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2235) took 1556 ms. Trace:
I20260709 13:05:13.352659 13942 rpcz_store.cc:276] 0709 13:05:11.794644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.594302 (+799658us) service_pool.cc:224] Handling call
0709 13:05:12.594766 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.892977 (+298211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.350643 (+457666us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b5400087854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:13.350966 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":683786496}
I20260709 13:05:13.353241 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:13.351210 16753 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:13.354195 13871 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:13.354362 16753 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.354583 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:13.354914 16753 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.355638 13866 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:13.358685 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:13.361756 13869 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:13.362886 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:13.261664 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:13.297619 16714 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.304301 13947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:05:13.322805 16808 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:13.324110 16682 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:13.329046 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:13.371363 16714 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:13.371546 14553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 631) took 1530 ms. Trace:
I20260709 13:05:13.372049 14553 rpcz_store.cc:276] 0709 13:05:11.841400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.841591 (+   191us) service_pool.cc:224] Handling call
0709 13:05:13.371026 (+1529435us) spinlock_profiling.cc:243] Waited 83.9 ms on lock 0x7b54003d72d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:13.371280 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192924032}
I20260709 13:05:13.379063 13943 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:13.374514 16771 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:13.379047 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:13.370021 16682 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.385339 16801 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:13.385460 13773 catalog_manager.cc:5697] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 1 to 2, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.383882 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.387145 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 304) took 1054 ms. Trace:
I20260709 13:05:13.396135 14022 rpcz_store.cc:276] 0709 13:05:12.332629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.332743 (+   114us) service_pool.cc:224] Handling call
0709 13:05:12.333073 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.678045 (+344972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.386493 (+708448us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54002eb654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:13.386855 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":759174016}
I20260709 13:05:13.396925 16879 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:13.397095 16682 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:13.406546 13873 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 2.
I20260709 13:05:13.404732 16879 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:13.417176 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.397358 16801 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:13.367074 13949 rpcz_store.cc:276] 0709 13:05:12.064603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.840688 (+776085us) service_pool.cc:224] Handling call
0709 13:05:12.841140 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:12.846689 (+  5549us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:12.846731 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:12.847739 (+  1008us) log.cc:844] Serialized 26 byte log entry
0709 13:05:12.971637 (+123898us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:12.971680 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.971756 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:13.168368 (+196612us) spinlock_profiling.cc:243] Waited 6.09 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:13.344341 (+175973us) raft_consensus.cc:1685] finished
0709 13:05:13.344418 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.344970 (+   552us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6425,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":13284,"spinlock_wait_cycles":14011136,"thread_start_us":75682,"threads_started":1,"wal-append.queue_time_us":133815}
I20260709 13:05:13.422952 16552 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:13.421123 13868 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:13.428960 13947 raft_consensus.cc:606] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:13.429747 13947 raft_consensus.cc:614] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:13.433166 13875 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:13.437407 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:13.437554 16777 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:05:13.439343 13941 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:05:13.439616 16777 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:13.439726 16768 raft_consensus.cc:3055] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:13.440104 16777 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:13.442318 16771 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:05:13.444414 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.446026 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:13.446607 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.447343 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:13.447572 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:13.447664 13874 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:05:13.446709 13867 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:13.448930 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:13.450122 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.451136 13870 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:13.451692 16864 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:13.452164 16864 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.452584 16864 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.452808 13870 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:05:13.453253 13870 pending_rounds.cc:85] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 3
I20260709 13:05:13.453538 13870 pending_rounds.cc:107] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:05:13.453840 13870 raft_consensus.cc:2889] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:13.440215 16768 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:13.423460 16801 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:13.432401 16879 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:13.469625 16844 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:13.471024 13872 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:13.471081 16790 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:13.471838 16790 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:13.478525 13871 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:13.479421 13866 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:13.480863 16790 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.482388 16790 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:13.487411 13873 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:13.488788 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:13.491611 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:13.492755 16879 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:13.493196 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:13.493376 16879 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:13.494807 16879 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.498728 16316 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:13.500638 13949 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:05:13.504169 16871 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:05:13.505860 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 244502us
I20260709 13:05:13.509093 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.506575 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:13.518231 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:13.518537 13949 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:05:13.519073 13949 pending_rounds.cc:85] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 2
I20260709 13:05:13.519325 13949 pending_rounds.cc:107] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:05:13.519570 13949 raft_consensus.cc:2889] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:13.441619 16777 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.526398 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:13.529675 16755 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.530705 16755 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.531850 16777 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:13.533443 16755 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:13.539950 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.543254 16775 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:13.543265 16808 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:13.553160 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:13.557637 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:13.541338 16871 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:05:13.550205 16808 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:13.558256 16753 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:13.571146 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44879.9us
I20260709 13:05:13.568961 16808 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:13.577972 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 84953.8us
I20260709 13:05:13.580351 13943 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:13.580874 13947 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:13.581579 13838 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:13.594558 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.602628 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:13.581041 16682 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:13.610102 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.611303 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:13.612432 13948 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
W20260709 13:05:13.612646 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.604831 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:13.623730 13943 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:13.626539 16884 consensus_queue.cc:1059] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:13.627130 16716 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.630546 16771 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:13.582424 16768 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.646725 13949 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:13.647027 16809 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:13.647341 13868 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:13.647686 16809 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.648958 13870 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:13.641377 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:13.654610 13867 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:13.656602 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:13.645195 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:05:13.669037 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 93163.8us
I20260709 13:05:13.702018 13869 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:13.688126 13838 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:13.714913 16679 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:13.715123 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.706429 16753 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:13.709452 16806 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:13.735496 13942 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:05:13.712030 16768 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:13.739511 16753 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:13.735334 16679 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.743040 16682 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:13.748564 16682 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.750396 16738 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.115s
I20260709 13:05:13.753439 16682 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.757886 16682 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:13.758396 16751 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:13.759855 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.762454 16601 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:13.762617 16884 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:13.763231 16601 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.763296 16884 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.681581 16716 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.744550 16679 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:13.715221 16316 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:05:13.777539 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 224004us
I20260709 13:05:13.794457 16837 consensus_queue.cc:1059] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:05:13.795940 16837 raft_consensus.cc:3055] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:13.796351 16837 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:13.798516 16418 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:05:13.799618 16418 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:13.799962 16418 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:13.800586 13942 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:05:13.788259 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 118356us
I20260709 13:05:13.797307 16837 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.810868 14524 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:13.817656 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 199784us
I20260709 13:05:13.777377 16864 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.820194 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:13.811167 16601 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:13.825094 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.801060 16418 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.828128 16871 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:13.828356 16866 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:05:13.827359 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:13.832163 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 144174us
W20260709 13:05:13.843549 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.842900 16601 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:13.844982 16601 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.831785 16755 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:13.834614 16884 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:13.849455 13838 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:13.850658 16837 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:13.861232 16601 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:13.861801 16775 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:13.865226 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:13.866446 16418 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:13.866426 16775 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:13.880733 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.874197 16751 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:13.879750 16775 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:13.892334 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.877730 16808 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.879561 16716 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:13.895205 16808 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:13.895543 16918 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:13.896286 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:13.880057 16771 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:13.901804 16921 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:13.882897 16801 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.863487 16884 consensus_queue.cc:1059] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:05:13.915555 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:13.917800 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:13.918532 14555 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.920838 16771 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:13.891899 13838 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:13.927141 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:13.927673 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 58360us
I20260709 13:05:13.927358 13873 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:13.917815 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:13.904803 16917 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.908818 16808 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:13.936573 13867 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:05:13.915665 16864 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:13.939488 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:13.940876 14552 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:13.940917 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:13.943461 16864 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:13.943725 16801 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:13.944694 16920 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:13.945462 16920 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.948026 16920 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:13.949067 16864 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:13.950261 16844 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:13.949816 16917 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.950711 16844 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:13.951872 16844 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.952975 16917 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:13.923555 16601 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:05:13.954150 16844 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:13.954712 16601 raft_consensus.cc:3055] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:13.955083 16601 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:13.955659 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:13.957672 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:13.958375 16716 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:13.958323 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:13.958963 16716 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:13.959439 16716 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:13.963555 16920 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:13.963959 16775 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.964627 16920 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:13.954087 16808 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:13.966360 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:13.967950 16808 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:13.968595 16552 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.539s	user 0.013s	sys 0.002s
I20260709 13:05:13.970395 16714 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:13.983743 16714 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:13.984253 16714 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:13.970353 16801 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.986639 16801 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.956066 16601 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:13.981868 16777 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:13.982583 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:13.982944 16832 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:13.915755 16809 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:13.991822 13873 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:13.990582 16832 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:14.011695 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:14.012974 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:14.015672 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:13.941789 16418 consensus_queue.cc:1059] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:05:14.011631 16809 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:14.020028 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:14.020972 14555 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:14.025272 14553 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:14.026750 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:14.026935 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:14.028189 16916 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:14.028519 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:14.028805 16916 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:14.029444 16790 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:14.029956 16916 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.032945 16915 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:14.033406 16915 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:14.033740 16915 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:14.021157 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 137142us
I20260709 13:05:13.994346 16921 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:14.023002 13866 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:14.011297 16601 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:14.046087 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 86085.9us
I20260709 13:05:14.059547 13867 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:14.065411 13873 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
I20260709 13:05:14.073303 13947 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:14.088376 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:14.088481 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.088866 13867 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:05:14.089341 13873 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:14.092823 16884 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:14.092963 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:14.113176 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:14.120028 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:14.077154 16751 consensus_queue.cc:1059] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
W20260709 13:05:14.126191 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.127148 14559 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:05:14.084760 16418 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:05:14.125938 13866 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:14.072695 16864 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:05:14.106338 16806 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.129452 16775 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:14.168490 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:14.135461 16777 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:14.169525 16777 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:14.138563 16925 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.170488 16925 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.170729 16777 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.089963 16316 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:05:14.173147 16790 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.173732 16790 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.175940 16790 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:14.176012 16775 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:14.177241 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.176782 16775 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.179085 13838 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:14.179378 16775 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:14.180004 16917 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:14.180461 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:14.180742 16844 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.181213 16809 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:14.181509 16917 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:14.181772 16809 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.182001 16917 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:14.106930 16801 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:14.169198 13947 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:14.139546 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:14.149653 16924 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:05:14.149653 16871 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:14.150887 16806 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.099896 16552 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 13:05:14.157332 16913 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.354s	user 0.005s	sys 0.000s
I20260709 13:05:14.158164 16751 consensus_queue.cc:1243] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:14.159373 16716 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.164965 16755 raft_consensus.cc:3055] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:05:14.166465 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 71272.9us
I20260709 13:05:14.128259 16832 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.203125 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:14.167757 16921 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.186710 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.187023 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:14.192862 13871 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:14.182255 16808 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:05:14.195982 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 213557us
I20260709 13:05:14.192131 16753 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:05:14.197438 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 104625us
I20260709 13:05:14.182348 16809 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:14.203967 13866 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:14.205729 13867 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:14.211799 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:14.212772 13869 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:14.181398 16844 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:14.220723 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.204602 16928 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.202435 16871 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:14.201087 16755 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:14.230281 16932 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:14.232756 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:14.233547 14559 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:14.234452 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:14.240952 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:14.200591 16714 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.193630 16915 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.196620 16917 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.241577 13867 raft_consensus.cc:2376] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7. Re-sending same reply.
W20260709 13:05:14.253098 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.244225 14554 raft_consensus.cc:2368] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12.
I20260709 13:05:14.238194 16932 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:14.254500 16871 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.255645 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:14.259898 16927 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.260411 16915 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:14.217574 16790 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:05:14.286326 13947 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:05:14.266551 13873 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:05:14.267097 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 286619us
I20260709 13:05:14.274880 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:14.244181 16316 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:05:14.296824 16916 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:14.296937 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:14.297570 16916 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:14.298779 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:14.298681 16916 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.302165 16917 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:14.305511 16832 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:14.310491 16932 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.312583 16932 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:14.314028 16790 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:14.314797 16790 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:14.319038 16755 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.321098 16755 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:14.253149 16928 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.325035 16928 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:14.327845 16716 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:14.267908 16753 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:05:14.240823 16801 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:05:14.292205 16927 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.350543 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:14.351544 16806 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:14.355846 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.373690 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.363852 16925 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:14.377131 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:14.377563 16808 consensus_queue.cc:1059] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:14.373416 16775 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.378688 16775 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.381389 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:14.381978 14554 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:14.382351 13945 raft_consensus.cc:2410] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:14.382746 16775 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:14.383183 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.385425 13873 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:14.391026 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.316325 16790 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.398186 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:14.400308 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.413909 13942 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:14.424438 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.398244 16864 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.425984 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:14.432003 13871 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:14.433107 13866 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:14.430368 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.435916 13942 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:14.436591 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:14.374584 16679 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:05:14.437733 16864 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.439910 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:14.440623 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:14.445401 13874 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:14.444252 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:14.442421 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 145636us
I20260709 13:05:14.447564 13949 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:14.429075 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:14.451674 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:14.452402 13866 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:14.452901 13870 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:14.405972 16809 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.394642 16936 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:05:14.427227 16806 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.462441 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:14.444749 16844 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:14.465008 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:14.462430 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:14.466908 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.467489 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.473551 13869 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
W20260709 13:05:14.473603 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.474483 13946 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:14.478971 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.479683 13945 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:14.484835 16714 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:14.451690 16920 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:05:14.451195 16871 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:05:14.487560 16809 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:14.467240 16771 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:05:14.490597 16771 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:14.490976 16771 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:14.491282 16864 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:14.458218 16777 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:05:14.491917 16771 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.454267 16716 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:05:14.471104 16866 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:14.493561 16771 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:14.468801 16915 consensus_queue.cc:1059] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:05:14.493855 16682 raft_consensus.cc:3055] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:05:14.510422 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 146384us
I20260709 13:05:14.512063 16944 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:14.512535 16944 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:14.512382 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:14.513412 16884 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:14.513620 16944 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.514470 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.514815 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:14.514997 16944 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:14.515244 13945 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:14.477407 16806 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.516412 16915 consensus_queue.cc:1243] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:14.516994 16915 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:14.517328 16915 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:14.518261 16806 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:14.518669 16915 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.520802 16915 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:14.502571 16866 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.522349 16927 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:14.522600 16866 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:14.522696 16777 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:14.523078 16927 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:14.523335 16777 raft_consensus.cc:697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:14.523571 16927 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.524045 16927 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:14.524825 16777 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.509171 16682 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:14.527348 16884 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:14.528059 16682 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.529439 16682 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:14.529872 13987 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:14.530697 16932 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:14.531267 16932 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:14.532415 16932 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:14.550113 13866 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:05:14.553532 13949 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:14.557083 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:14.558894 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.559583 13949 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:14.558802 13989 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:14.561941 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 198909us
I20260709 13:05:14.571017 16753 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:14.571528 16753 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:14.572580 16753 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.574092 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.574749 13941 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:14.577107 13945 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:14.577219 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 67335.5us
I20260709 13:05:14.578261 16936 consensus_queue.cc:1059] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:14.579176 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:14.579589 16884 raft_consensus.cc:3055] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:14.579806 13946 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:14.580001 16884 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:14.580929 16884 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.583709 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:14.584548 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.578439 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:14.590915 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:14.591463 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:14.569660 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 105645us
I20260709 13:05:14.592715 13988 leader_election.cc:400] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12.
I20260709 13:05:14.604146 13988 leader_election.cc:403] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:14.573877 16753 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:14.561479 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:14.610333 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:14.613878 13866 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:14.603940 16917 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:14.613075 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:14.616046 13875 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:14.621040 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:14.621583 13838 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:14.621654 13866 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:14.627030 13874 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:14.628189 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:14.628746 13874 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:14.628697 13947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fdef734205364543b02ed4158216bf36"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:05:14.629467 13947 raft_consensus.cc:606] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:14.607641 16921 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:14.635128 16866 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:14.628324 16943 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:05:14.641894 14021 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:05:14.636808 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 70677.3us
I20260709 13:05:14.643247 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:14.646806 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 63284.3us
I20260709 13:05:14.649778 13875 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:14.615438 16929 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:14.654008 13838 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:14.652190 14522 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:14.658690 13866 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:14.684664 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:14.685331 13870 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:14.685573 14553 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:05:14.686776 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 93201.6us
I20260709 13:05:14.686909 16552 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:14.687647 16552 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:14.689287 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:14.657881 16809 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:14.637629 16917 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.690819 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:14.693567 16682 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:14.695753 16552 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.699051 14018 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:14.709821 16682 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:14.710317 16682 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:14.693542 16806 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:14.710852 16806 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.711215 16806 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:14.700703 16951 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:05:14.719182 16682 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:14.720273 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.721596 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:14.734496 14020 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:14.707656 13838 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:14.723955 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:14.726079 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:14.702868 14021 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:14.730756 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:14.721596 16682 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:14.732508 14015 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:14.738025 13870 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:14.738660 13868 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
W20260709 13:05:14.754570 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 63912.4us
I20260709 13:05:14.755642 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:14.757604 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 50031.5us
I20260709 13:05:14.763243 14554 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:14.768707 14017 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:14.767902 13838 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:14.683192 16864 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.735688 16944 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:05:14.792613 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 104505us
I20260709 13:05:14.792976 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:14.774129 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:14.754931 16808 raft_consensus.cc:3055] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:14.799705 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:14.799149 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:14.791792 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:14.801368 14018 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:14.769524 16918 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:14.766650 16936 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:14.817551 16936 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.818121 16936 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:14.817416 13869 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:14.805768 14020 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:14.832952 13874 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 3.
I20260709 13:05:14.803388 16775 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.802273 16679 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:14.802563 16808 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:14.849893 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 83764.1us
I20260709 13:05:14.852027 14020 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:05:14.845053 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 43880.2us
I20260709 13:05:14.808919 16944 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:14.856897 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:14.863222 13949 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:14.859045 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 136558us
I20260709 13:05:14.861893 14552 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:14.800213 16768 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:05:14.834977 16777 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.838994 16884 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:14.834738 16918 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.834895 16790 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:14.868510 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.869726 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:14.876273 13947 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:14.841521 16866 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:14.844025 16679 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.893700 13948 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:14.904620 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:14.815899 16864 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:14.891340 13915 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.501s (SENT) [suppressed 32 similar messages]
I20260709 13:05:14.879047 16918 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:14.878814 16790 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:14.904052 16679 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:14.948247 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 61562.1us
I20260709 13:05:14.948354 13949 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
W20260709 13:05:14.950112 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.950336 13838 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:14.849480 16775 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:14.954504 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:14.978436 16965 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:14.941547 16790 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:14.970009 16832 consensus_meta.cc:287] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.622s	user 0.007s	sys 0.007s
I20260709 13:05:14.973945 16949 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:14.969007 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:14.939998 16932 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:14.970193 16948 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 13:05:14.886358 16777 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:14.975225 16956 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:05:14.968029 13989 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:14.969481 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:14.970289 16809 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:14.970950 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.965684 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:15.016577 14015 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:14.961480 13948 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:15.034962 13942 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:15.023455 13947 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:15.024377 14017 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:15.028316 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:15.033481 14024 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:14.942695 16801 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
W20260709 13:05:15.047344 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 79405.3us
I20260709 13:05:15.038991 14020 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:14.953488 16945 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.271s
I20260709 13:05:15.069712 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:15.071841 14020 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:15.030499 16965 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.049757 16832 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:15.081526 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 102897us
I20260709 13:05:15.040517 16917 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:15.080391 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:15.052129 16932 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.033772 16944 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.052906 16966 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:15.039165 16809 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.118546 13872 raft_consensus.cc:2410] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:05:14.959125 16808 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:15.119127 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:15.128054 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 236745us
I20260709 13:05:15.165131 13989 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:15.084056 16832 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.189685 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:15.200917 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.140414 16753 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:15.213054 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:15.169657 16966 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:15.213852 14017 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:15.190333 16832 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:15.217813 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 99596.6us
I20260709 13:05:15.113440 16949 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.120s
I20260709 13:05:15.209079 16808 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:15.165786 16932 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.209534 16917 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:15.227023 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:15.204443 16714 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:05:15.219374 16945 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:15.277109 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 47069.6us
W20260709 13:05:15.296674 13870 consensus_meta.cc:287] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.530s	user 0.012s	sys 0.000s
I20260709 13:05:15.299929 13870 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:15.229656 16917 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:15.308846 16964 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:05:15.309036 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:15.301110 16950 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.384s	user 0.001s	sys 0.001s
W20260709 13:05:15.312780 16806 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.000s
W20260709 13:05:15.313517 16978 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:05:15.314456 13948 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:15.317693 13948 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:15.319238 16915 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.798s	user 0.012s	sys 0.000s
I20260709 13:05:15.320153 16966 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.323175 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:15.325297 16864 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:15.326011 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:15.326934 13947 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 5: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:15.321985 13948 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:15.330186 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:15.313344 16806 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:15.355387 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:15.369426 13868 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:15.369987 16957 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.378s	user 0.001s	sys 0.000s
I20260709 13:05:15.371291 13942 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:15.371934 16930 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:15.372440 16930 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:15.371439 16963 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:05:15.372826 16930 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:15.369310 16971 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:05:15.377082 16806 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:15.378304 16952 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.726s	user 0.007s	sys 0.000s
I20260709 13:05:15.294785 16933 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
W20260709 13:05:15.394173 16768 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.007s	sys 0.007s
W20260709 13:05:15.399188 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:15.400465 16972 log.cc:927] Time spent T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:05:15.405654 14024 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:15.305965 16945 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:15.387261 16960 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
I20260709 13:05:15.430621 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:15.442847 13945 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:15.443492 13943 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:05:15.443923 13943 pending_rounds.cc:85] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 2
I20260709 13:05:15.444211 13943 pending_rounds.cc:107] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:05:15.444483 13943 raft_consensus.cc:2889] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:15.446811 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.446746 13945 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:05:15.448632 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 273159us
I20260709 13:05:15.450210 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:15.431689 16949 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:15.461840 14524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.017s	sys 0.023s
W20260709 13:05:15.462381 14524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.017s	sys 0.023s
I20260709 13:05:15.393846 16917 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.474179 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:15.468086 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 513678us
I20260709 13:05:15.442618 16679 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.454504 16915 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:15.482359 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:15.486075 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 156007us
I20260709 13:05:15.484583 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:15.503208 16983 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:15.488880 16974 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:15.504909 16983 raft_consensus.cc:3055] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:15.505420 16983 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:15.506444 13875 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:05:15.506848 13875 pending_rounds.cc:85] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 6
I20260709 13:05:15.506562 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:15.507086 13875 pending_rounds.cc:107] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:05:15.507382 13875 raft_consensus.cc:2889] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:15.507633 14554 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:15.508821 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:15.509801 14553 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:15.511497 16679 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:15.512310 16801 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:15.512815 16801 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:15.514218 16984 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.514859 16984 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:15.516923 13868 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:15.486112 16915 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.513794 16801 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.480979 16864 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:05:15.518358 16915 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:15.496745 16809 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:15.534544 13875 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:05:15.548895 13875 pending_rounds.cc:85] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 6
I20260709 13:05:15.549150 13875 pending_rounds.cc:107] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:05:15.549388 13875 raft_consensus.cc:2889] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:15.524690 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.490140 16932 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:15.519222 16801 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:15.538566 16949 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:05:15.548311 14522 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:15.518014 16790 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.490558 16917 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:15.501612 16716 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:15.596259 16716 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.596760 16716 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:15.596892 16970 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.390s	user 0.001s	sys 0.000s
I20260709 13:05:15.598750 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:15.599807 16974 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:15.600225 16974 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:05:15.529697 16927 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.005s	user 0.010s	sys 0.001s
I20260709 13:05:15.538273 16984 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:15.555824 16771 consensus_meta.cc:287] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.049s	user 0.011s	sys 0.003s
I20260709 13:05:15.538988 16866 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:05:15.574282 16965 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:15.520110 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:15.555938 16987 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:15.579622 16983 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:15.582953 16915 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.600781 14556 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:05:15.585362 16832 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:15.575286 16951 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.861s	user 0.013s	sys 0.000s
W20260709 13:05:15.660496 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 175977us
W20260709 13:05:15.538901 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 48581.8us
I20260709 13:05:15.523515 16945 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:15.595368 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.533326 16930 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.526279 16929 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:15.621927 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:15.603866 16981 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.158s	user 0.000s	sys 0.002s
W20260709 13:05:15.605757 16959 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.432s	user 0.000s	sys 0.003s
I20260709 13:05:15.638810 16927 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.640120 16915 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:15.600628 16974 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:15.641324 16983 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:15.522536 16864 consensus_queue.cc:1243] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:15.624619 14556 pending_rounds.cc:85] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting all ops after (but not including) 4
I20260709 13:05:15.682366 13874 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:05:15.605522 16936 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.769s	user 0.006s	sys 0.005s
I20260709 13:05:15.601262 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:15.607890 16958 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:15.751869 16958 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:15.751407 13769 catalog_manager.cc:5697] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 5 to 6, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 6 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:15.612864 16777 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:15.753841 16777 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.603893 16790 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:15.661286 16965 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.754253 16809 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.755045 16790 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.754974 16965 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.755244 16809 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.755753 16790 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:15.756407 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.757544 16965 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:15.733500 14556 pending_rounds.cc:107] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.5
I20260709 13:05:15.758296 14556 raft_consensus.cc:2917] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Aborting config change with OpId 3.5: Aborted: Op aborted by new leader
I20260709 13:05:15.758936 14556 raft_consensus.cc:2889] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:15.759670 14556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49438 (request call id 4) took 4131 ms. Trace:
I20260709 13:05:15.759996 14556 rpcz_store.cc:276] 0709 13:05:11.628247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.631316 (+  3069us) service_pool.cc:224] Handling call
0709 13:05:11.660825 (+ 29509us) log.cc:844] Serialized 501 byte log entry
0709 13:05:12.478448 (+817623us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b540018d1a0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33b3b26 00007fb2c33ad49a 00007fb2c33ac714 00007fb2c338ceaa 00007fb2c3393d53 00007fb2c3393cd9 00007fb2c3393c69 00007fb2c3393c31 00007fb2c3392dfd 000000000038f354
0709 13:05:13.047956 (+569508us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b1800197d40. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c338cb0b 00007fb2c3411b63 00007fb2c3439752
0709 13:05:15.759534 (+2711578us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17302,"raft.queue_time_us":334336,"raft.run_cpu_time_us":7183,"raft.run_wall_time_us":342498,"spinlock_wait_cycles":79806208,"thread_start_us":451845,"threads_started":2,"wal-append.queue_time_us":512685,"wal-append.run_cpu_time_us":2703,"wal-append.run_wall_time_us":1118600}
I20260709 13:05:15.762212 16864 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:15.752926 16958 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:15.762542 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Aborted: Op aborted by new leader
I20260709 13:05:15.762897 16864 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:15.763734 16958 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:15.764236 16864 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.765652 16864 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:15.757951 16790 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:15.754355 16777 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.673777 16951 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12.
I20260709 13:05:15.681209 16771 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:15.750200 16936 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:15.773967 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 225749us
I20260709 13:05:15.775758 14024 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:05:15.750478 16966 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.779156 13769 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 4 to 5, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 5 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:15.748972 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:05:15.792002 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.796119 14022 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:05:15.681901 16929 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:16.265820 16716 consensus_meta.cc:287] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.658s	user 0.011s	sys 0.001s
I20260709 13:05:17.690182 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:15.915886 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.079581 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 410) took 1715 ms. Trace:
W20260709 13:05:15.931169 16918 consensus_meta.cc:287] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.962s	user 0.019s	sys 0.000s
W20260709 13:05:15.917991 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.736140 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2439) took 3054 ms. Trace:
I20260709 13:05:16.376468 16832 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:16.276813 16980 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.922391 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 419) took 1197 ms. Trace:
I20260709 13:05:16.278366 17001 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.828590 16945 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:16.590080 16771 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.018s	sys 0.022s
I20260709 13:05:17.745756 16945 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:17.746299 16771 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.891s	user 0.050s	sys 0.074s
I20260709 13:05:17.746829 16771 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:17.748235 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2274) took 2391 ms. Trace:
I20260709 13:05:16.278285 16979 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.749029 16979 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.748812 13947 rpcz_store.cc:276] 0709 13:05:15.357373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.357804 (+   431us) service_pool.cc:224] Handling call
0709 13:05:15.391509 (+ 33705us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:15.391970 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:15.392016 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:15.405893 (+ 13877us) log.cc:844] Serialized 26 byte log entry
0709 13:05:15.696916 (+291023us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:15.696965 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.697047 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:17.747969 (+2050922us) raft_consensus.cc:1685] finished
0709 13:05:17.747992 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.748149 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":916,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":17125,"thread_start_us":249495,"threads_started":1,"wal-append.queue_time_us":2316640}
I20260709 13:05:17.751195 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:17.751437 16979 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:17.752012 16979 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:17.752877 13947 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:17.752549 16979 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:16.366305 16991 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 13:05:16.893141 13866 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.977s	user 0.003s	sys 0.008s
W20260709 13:05:17.754699 13866 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.839s	user 0.005s	sys 0.013s
I20260709 13:05:17.755522 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 421) took 3030 ms. Trace:
I20260709 13:05:17.755915 13866 rpcz_store.cc:276] 0709 13:05:14.725474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.494242 (+768768us) service_pool.cc:224] Handling call
0709 13:05:15.494741 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:15.495049 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:15.849944 (+354895us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.755170 (+1905226us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:17.755227 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:17.755283 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.755356 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.755465 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":157145,"prepare.queue_time_us":1155597,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":21720,"thread_start_us":2003862,"threads_started":2,"wal-append.queue_time_us":1779359}
I20260709 13:05:16.189908 16866 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:16.275864 16995 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.427s
I20260709 13:05:17.758015 16866 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.741655 16918 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:17.760823 16918 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:17.760982 16866 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:16.847685 13942 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:16.904585 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:17.763178 13773 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57648 (request call id 278) took 21217 ms (client timeout 15000 ms). Trace:
W20260709 13:05:16.639120 16983 consensus_meta.cc:287] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.894s	user 0.014s	sys 0.000s
I20260709 13:05:16.366921 16994 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.764864 16994 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:16.386943 16930 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:17.636355 13990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.032s	sys 0.085s
W20260709 13:05:17.766346 13990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.032s	sys 0.086s
I20260709 13:05:17.768062 16930 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.768548 16930 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.769249 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:17.769996 14023 raft_consensus.cc:2368] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8.
I20260709 13:05:17.770229 13945 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:05:17.770432 14023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1574) took 1987 ms (client timeout 2169 ms). Trace:
W20260709 13:05:17.770905 14023 rpcz_store.cc:269] 0709 13:05:15.782842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.927611 (+144769us) service_pool.cc:224] Handling call
0709 13:05:17.768992 (+1841381us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:17.770350 (+  1358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3698042752}
I20260709 13:05:17.771384 16930 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:17.771703 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:16.903939 13943 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:17.773082 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:15.829752 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 168134us
W20260709 13:05:17.041939 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.773730 14017 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
W20260709 13:05:16.023304 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 363707us
W20260709 13:05:17.774257 14017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1569) took 1998 ms (client timeout 1711 ms). Trace:
I20260709 13:05:17.774020 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:17.689178 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:17.686869 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.830426 16966 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.686421 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:17.693837 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 2.01186e+06us
I20260709 13:05:17.783972 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:17.775974 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.784528 13941 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
W20260709 13:05:16.731062 13871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.141s	user 0.008s	sys 0.002s
W20260709 13:05:15.839318 16884 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.958s	user 0.012s	sys 0.003s
W20260709 13:05:16.447330 16987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.022s	sys 0.017s
I20260709 13:05:15.828004 16984 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.307758 16777 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:17.821805 16777 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:17.822358 16777 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.822716 16777 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:17.824954 16929 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:16.373680 16801 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.819s	user 0.007s	sys 0.003s
W20260709 13:05:17.809780 13840 connection.cc:256] RPC call timeout handler was delayed by -2.0561922750000576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17070000000000002s for a response
W20260709 13:05:17.830436 13840 connection.cc:256] RPC call timeout handler was delayed by -2.0552922750000562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1708s for a response
I20260709 13:05:17.830556 16801 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:17.830929 13840 connection.cc:256] RPC call timeout handler was delayed by -2.0271814740000309s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27940000000000004s for a response
W20260709 13:05:17.831390 13840 connection.cc:256] RPC call timeout handler was delayed by -2.0093922750000957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
I20260709 13:05:17.831192 16801 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:17.829926 16929 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:16.367411 16996 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:17.088117 13769 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 6 to 9, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 9 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:17.834381 16996 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:17.836688 16996 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:17.735872 13950 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:16.334757 16775 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:17.743243 16974 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.998s	user 0.007s	sys 0.008s
I20260709 13:05:17.743916 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:17.838327 16974 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.840557 16974 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:17.841169 16974 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:17.736943 16716 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:17.842357 16984 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.843744 16716 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:17.734999 13867 rpcz_store.cc:276] 0709 13:05:14.364305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.364569 (+   264us) service_pool.cc:224] Handling call
0709 13:05:14.394431 (+ 29862us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:14.476661 (+ 82230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:14.476709 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:14.508189 (+ 31480us) log.cc:844] Serialized 26 byte log entry
0709 13:05:14.815346 (+307157us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:14.815387 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.815466 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:15.604760 (+789294us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:16.031560 (+426800us) raft_consensus.cc:1685] finished
0709 13:05:16.031596 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.079468 (+ 47872us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34735,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":10127,"spinlock_wait_cycles":7132160,"thread_start_us":305998,"threads_started":2,"wal-append.queue_time_us":520298,"wal-append.run_cpu_time_us":13995,"wal-append.run_wall_time_us":2062464}
I20260709 13:05:17.736552 13869 rpcz_store.cc:276] 0709 13:05:14.645287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.895369 (+250082us) service_pool.cc:224] Handling call
0709 13:05:14.895896 (+   527us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:15.202847 (+306951us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:15.202884 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:15.222860 (+ 19976us) log.cc:844] Serialized 52 byte log entry
0709 13:05:15.363799 (+140939us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:15.363870 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.363962 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:15.603439 (+239477us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b2400120878. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c01cc9c4 00007fb2c01caacb 00007fb2c0385dc9 00007fb2c0385a55 00007fb2c038509d 00007fb2c2fb710e 00007fb2c2fce88c 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221
0709 13:05:17.685382 (+2081943us) raft_consensus.cc:1685] finished
0709 13:05:17.685421 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.699341 (+ 13920us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":69585,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":60418,"spinlock_wait_cycles":9710976,"thread_start_us":248903,"threads_started":2,"wal-append.queue_time_us":193669,"wal-append.run_cpu_time_us":7128,"wal-append.run_wall_time_us":1764189}
I20260709 13:05:17.747928 16771 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.844732 16984 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:17.235067 16714 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:17.845764 16714 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.748337 16945 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:17.774242 16994 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:16.390830 16927 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:16.497709 16951 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.285722 16928 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.849887 16951 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:17.850368 16928 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.852239 16951 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:16.670495 16916 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.758273 17016 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:17.762512 16809 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:17.772148 14023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1580) took 1858 ms (client timeout 1707 ms). Trace:
I20260709 13:05:17.772326 16980 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:16.925630 17006 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:17.331611 14523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.306s	user 0.007s	sys 0.078s
I20260709 13:05:17.858934 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:17.785444 13871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.196s	user 0.008s	sys 0.009s
W20260709 13:05:17.774691 14017 rpcz_store.cc:269] 0709 13:05:15.776242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.794433 (+ 18191us) service_pool.cc:224] Handling call
0709 13:05:17.772916 (+1978483us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:17.774190 (+  1274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3763637120}
W20260709 13:05:17.860883 14017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1581) took 1928 ms (client timeout 1708 ms). Trace:
W20260709 13:05:17.861235 14017 rpcz_store.cc:269] 0709 13:05:15.932363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.860701 (+1928338us) service_pool.cc:211] Skipping call since client already timed out
0709 13:05:17.860821 (+   120us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:05:17.785557 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.786410 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:17.080870 16958 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.316s	user 0.016s	sys 0.000s
W20260709 13:05:16.397837 16988 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.476s	user 0.000s	sys 0.000s
W20260709 13:05:16.919018 16917 consensus_meta.cc:287] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.289s	user 0.008s	sys 0.006s
I20260709 13:05:17.776535 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2278) took 2302 ms. Trace:
I20260709 13:05:17.786983 14020 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:17.868127 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1568) took 2092 ms. Trace:
W20260709 13:05:17.791399 16987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.046s	user 0.043s	sys 0.043s
I20260709 13:05:17.870146 16987 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:17.871178 16987 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:17.706152 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:17.873147 14018 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:17.763598 13773 rpcz_store.cc:269] 0709 13:04:56.546379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.546666 (+   287us) service_pool.cc:224] Handling call
0709 13:05:17.763021 (+21216355us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:13.683953 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:13.798531 (+114578us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:13.798594 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:05:13.798632 (+    38us) tablet.cc:662] Decoding operations
0709 13:05:13.799935 (+  1303us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:13.800037 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:13.800067 (+    30us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:05:13.801400 (+  1333us) tablet.cc:701] Row locks acquired
0709 13:05:13.801453 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:05:13.801645 (+   192us) write_op.cc:270] Start()
0709 13:05:13.801816 (+   171us) write_op.cc:276] Timestamp: P: 1783602313801582 usec, L: 0
0709 13:05:13.801838 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:05:13.841349 (+ 39511us) log.cc:844] Serialized 4402 byte log entry
0709 13:05:14.017291 (+175942us) op_driver.cc:464] REPLICATION: finished
0709 13:05:14.252199 (+234908us) write_op.cc:301] APPLY: starting
0709 13:05:14.274929 (+ 22730us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:14.329208 (+ 54279us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:14.329408 (+   200us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:14.331430 (+  2022us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:14.331928 (+   498us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:05:14.331997 (+    69us) write_op.cc:312] APPLY: finished
0709 13:05:14.332612 (+   615us) log.cc:844] Serialized 83 byte log entry
0709 13:05:14.365245 (+ 32633us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:14.377832 (+ 12587us) write_op.cc:454] Released schema lock
0709 13:05:14.377923 (+    91us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:14.380412 (+  2489us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:14.380445 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":49335,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":204520,"apply.run_cpu_time_us":5553,"apply.run_wall_time_us":144648,"num_ops":11,"prepare.queue_time_us":180134,"prepare.run_cpu_time_us":11946,"prepare.run_wall_time_us":235462,"raft.queue_time_us":127526,"raft.run_cpu_time_us":15604,"raft.run_wall_time_us":72903,"replication_time_us":215272,"spinlock_wait_cycles":60160,"thread_start_us":327056,"threads_started":4,"wal-append.queue_time_us":34270,"wal-append.run_cpu_time_us":39219,"wal-append.run_wall_time_us":1463928}]]}
I20260709 13:05:17.808109 13945 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 6: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:17.786481 16918 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.785981 16998 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.783821 13948 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:17.738423 13873 rpcz_store.cc:276] 0709 13:05:14.724914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.371721 (+646807us) service_pool.cc:224] Handling call
0709 13:05:15.372077 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:15.390258 (+ 18181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:15.390292 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:15.402725 (+ 12433us) log.cc:844] Serialized 26 byte log entry
0709 13:05:15.482898 (+ 80173us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:15.921592 (+438694us) log.cc:844] Serialized 14 byte log entry
0709 13:05:15.921952 (+   360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.922062 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:15.922130 (+    68us) raft_consensus.cc:1685] finished
0709 13:05:15.922147 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:15.922311 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60700,"prepare.queue_time_us":1035200,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":67,"raft.queue_time_us":31763,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":104127,"thread_start_us":338736,"threads_started":2,"wal-append.queue_time_us":134307}
I20260709 13:05:17.808532 16936 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:17.801981 17001 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.803972 16866 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:17.857986 16832 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.809970 14019 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:17.811159 16966 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:17.866369 16994 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:17.786824 16983 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:17.812340 17013 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.116s	user 0.000s	sys 0.003s
I20260709 13:05:17.812968 17017 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.813608 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1571) took 2036 ms. Trace:
I20260709 13:05:17.788414 16995 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.815193 16979 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:17.818284 16884 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:05:17.263829 16982 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.000s	sys 0.008s
W20260709 13:05:17.819082 13988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.015s	sys 0.013s
W20260709 13:05:17.918602 13988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.015s	sys 0.014s
I20260709 13:05:17.916117 16995 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.836102 13769 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:17.908747 16951 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.841630 16974 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:17.854879 14023 rpcz_store.cc:269] 0709 13:05:15.913537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.771967 (+1858430us) service_pool.cc:211] Skipping call since client already timed out
0709 13:05:17.772069 (+   102us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:05:17.833773 16801 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:17.242403 16915 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:15.869282 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:17.859246 14523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.837s	user 0.012s	sys 0.078s
I20260709 13:05:17.898403 16918 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:17.926642 13769 catalog_manager.cc:5697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 3 to 4, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 4 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:17.859668 13950 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:05:17.852980 16928 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:17.862317 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 76000.4us
W20260709 13:05:17.934644 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.935563 17001 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:17.935900 16979 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.936458 16979 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.936906 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:17.937806 16918 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:17.938822 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:17.869424 14017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1578) took 2017 ms (client timeout 1874 ms). Trace:
W20260709 13:05:17.870316 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.807775 17003 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:17.868530 14020 rpcz_store.cc:276] 0709 13:05:15.775879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.795036 (+ 19157us) service_pool.cc:224] Handling call
0709 13:05:17.776060 (+1981024us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:05:17.776303 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:17.867846 (+ 91543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.868069 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3766408064}
I20260709 13:05:17.809053 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:17.955881 13838 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:17.956502 14024 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:17.866382 13946 rpcz_store.cc:276] 0709 13:05:15.434687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.434875 (+   188us) service_pool.cc:224] Handling call
0709 13:05:15.454083 (+ 19208us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:15.483519 (+ 29436us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:15.483552 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:15.600664 (+117112us) spinlock_profiling.cc:243] Waited 7.47 ms on lock 0x7b2c000e5d98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:05:15.601164 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:05:15.838759 (+237595us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:15.838803 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.838878 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:17.706112 (+1867234us) raft_consensus.cc:1685] finished
0709 13:05:17.706144 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.736477 (+ 30333us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":423,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":5139,"spinlock_wait_cycles":17188736,"thread_start_us":237119,"threads_started":1,"wal-append.queue_time_us":1120932,"wal-append.run_cpu_time_us":2229,"wal-append.run_wall_time_us":1003110}
I20260709 13:05:17.902163 16998 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:17.788530 16864 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.995s	user 0.013s	sys 0.002s
I20260709 13:05:17.903548 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1566) took 2120 ms. Trace:
I20260709 13:05:17.903513 14018 raft_consensus.cc:2410] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:17.966671 13769 catalog_manager.cc:5697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 4 to 5, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 5 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:17.904659 14015 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:05:17.898747 16984 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.865149 16967 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.894455 16927 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.868460 16714 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:17.911649 14022 rpcz_store.cc:276] 0709 13:05:15.776906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.795284 (+ 18378us) service_pool.cc:224] Handling call
0709 13:05:15.795763 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:17.812816 (+2017053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.813540 (+   724us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:17.979171 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:17.910367 16994 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:17.909296 16996 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.692919 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:17.920881 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2276) took 2546 ms. Trace:
I20260709 13:05:17.982712 13949 rpcz_store.cc:276] 0709 13:05:15.374541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.376745 (+  2204us) service_pool.cc:224] Handling call
0709 13:05:15.377100 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:15.377545 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:15.377576 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:15.378247 (+   671us) log.cc:844] Serialized 26 byte log entry
0709 13:05:15.470182 (+ 91935us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:15.470227 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.470307 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:16.396948 (+926641us) spinlock_profiling.cc:243] Waited 6.35 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:17.913994 (+1517046us) raft_consensus.cc:1685] finished
0709 13:05:17.914019 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.920809 (+  6790us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1749,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":16818,"spinlock_wait_cycles":14596480,"thread_start_us":91457,"threads_started":1,"wal-append.queue_time_us":303222}
I20260709 13:05:17.857683 17016 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:15.820487 16990 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.985560 16990 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:17.988060 16990 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:17.931850 14023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1577) took 2090 ms (client timeout 1886 ms). Trace:
W20260709 13:05:17.988636 14023 rpcz_store.cc:269] 0709 13:05:15.846954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.931659 (+2084705us) service_pool.cc:211] Skipping call since client already timed out
0709 13:05:17.931784 (+   125us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:05:17.989650 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:17.990460 14023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1584) took 1851 ms (client timeout 1964 ms). Trace:
W20260709 13:05:17.990785 14023 rpcz_store.cc:269] 0709 13:05:16.138871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.989356 (+1850485us) service_pool.cc:224] Handling call
0709 13:05:17.990382 (+  1026us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:17.991423 17014 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:05:17.991688 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:17.992555 14021 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:17.993188 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1653) took 1109 ms. Trace:
I20260709 13:05:17.993726 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1573) took 2211 ms. Trace:
I20260709 13:05:17.993691 13944 rpcz_store.cc:276] 0709 13:05:16.883805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.883946 (+   141us) service_pool.cc:224] Handling call
0709 13:05:17.993101 (+1109155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:17.994278 14021 rpcz_store.cc:276] 0709 13:05:15.782466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.782605 (+   139us) service_pool.cc:224] Handling call
0709 13:05:17.768172 (+1985567us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030eeee 0000000000375939 00007fb2c83821e0 00007fb2c844b0c4 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:05:17.768329 (+   157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:17.992958 (+224629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.993507 (+   549us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b540023b254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:17.993670 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4280722432}
I20260709 13:05:17.995532 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:17.946432 16979 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:17.948166 16974 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:17.938686 16915 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:17.940332 14017 rpcz_store.cc:269] 0709 13:05:15.847389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.864602 (+2017213us) service_pool.cc:211] Skipping call since client already timed out
0709 13:05:17.864716 (+   114us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:05:17.997967 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:17.998608 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:17.998822 14017 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
W20260709 13:05:17.999569 14017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1576) took 2188 ms (client timeout 2582 ms). Trace:
I20260709 13:05:17.981828 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:18.000108 14017 rpcz_store.cc:269] 0709 13:05:15.810925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.997620 (+2186695us) service_pool.cc:224] Handling call
0709 13:05:17.999494 (+  1874us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:17.964010 16966 consensus_queue.cc:1059] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:05:18.001363 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2444) took 3241 ms. Trace:
I20260709 13:05:17.955365 16918 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:17.950281 16958 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.941838 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:17.952508 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:17.953655 16801 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:17.954145 16809 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.939321 14522 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:17.954715 16945 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:17.864681 17006 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:17.906522 16832 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:17.923537 16982 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.000s	user 0.000s	sys 0.021s
I20260709 13:05:18.018733 16966 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:18.018450 16801 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:18.019212 16966 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:18.019577 14023 raft_consensus.cc:2410] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:05:18.020093 16966 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:17.962594 13950 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:18.018314 16714 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:17.903292 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1648) took 1078 ms. Trace:
I20260709 13:05:17.968972 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:17.968614 13769 catalog_manager.cc:5697] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 5, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 5 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:17.970508 14015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1570) took 2191 ms (client timeout 1712 ms). Trace:
W20260709 13:05:17.971069 14018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1572) took 2189 ms (client timeout 1774 ms). Trace:
W20260709 13:05:17.971467 14024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1575) took 2154 ms (client timeout 2794 ms). Trace:
I20260709 13:05:17.966878 14019 rpcz_store.cc:276] 0709 13:05:15.775162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.795116 (+ 19954us) service_pool.cc:224] Handling call
0709 13:05:17.789370 (+1994254us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:05:17.809661 (+ 20291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:17.895340 (+ 85679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.895542 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3768605184}
I20260709 13:05:17.924122 16929 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.913177 16983 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.967188 16917 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:17.853050 16916 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:17.979269 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:17.995476 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:17.950757 17001 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:17.969532 17010 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:05:18.007483 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 65718us
I20260709 13:05:18.051007 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:18.016790 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:17.931638 16777 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:17.916635 16884 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.018476 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:18.001943 13871 rpcz_store.cc:276] 0709 13:05:14.759696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.498651 (+738955us) service_pool.cc:224] Handling call
0709 13:05:15.499160 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:15.548917 (+ 49757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:15.548971 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:15.589269 (+ 40298us) log.cc:844] Serialized 26 byte log entry
0709 13:05:17.860206 (+2270937us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:17.860248 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.860331 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.001070 (+140739us) raft_consensus.cc:1685] finished
0709 13:05:18.001099 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.001288 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":56134,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":13123,"thread_start_us":2270517,"threads_started":1,"wal-append.queue_time_us":2274300}
W20260709 13:05:18.023012 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 83768.8us
W20260709 13:05:18.024863 14021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1588) took 1671 ms (client timeout 1945 ms). Trace:
I20260709 13:05:17.941895 16951 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.023468 13948 rpcz_store.cc:276] 0709 13:05:16.816716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.816847 (+   131us) service_pool.cc:224] Handling call
0709 13:05:17.895358 (+1078511us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:18.025164 14018 rpcz_store.cc:269] 0709 13:05:15.778749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.796760 (+ 18011us) service_pool.cc:224] Handling call
0709 13:05:17.967884 (+2171124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2816}
W20260709 13:05:18.024578 14015 rpcz_store.cc:269] 0709 13:05:15.776588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.794939 (+ 18351us) service_pool.cc:224] Handling call
0709 13:05:17.743765 (+1948826us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030eeee 0000000000375939 00007fb2c83821e0 00007fb2c844b0c4 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:17.968183 (+224418us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46103808}
W20260709 13:05:18.025435 14024 rpcz_store.cc:269] 0709 13:05:15.810495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.810644 (+   149us) service_pool.cc:224] Handling call
0709 13:05:17.788863 (+1978219us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:17.964312 (+175449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3769339776}
I20260709 13:05:18.029784 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:18.032989 16866 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:18.065619 16866 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.066048 16866 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:18.066550 17009 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:18.066905 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:18.067677 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1590) took 1189 ms. Trace:
I20260709 13:05:18.068007 14015 rpcz_store.cc:276] 0709 13:05:16.878614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.066670 (+1188056us) service_pool.cc:224] Handling call
0709 13:05:18.067617 (+   947us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.068924 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:18.070264 14023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1582) took 2010 ms (client timeout 2047 ms). Trace:
I20260709 13:05:18.070349 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:18.070756 14023 rpcz_store.cc:269] 0709 13:05:16.060232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.991434 (+1931202us) service_pool.cc:224] Handling call
0709 13:05:18.062672 (+ 71238us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b5400020654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:18.070201 (+  7529us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62143232}
I20260709 13:05:18.071672 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:18.071767 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:18.030994 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:18.020738 16801 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:18.072901 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:18.073081 16801 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:18.073650 16801 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:17.981057 16967 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:17.914836 17017 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:18.048960 14022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1587) took 1786 ms (client timeout 1805 ms). Trace:
W20260709 13:05:18.074864 14022 rpcz_store.cc:269] 0709 13:05:16.262254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.978970 (+1716716us) service_pool.cc:224] Handling call
0709 13:05:18.048871 (+ 69901us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.020644 16832 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:18.055877 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:17.966792 17003 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:18.034500 16917 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:18.054042 16966 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:18.054409 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:18.058377 14021 rpcz_store.cc:269] 0709 13:05:16.346760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.995285 (+1648525us) service_pool.cc:224] Handling call
0709 13:05:18.017825 (+ 22540us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.061134 14017 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:18.079406 14019 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:18.079921 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1589) took 1523 ms. Trace:
I20260709 13:05:18.080466 14019 rpcz_store.cc:276] 0709 13:05:16.556324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.030493 (+1474169us) service_pool.cc:224] Handling call
0709 13:05:18.079859 (+ 49366us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.039628 16714 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:17.980990 16927 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.063272 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:18.085397 13950 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:18.063375 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:18.085992 14024 raft_consensus.cc:2376] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 3. Re-sending same reply.
I20260709 13:05:18.063827 14020 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:18.026216 16918 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.064659 16951 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:18.072412 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1583) took 1986 ms. Trace:
I20260709 13:05:18.021164 16958 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:18.079874 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1591) took 1182 ms. Trace:
I20260709 13:05:18.091336 14015 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:05:18.091728 14017 rpcz_store.cc:276] 0709 13:05:16.897412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.011367 (+1113955us) service_pool.cc:224] Handling call
0709 13:05:18.079795 (+ 68428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.060436 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:18.092453 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:18.093007 14018 raft_consensus.cc:2410] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:18.093133 14015 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:05:17.981197 16984 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.081887 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:18.026651 16809 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.083518 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:18.098685 14022 raft_consensus.cc:2410] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:18.047621 17001 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:18.099831 17001 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:18.084457 13949 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:18.018347 16945 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.087846 13944 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:18.054924 16936 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.056284 13769 catalog_manager.cc:5697] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 3, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:18.088850 14023 rpcz_store.cc:276] 0709 13:05:16.086782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.071532 (+1984750us) service_pool.cc:224] Handling call
0709 13:05:18.072333 (+   801us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.053639 16915 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:18.060412 16884 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:18.093498 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1586) took 1843 ms. Trace:
I20260709 13:05:18.094452 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:18.094621 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 4dd265d61fc141a48ca9d3d5c8f52921 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:18.021422 16996 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:18.096065 14021 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:18.082280 16980 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:18.102620 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:18.038882 16929 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.113523 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:18.104043 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:18.114352 14023 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:18.105022 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:18.115204 14017 raft_consensus.cc:2376] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12. Re-sending same reply.
I20260709 13:05:18.107335 14018 rpcz_store.cc:276] 0709 13:05:16.250269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.060198 (+1809929us) service_pool.cc:224] Handling call
0709 13:05:18.093437 (+ 33239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.107784 14024 raft_consensus.cc:2410] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:05:18.088229 16917 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:18.107261 13950 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:18.058308 17010 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:18.102578 16951 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.113903 14019 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:18.125213 14015 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:18.148592 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:18.149744 14018 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:18.151441 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:18.161190 16980 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.161940 16980 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.165985 16980 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:18.167762 16832 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.168449 16832 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:18.170980 16990 consensus_queue.cc:1059] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:18.171180 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:18.171918 16990 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:18.172979 16990 raft_consensus.cc:3055] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:18.173362 16990 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:18.174203 16990 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.175423 16990 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:18.175801 16832 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:18.177304 16832 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:18.179044 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 392447us
I20260709 13:05:18.181654 14015 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:05:18.183328 14020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1585) took 1941 ms (client timeout 1759 ms). Trace:
W20260709 13:05:18.183955 14020 rpcz_store.cc:269] 0709 13:05:16.211854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.967387 (+1755533us) service_pool.cc:224] Handling call
0709 13:05:18.152808 (+185421us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b54001220d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:18.152951 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144688384}
I20260709 13:05:18.201066 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:18.202308 17010 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:18.202700 14024 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:18.206535 17020 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:05:18.201174 13769 catalog_manager.cc:5697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 3, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 3 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.208451 17016 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:18.208760 13769 catalog_manager.cc:5697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:18.209308 16866 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:18.210848 16958 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:18.211227 16984 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:18.212571 14024 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:18.213235 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:18.215526 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:18.216356 14023 raft_consensus.cc:2410] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:05:18.178244 16930 consensus_queue.cc:1059] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:05:18.214282 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:18.115298 16936 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.098157 16994 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.115005 16801 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:18.179678 14524 connection.cc:256] RPC call timeout handler was delayed by -2.0639616610000076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
W20260709 13:05:18.316354 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.259810 14023 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:05:18.219771 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 167301us
W20260709 13:05:18.190603 13840 connection.cc:256] RPC call timeout handler was delayed by -1.9679922750000287s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
I20260709 13:05:18.256861 13769 catalog_manager.cc:5697] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 3 to 4, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 4 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:05:18.358664 13840 connection.cc:256] RPC call timeout handler was delayed by -1.8248922750000247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
W20260709 13:05:18.359182 13840 connection.cc:256] RPC call timeout handler was delayed by -1.7837980829999651s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
I20260709 13:05:18.222607 14024 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:05:18.352847 14016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.005s	sys 0.005s
W20260709 13:05:18.361697 14016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.005s	sys 0.006s
W20260709 13:05:18.369344 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.296250 17001 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.242676 16995 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:18.253733 17026 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.180781 16951 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.270900 17016 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.105s
I20260709 13:05:18.222558 16979 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.231206 16980 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.141s
I20260709 13:05:18.375231 16866 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:18.184991 16832 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:18.231241 16974 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:18.295795 16996 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:18.377174 16809 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:18.447052 16951 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:18.448009 16930 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:18.448366 16809 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.448568 16930 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:18.449067 16809 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:18.377353 16982 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:18.377645 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.450444 16982 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:18.378011 16958 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.378803 17010 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.260914 16716 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.139s
I20260709 13:05:18.356559 16864 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.370153 13769 catalog_manager.cc:5697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:18.353361 16771 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.000s	sys 0.010s
I20260709 13:05:18.379289 16945 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:18.256832 17023 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:18.251477 17021 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.401979 16929 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:18.401857 16979 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.485168 13769 catalog_manager.cc:5697] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 4 to 5, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 5 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:18.365500 16884 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.183463 16714 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:18.464248 16771 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.000s	sys 0.013s
I20260709 13:05:18.291023 17003 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:18.294683 16918 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:18.266671 16984 consensus_queue.cc:1059] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.137s
W20260709 13:05:18.433215 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 182148us
I20260709 13:05:18.291463 16983 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:18.464174 16927 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:18.431545 17006 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:18.443195 17001 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:18.444872 16777 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:18.446255 16974 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:18.378895 14524 connection.cc:256] RPC call timeout handler was delayed by -1.5599616609999885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21200000000000002s for a response
I20260709 13:05:18.452816 16982 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:18.465163 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 282496us
W20260709 13:05:18.483963 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.484378 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 302046us
I20260709 13:05:18.434352 16832 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:18.408188 17026 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.440140 14022 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:18.464694 16958 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:18.449925 16930 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.479557 16864 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:18.663956 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 258783us
I20260709 13:05:18.466921 16987 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.174s
I20260709 13:05:18.751875 13950 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:05:18.752779 14524 connection.cc:256] RPC call timeout handler was delayed by -1.5455616609999652s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
W20260709 13:05:18.753235 14524 connection.cc:256] RPC call timeout handler was delayed by -1.4275391209999952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3227s for a response
I20260709 13:05:18.527596 13769 catalog_manager.cc:5697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 4, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 4 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.748833 16979 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:18.749311 16929 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:18.616633 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1567) took 2841 ms. Trace:
I20260709 13:05:18.478631 17010 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:18.751122 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:18.751833 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.756628 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:18.757725 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:18.492727 17021 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:18.759533 14016 rpcz_store.cc:276] 0709 13:05:15.775525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.777101 (+  1576us) service_pool.cc:224] Handling call
0709 13:05:17.752886 (+1975785us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:05:17.753138 (+   252us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:17.753441 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:17.753471 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:17.754039 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.362255 (+608216us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:18.526143 (+163888us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.583168 (+ 57025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.583315 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.583379 (+    64us) raft_consensus.cc:1685] finished
0709 13:05:18.616317 (+ 32938us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.616552 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56545,"raft.queue_time_us":56704,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":65250,"spinlock_wait_cycles":3765654272,"thread_start_us":765063,"threads_started":2,"wal-append.queue_time_us":485334}
I20260709 13:05:19.240617 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:19.007388 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.754526 13769 catalog_manager.cc:5697] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:19.242645 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:18.832329 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 47200.7us
I20260709 13:05:19.250392 13942 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
W20260709 13:05:18.936626 13840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.001s	sys 0.015s
W20260709 13:05:19.251502 13840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.005s	sys 0.026s
W20260709 13:05:19.251823 13840 connection.cc:256] RPC call timeout handler was delayed by -1.7501922750000176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
W20260709 13:05:19.252609 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.865976 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 118365us
I20260709 13:05:19.241691 13869 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:18.815410 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:18.802274 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.248370 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 490820us
I20260709 13:05:19.258543 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.261868 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:19.264837 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.265686 13950 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
W20260709 13:05:19.249043 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.242071 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.277122 16979 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:19.277554 13840 connection.cc:256] RPC call timeout handler was delayed by -1.3339630530000477s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
W20260709 13:05:19.277980 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2993247140000221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
I20260709 13:05:19.277802 16979 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:19.282775 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.285136 17006 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.285701 17006 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:19.285084 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.287981 17006 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:19.289070 17006 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.289777 17006 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:19.290989 16979 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:19.292284 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:19.293476 13914 connection.cc:256] RPC call timeout handler was delayed by -2.0401505199999974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:05:19.293874 13942 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:19.293057 17006 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:19.289125 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.295352 13943 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:19.293131 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.296730 13950 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
W20260709 13:05:18.793869 16917 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.004s
W20260709 13:05:19.278347 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2938922750000756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2554s for a response
I20260709 13:05:18.757016 16918 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:19.238762 17046 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
I20260709 13:05:18.763939 16982 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:19.314293 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.313793 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:18.828966 17037 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.388s	user 0.001s	sys 0.000s
W20260709 13:05:18.908016 17038 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.540s	user 0.002s	sys 0.003s
I20260709 13:05:19.311054 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:19.275455 16929 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:19.319216 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:19.319957 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:19.320142 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:19.352767 13866 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:05:19.320700 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 4dd265d61fc141a48ca9d3d5c8f52921 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:19.321499 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:19.358031 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:19.358415 14553 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:19.358943 13873 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:19.321776 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:19.041301 16884 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:18.827898 16864 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:19.365882 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:18.819633 16998 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:19.337172 14554 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:05:19.392814 16966 consensus_meta.cc:287] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.303s	user 0.016s	sys 0.000s
I20260709 13:05:19.318080 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.393517 16966 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.824523 16974 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:19.276041 16958 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:19.352210 14561 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:18.854794 16928 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.472s
I20260709 13:05:19.362027 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.396001 13866 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:19.397116 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.397737 13866 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:19.364514 13872 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:05:19.399192 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:19.301144 16994 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:05:19.272697 16930 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:18.788793 16832 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:19.366681 13867 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:19.367311 13874 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:05:19.399830 13868 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:19.346292 17053 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:19.368875 13758 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 48246.7us
I20260709 13:05:19.400352 17052 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:05:19.411147 17021 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:19.412990 17010 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:19.414161 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:19.444890 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 145499us
I20260709 13:05:19.446631 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:19.415743 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1579) took 3568 ms. Trace:
I20260709 13:05:19.335987 16979 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 13:05:19.414944 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.461534 13944 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:19.443600 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.316706 16917 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:19.447331 13947 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:19.408231 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.446211 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.401546 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.465926 13872 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:19.410287 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 319) took 1645 ms. Trace:
I20260709 13:05:19.414719 17017 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:19.347977 16982 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:19.437641 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.480466 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:19.480785 13866 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:19.369484 16929 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:19.369700 16951 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.494654 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.331302 16918 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:19.366154 17056 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:19.415417 17046 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:05:19.471830 13869 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:19.486181 13838 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:19.493314 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.466969 14020 rpcz_store.cc:276] 0709 13:05:17.765136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.211626 (+446490us) service_pool.cc:224] Handling call
0709 13:05:18.212050 (+   424us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:18.212484 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.212515 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:18.213151 (+   636us) log.cc:844] Serialized 52 byte log entry
0709 13:05:18.367209 (+154058us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:18.588534 (+221325us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.589339 (+   805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.589466 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.907148 (+317682us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:19.403654 (+496506us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b48002115d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:05:19.410033 (+  6379us) raft_consensus.cc:1685] finished
0709 13:05:19.410082 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.410220 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32529,"prepare.queue_time_us":742858,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":59,"raft.queue_time_us":32765,"raft.run_cpu_time_us":17309,"raft.run_wall_time_us":808208,"spinlock_wait_cycles":84090880,"thread_start_us":311409,"threads_started":2,"wal-append.queue_time_us":154056}
I20260709 13:05:19.494225 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.501437 13949 raft_consensus.cc:2410] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:19.501236 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.494472 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.523746 13874 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:05:19.502810 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.401822 16884 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.504189 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:19.494086 16990 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.006s	sys 0.016s
I20260709 13:05:19.506455 16967 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:19.506821 16982 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:19.411168 17026 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:19.401171 16966 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:19.507200 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:19.507575 17017 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.507716 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:19.508438 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.563890 14023 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:19.509137 16929 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:19.510345 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.401995 17003 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:19.514837 13871 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
W20260709 13:05:19.519099 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.519265 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:19.515604 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:19.447631 14019 rpcz_store.cc:276] 0709 13:05:15.847752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.144627 (+2296875us) service_pool.cc:224] Handling call
0709 13:05:18.198999 (+ 54372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.199541 (+   542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.199581 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.221667 (+ 22086us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.405274 (+183607us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:18.702506 (+297232us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.702861 (+   355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.702958 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.415460 (+712502us) raft_consensus.cc:1685] finished
0709 13:05:19.415490 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.415668 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25397,"prepare.queue_time_us":736963,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"raft.queue_time_us":6849,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":21354,"thread_start_us":358290,"threads_started":2,"wal-append.queue_time_us":150203}
I20260709 13:05:19.571388 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:19.599802 14555 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:19.520871 13915 connection.cc:256] RPC call timeout handler was delayed by -3.8265909369999918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
I20260709 13:05:19.530925 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:19.538569 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.543695 14024 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:05:19.546355 14016 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:19.516727 16832 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:19.558843 14015 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:19.541666 16990 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.007s	sys 0.016s
I20260709 13:05:19.648788 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:19.661695 13915 connection.cc:256] RPC call timeout handler was delayed by -3.7437909369999716s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18930000000000002s for a response
I20260709 13:05:19.662125 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.648197 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.674116 14018 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:19.648902 13947 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:19.662027 13942 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
W20260709 13:05:19.650599 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.676043 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.602326 17003 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:19.672359 13950 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:05:19.601542 17038 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:05:19.647918 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:19.534847 17035 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:19.593207 17017 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:19.779446 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:19.780491 14018 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:19.780704 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:19.675348 14023 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:05:19.781653 14020 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:19.602360 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.786131 13949 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:19.601891 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.787770 13874 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:19.691009 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 175468us
I20260709 13:05:19.707734 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:19.708833 17031 raft_consensus.cc:3055] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:19.710515 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.710786 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:19.711352 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.712179 13868 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:19.712347 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:19.713578 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.789319 14560 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:19.744935 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.795238 14019 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
W20260709 13:05:19.746454 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.748245 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.797725 14024 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:19.750869 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.798910 13869 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:05:19.714964 16928 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:19.743196 16984 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.017s	sys 0.041s
I20260709 13:05:19.811316 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.812343 13869 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:19.759609 14555 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:19.813840 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.810670 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.562968 17065 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:05:19.817129 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.674245 13944 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:19.818554 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.819226 13949 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:19.820396 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.821352 13866 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:19.815220 14023 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:19.822599 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.526877 16951 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:19.717749 16916 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:19.783782 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:19.779917 17035 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.833498 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:19.834414 13874 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:19.791698 17031 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:19.792173 14015 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:19.542979 16884 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:19.802016 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:19.845465 13868 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:19.849817 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.850543 13871 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:05:19.855078 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.855760 13874 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:19.857761 14018 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:19.830132 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.858567 13868 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
W20260709 13:05:19.829466 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 156790us
W20260709 13:05:19.810674 16984 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.018s	sys 0.041s
I20260709 13:05:19.839337 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.867453 13950 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:19.841634 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.868389 13949 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:05:19.801623 16928 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.784852 14551 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:19.529491 17056 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:19.755492 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 226987us
I20260709 13:05:19.878962 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:19.786479 16936 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:19.859787 13915 connection.cc:256] RPC call timeout handler was delayed by -3.2682616529999677s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22480000000000003s for a response
I20260709 13:05:19.823738 13866 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:19.865285 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.882961 13871 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:19.870875 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.883955 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.884307 14015 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:19.884606 13868 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:19.871831 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.837006 17035 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:19.837977 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.888659 13944 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:05:19.889914 13944 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:19.875370 14017 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:19.858733 14021 raft_consensus.cc:2368] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8.
I20260709 13:05:19.881302 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:19.887985 13942 raft_consensus.cc:2410] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:19.891544 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.891595 13941 raft_consensus.cc:2410] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:19.892263 14015 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:19.902357 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:19.906416 13915 connection.cc:256] RPC call timeout handler was delayed by -2.6838130840000076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
W20260709 13:05:19.906984 13915 connection.cc:256] RPC call timeout handler was delayed by -2.6658130839999785s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16510000000000002s for a response
W20260709 13:05:19.907384 13915 connection.cc:256] RPC call timeout handler was delayed by -2.6289130840000325s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
W20260709 13:05:19.907801 13915 connection.cc:256] RPC call timeout handler was delayed by -2.5470130840000138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
W20260709 13:05:19.908191 13915 connection.cc:256] RPC call timeout handler was delayed by -2.4057130840000127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
W20260709 13:05:19.908605 13915 connection.cc:256] RPC call timeout handler was delayed by -2.267113084000016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
W20260709 13:05:19.909052 13915 connection.cc:256] RPC call timeout handler was delayed by -1.8315130839999938s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2578s for a response
I20260709 13:05:19.912618 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:19.869870 16864 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:19.880249 16928 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:19.451736 17053 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:19.882413 16917 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:19.922708 13840 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:19.890004 16918 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:19.896850 16979 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:19.930281 13942 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:19.944977 13839 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.721s (SENT) [suppressed 19 similar messages]
I20260709 13:05:19.925367 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:19.973315 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.895586 17009 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:19.974522 14015 raft_consensus.cc:2410] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:19.917192 16966 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:19.984170 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 61524.8us
I20260709 13:05:19.985725 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:19.989063 16809 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:19.852197 17065 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:19.990203 17039 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.532s	user 0.005s	sys 0.012s
I20260709 13:05:19.991195 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:20.005616 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.006119 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1671) took 1250 ms. Trace:
I20260709 13:05:20.006727 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.006625 13946 rpcz_store.cc:276] 0709 13:05:18.761430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.761611 (+   181us) service_pool.cc:224] Handling call
0709 13:05:19.991025 (+1229414us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b5800202e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030eeee 0000000000375939 00007fb2c83821e0 00007fb2c844b0c4 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:20.006022 (+ 14997us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6820736}
I20260709 13:05:20.007637 14015 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:05:19.996522 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:19.918987 16994 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:19.921532 17064 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:20.015607 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:19.915632 16916 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.017880 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:20.010319 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.022383 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.023341 14021 raft_consensus.cc:2376] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12. Re-sending same reply.
I20260709 13:05:20.026202 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:20.026233 13987 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:20.027366 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:19.998838 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 77098us
I20260709 13:05:20.006549 14022 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:20.016213 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.031782 14015 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:05:19.998569 16918 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:20.016240 14024 raft_consensus.cc:2376] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6. Re-sending same reply.
I20260709 13:05:20.033195 14019 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
W20260709 13:05:20.033406 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 438804us
W20260709 13:05:20.035499 13870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.010s	sys 0.003s
W20260709 13:05:20.040047 13870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.010s	sys 0.003s
W20260709 13:05:20.039578 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.040534 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 44140.9us
W20260709 13:05:20.033972 16714 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.004s	sys 0.008s
I20260709 13:05:20.041785 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.043190 16809 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:20.043654 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.044718 16809 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:20.045264 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.048311 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:19.979720 16864 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:20.042210 16714 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.004s	sys 0.008s
W20260709 13:05:20.055018 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.997710 17029 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:05:20.076439 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.077474 14023 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
W20260709 13:05:20.053141 13867 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.000s	sys 0.013s
W20260709 13:05:20.078398 13867 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.001s	sys 0.013s
I20260709 13:05:20.058732 13914 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:20.080209 13914 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:05:20.071393 14524 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.014s	sys 0.015s
W20260709 13:05:20.081367 14524 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.014s	sys 0.015s
W20260709 13:05:20.081818 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 521386us
I20260709 13:05:20.087845 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:20.086645 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.039650 17003 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:20.092880 13948 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:19.995362 17009 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:19.919961 17031 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:20.062744 16918 raft_consensus.cc:697] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:20.030468 16994 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:20.032223 16916 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:20.144508 16866 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:20.146688 16983 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:20.074342 17017 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:20.161348 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 68395.1us
I20260709 13:05:20.133580 17052 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:20.090410 16995 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:05:20.155849 16714 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:20.162801 17003 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:20.135388 16958 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:05:20.164551 17056 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:20.180002 16980 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.128s	user 0.029s	sys 0.025s
I20260709 13:05:20.197573 14020 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:20.182747 16884 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:20.185698 17017 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:20.160915 17092 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
W20260709 13:05:20.203841 16980 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.167s	user 0.029s	sys 0.025s
I20260709 13:05:20.189750 16714 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.191155 17003 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:20.226213 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 132685us
I20260709 13:05:20.202956 16951 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:20.236618 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 97142.7us
I20260709 13:05:20.226069 16714 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:20.231653 17003 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:20.210178 17029 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:05:20.246349 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:20.203068 17091 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:05:20.256392 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:20.257771 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:20.261072 17075 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
W20260709 13:05:20.261931 17081 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:05:20.264096 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:20.251349 17053 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:20.270632 17009 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:20.270973 16916 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:20.274214 14555 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:05:20.313760 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 227956us
I20260709 13:05:20.315112 13840 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:20.319355 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:20.325587 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:20.222411 16918 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:20.334620 14559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 652) took 1015 ms. Trace:
I20260709 13:05:20.337037 14559 rpcz_store.cc:276] 0709 13:05:19.319468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.320410 (+   942us) service_pool.cc:224] Handling call
0709 13:05:19.320757 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:20.038936 (+718179us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b1800215f20. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c342bbfb 00007fb2c343664b
0709 13:05:20.039098 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:20.039134 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:20.049294 (+ 10160us) log.cc:844] Serialized 26 byte log entry
0709 13:05:20.252314 (+203020us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:20.252412 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.252501 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.334286 (+ 81785us) raft_consensus.cc:1685] finished
0709 13:05:20.334309 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.334548 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10485,"raft.queue_time_us":46669,"raft.run_cpu_time_us":4951,"raft.run_wall_time_us":52041,"spinlock_wait_cycles":32404352,"thread_start_us":276328,"threads_started":2,"wal-append.queue_time_us":198173}
I20260709 13:05:20.338941 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:20.347934 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 321771us
W20260709 13:05:20.327386 17082 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.117s	user 0.002s	sys 0.000s
I20260709 13:05:20.271395 17009 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
W20260709 13:05:20.317811 17080 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
W20260709 13:05:20.327594 17096 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 13:05:20.327741 17094 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.074s	user 0.000s	sys 0.002s
I20260709 13:05:20.354332 17116 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:05:20.350865 17090 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.137s	user 0.002s	sys 0.000s
W20260709 13:05:20.460856 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 145856us
I20260709 13:05:20.357380 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:20.363296 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.363914 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:20.335067 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 115711us
W20260709 13:05:20.462883 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 113534us
I20260709 13:05:20.463191 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:20.463824 14524 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.926s (SENT)
W20260709 13:05:20.397621 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.442983 13839 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:20.447299 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:20.328061 16864 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:20.355422 17098 log.cc:927] Time spent T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:05:20.467748 14555 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:20.462206 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:20.464359 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:20.475795 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:20.476823 13987 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:20.360204 17065 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:20.479672 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.481387 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.468971 16994 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:20.466192 16951 raft_consensus.cc:2764] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:05:20.470551 16809 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:20.500495 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2466) took 1003 ms. Trace:
I20260709 13:05:20.469372 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:20.502274 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:20.486155 17097 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:05:20.499621 17109 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
W20260709 13:05:20.510016 17101 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:05:20.510366 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:20.489018 17056 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:20.498515 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:20.475489 17116 raft_consensus.cc:697] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:20.483647 16884 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:20.475754 16987 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:20.505561 17112 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:20.506119 16979 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.531s	user 0.011s	sys 0.001s
I20260709 13:05:20.502375 17064 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:20.500885 13872 rpcz_store.cc:276] 0709 13:05:19.496788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.497006 (+   218us) service_pool.cc:224] Handling call
0709 13:05:19.517989 (+ 20983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:19.518233 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:20.013037 (+494804us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.499990 (+486953us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:20.500051 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:20.500111 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.500286 (+   175us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.500430 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83525,"prepare.queue_time_us":490482,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":109317,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"thread_start_us":834376,"threads_started":3,"wal-append.queue_time_us":226844}
I20260709 13:05:20.507812 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:20.509171 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2455) took 1157 ms. Trace:
I20260709 13:05:20.523104 14558 raft_consensus.cc:2376] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8. Re-sending same reply.
I20260709 13:05:20.490721 17065 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:20.497524 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2463) took 1037 ms. Trace:
I20260709 13:05:20.551527 13867 rpcz_store.cc:276] 0709 13:05:19.460137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.460290 (+   153us) service_pool.cc:224] Handling call
0709 13:05:19.495026 (+ 34736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:19.495330 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:20.090813 (+595483us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.351670 (+260857us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.496858 (+145188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:20.496949 (+    91us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:20.497032 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.497319 (+   287us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.497458 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44442,"prepare.queue_time_us":397664,"prepare.run_cpu_time_us":151,"prepare.run_wall_time_us":5241,"raft.queue_time_us":280183,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":171,"thread_start_us":945951,"threads_started":4,"wal-append.queue_time_us":218307}
I20260709 13:05:20.511016 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:20.528112 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:20.518703 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:20.510722 17017 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:20.499914 16966 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:20.512215 16930 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.044s	user 0.006s	sys 0.007s
I20260709 13:05:20.548288 14560 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:20.512938 17118 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:20.553359 16979 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:20.567999 14555 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:05:20.517019 16832 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.859s	user 0.009s	sys 0.002s
I20260709 13:05:20.548223 13870 rpcz_store.cc:276] 0709 13:05:19.351469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.351825 (+   356us) service_pool.cc:224] Handling call
0709 13:05:19.352165 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.447895 (+ 95730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.447927 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:19.448418 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:05:20.040565 (+592147us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:20.040617 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.040709 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.508951 (+468242us) raft_consensus.cc:1685] finished
0709 13:05:20.508977 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.509108 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42454,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":55,"spinlock_wait_cycles":11136,"thread_start_us":621451,"threads_started":2,"wal-append.queue_time_us":650751}
I20260709 13:05:20.510710 17106 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:20.519510 17053 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.578171 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2474) took 1036 ms. Trace:
I20260709 13:05:20.567631 14556 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:20.522866 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:20.573523 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 54920.6us
I20260709 13:05:20.524626 17056 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:20.574553 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 81920.8us
I20260709 13:05:20.584844 13875 rpcz_store.cc:276] 0709 13:05:19.542135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.325616 (+783481us) service_pool.cc:224] Handling call
0709 13:05:20.458456 (+132840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:20.458904 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:20.499815 (+ 40911us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.576589 (+ 76774us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:20.576644 (+    55us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:20.576692 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.576767 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.578103 (+  1336us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34530,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":46801,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":2254,"thread_start_us":63625,"threads_started":1}
I20260709 13:05:20.530833 16884 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.565672 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:20.570614 17099 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:05:20.567605 17065 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:20.582396 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 101125us
I20260709 13:05:20.571764 17112 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:20.517102 17052 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:20.586380 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:20.584594 16984 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.577037 16982 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:20.578569 17016 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:20.586169 16928 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.618s	user 0.013s	sys 0.000s
I20260709 13:05:20.602736 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:20.605685 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.612593 13871 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:20.576584 17009 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:20.590109 16884 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:20.615176 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2305) took 1099 ms. Trace:
I20260709 13:05:20.616030 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.615171 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2307) took 1092 ms. Trace:
I20260709 13:05:20.617148 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:20.617974 17035 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.004s
I20260709 13:05:20.648252 14024 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 3: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:20.648391 14021 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:20.626250 17116 consensus_queue.cc:237] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:20.584290 17106 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:20.612905 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:20.589413 17056 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:20.615206 17114 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:20.654593 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.593243 17065 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:20.647157 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.648037 13949 rpcz_store.cc:276] 0709 13:05:19.523003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.900049 (+377046us) service_pool.cc:224] Handling call
0709 13:05:19.919662 (+ 19613us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:20.070173 (+150511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:20.070220 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:20.070932 (+   712us) log.cc:844] Serialized 26 byte log entry
0709 13:05:20.448046 (+377114us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:20.448373 (+   327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.448904 (+   531us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.614937 (+166033us) raft_consensus.cc:1685] finished
0709 13:05:20.614955 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.615106 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3167,"raft.queue_time_us":61787,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":10751,"thread_start_us":440008,"threads_started":2,"wal-append.queue_time_us":465662}
I20260709 13:05:20.651407 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:20.652472 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:20.577476 17017 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:20.655243 14016 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:05:20.625588 13840 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:20.662154 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.647500 13941 rpcz_store.cc:276] 0709 13:05:19.515998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.892638 (+376640us) service_pool.cc:224] Handling call
0709 13:05:19.918125 (+ 25487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.922905 (+  4780us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:19.995205 (+ 72300us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.208645 (+213440us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:20.211976 (+  3331us) log.cc:844] Serialized 26 byte log entry
0709 13:05:20.222893 (+ 10917us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:20.222953 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.223052 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.614956 (+391904us) raft_consensus.cc:1685] finished
0709 13:05:20.614983 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.615114 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4354,"prepare.queue_time_us":72863,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":2213,"raft.queue_time_us":164564,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":25938,"thread_start_us":158909,"threads_started":1,"wal-append.queue_time_us":308903}
I20260709 13:05:20.647704 16982 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.663173 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:20.664364 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:20.664500 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:20.664654 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 435) took 1116 ms. Trace:
I20260709 13:05:20.665302 13840 leader_election.cc:400] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8.
I20260709 13:05:20.666122 13840 leader_election.cc:403] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:20.587965 17053 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:20.670500 13915 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.673259 16983 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:20.673527 17120 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:05:20.673902 16983 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:20.675026 16983 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:20.655768 17009 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:20.669677 17107 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:05:20.665246 13874 rpcz_store.cc:276] 0709 13:05:19.548113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.502977 (+954864us) service_pool.cc:224] Handling call
0709 13:05:20.534499 (+ 31522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:20.534704 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:20.566942 (+ 32238us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.664189 (+ 97247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:20.664259 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:20.664329 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.664433 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.664582 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35506,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"thread_start_us":96692,"threads_started":1}
I20260709 13:05:20.681751 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.683517 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.654785 17114 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.694393 17114 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:20.670188 16918 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:20.698504 16918 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.698997 16918 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:20.651015 16928 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:20.705027 17017 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:20.660730 17121 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:20.687309 17112 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:05:20.688426 17059 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.934s	user 0.000s	sys 0.014s
I20260709 13:05:20.667157 17102 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:20.678025 17009 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:20.671957 17109 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:20.680958 17100 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:20.691394 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2473) took 1157 ms. Trace:
I20260709 13:05:20.669687 16958 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:20.667392 16982 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:20.748474 13868 rpcz_store.cc:276] 0709 13:05:19.534834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.084604 (+549770us) service_pool.cc:224] Handling call
0709 13:05:20.085064 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:20.104135 (+ 19071us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:20.104176 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:20.137176 (+ 33000us) log.cc:844] Serialized 26 byte log entry
0709 13:05:20.361094 (+223918us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:20.361147 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.361230 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.691064 (+329834us) raft_consensus.cc:1685] finished
0709 13:05:20.691098 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.691306 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6190,"raft.queue_time_us":18724,"spinlock_wait_cycles":6289664,"thread_start_us":172815,"threads_started":1,"wal-append.queue_time_us":321513}
I20260709 13:05:20.653306 17106 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:20.727905 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 64240.7us
I20260709 13:05:20.651304 16984 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:20.650767 17016 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:20.653405 17035 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:20.745055 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.763151 14024 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:20.737543 16987 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:20.725193 17121 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.669952 16979 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:20.751971 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:20.753341 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.755039 17119 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:20.755371 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.756095 17109 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:20.771342 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:20.772046 13946 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:20.757428 16917 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:20.758442 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 94007.7us
I20260709 13:05:20.749068 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.765579 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:20.741238 17102 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:20.776010 14015 raft_consensus.cc:2393] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:20.776263 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.773406 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:20.774226 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:20.774273 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:20.777084 14024 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:20.769496 14019 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:20.750228 16958 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:20.776073 13944 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:20.777287 13947 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:20.778121 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:20.776082 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:20.779855 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.780841 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:20.781229 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:20.789947 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:20.790191 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:20.790807 14020 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:20.768003 16979 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:20.768617 17121 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:20.792415 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:20.797295 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.798487 14024 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:20.792706 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.782440 16866 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:20.797283 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:20.799578 14015 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:20.799741 14019 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:20.800328 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:20.801208 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:20.781616 16958 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:20.781863 17091 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:20.802825 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:20.803367 16994 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:20.801992 16866 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.782318 17017 consensus_queue.cc:1059] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:05:20.792476 16928 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:20.804410 17108 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:20.805974 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.789539 16929 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:20.779337 17102 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:20.808936 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.827880 14024 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:20.828823 17109 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:20.827505 16929 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.832053 16929 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:20.803860 17092 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:05:20.823309 16714 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.570s	user 0.015s	sys 0.000s
I20260709 13:05:20.817378 16866 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:20.807840 16983 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:20.822016 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.798291 16979 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:20.824242 13947 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:20.825556 14015 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:20.782198 16917 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:20.867192 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:20.867833 16917 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:20.869385 17003 consensus_meta.cc:287] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.000s
I20260709 13:05:20.878780 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.879540 14023 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:20.880057 16866 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:20.880616 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.871363 17035 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:20.885259 14019 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:20.818598 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:20.807667 17091 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.899231 17091 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:20.795588 16987 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:05:20.782727 17100 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:05:20.811928 16994 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.805217 16915 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:05:20.905035 14019 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:20.848539 17102 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:20.825747 17029 raft_consensus.cc:2764] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:05:20.827823 17108 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.854972 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.856529 17053 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:20.862282 17056 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:20.885421 16984 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:20.852828 16714 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:20.834728 14020 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:20.888317 14522 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.881821 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.889379 17003 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:20.913482 17056 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:20.914011 17003 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:20.914202 16928 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:20.914740 16928 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:20.915199 14522 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:05:20.892149 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 73630.5us
I20260709 13:05:20.917870 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:20.918689 13868 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:20.861584 17121 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:20.907766 17108 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:20.931345 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:20.905544 16994 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:20.910351 16866 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:20.910609 17102 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:20.904531 17006 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:20.934494 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:20.941967 13870 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
W20260709 13:05:20.936489 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 156693us
I20260709 13:05:20.942797 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.943675 14015 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:20.943689 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:20.915558 17064 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:20.957261 13867 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:20.970047 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:20.970813 13875 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:20.982559 14019 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:20.986156 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.996301 14015 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:20.996757 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:20.990474 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:20.985436 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:20.957006 17064 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:20.967068 17006 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:20.915714 16928 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:21.007668 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.008924 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:20.967691 17102 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:05:21.003904 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 88365us
I20260709 13:05:20.984885 16866 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:21.021660 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.018949 17109 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:21.024124 17121 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:21.028095 16979 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.028662 17016 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:21.030841 16714 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:21.061419 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:21.064102 14556 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:21.034854 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.066146 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.067710 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.036689 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.069923 17114 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:21.070133 16958 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:21.070399 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.071929 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:21.072727 16958 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:21.073359 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:21.006456 16884 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.074090 16982 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:21.075861 16884 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:21.076222 16982 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:21.049528 14019 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:20.996044 17009 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
W20260709 13:05:21.040844 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55517us
I20260709 13:05:20.987463 17065 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.052546 14020 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:05:21.053879 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:21.003245 16918 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.039338 17102 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.055176 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:21.057098 16929 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:21.061398 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2291) took 1780 ms. Trace:
I20260709 13:05:21.062731 17006 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.078130 14019 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:21.078385 14015 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:21.078662 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 45735.8us
I20260709 13:05:21.031319 17120 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8.
I20260709 13:05:21.079838 13867 raft_consensus.cc:2368] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8.
I20260709 13:05:21.079070 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:21.081806 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.272236 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:21.460871 14020 pending_rounds.cc:85] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 2
I20260709 13:05:21.461306 14020 pending_rounds.cc:107] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:05:21.461710 14020 raft_consensus.cc:2889] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:21.463940 17003 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:21.464432 17003 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:21.464854 17003 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:21.072327 17114 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:21.467293 13875 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:21.074092 17106 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:21.138739 17137 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:05:21.470986 17106 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:21.140010 14023 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:21.177280 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:21.103734 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 160158us
I20260709 13:05:21.475528 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:21.462788 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 408990us
W20260709 13:05:21.476490 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 397466us
I20260709 13:05:21.479084 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:21.462175 13948 rpcz_store.cc:276] 0709 13:05:19.281392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.281537 (+   145us) service_pool.cc:224] Handling call
0709 13:05:19.281899 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:20.098101 (+816202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.061009 (+962908us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b5400147dd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:21.061280 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1864462720}
I20260709 13:05:21.483769 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.484701 13868 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:21.038614 17109 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:21.489450 13871 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:21.501922 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.493920 17142 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:21.504731 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:21.482452 17062 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:21.525283 14024 raft_consensus.cc:2368] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7.
I20260709 13:05:21.497004 17064 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.499037 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:21.483536 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:21.498451 16979 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:21.506134 17109 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:21.485849 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:21.507522 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.520519 13871 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:21.531147 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:21.534206 17062 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.482255 17102 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:21.537302 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 53849.7us
I20260709 13:05:21.497207 14023 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:05:21.495944 17108 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.551s	user 0.002s	sys 0.009s
I20260709 13:05:21.540498 14551 raft_consensus.cc:2368] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8.
I20260709 13:05:21.545377 13867 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:21.505352 17142 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:21.535666 14019 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.550756 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:21.564030 13947 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:05:21.498453 17084 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:21.538867 17106 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:21.570839 14019 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:21.555090 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.541895 14020 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:21.554096 14023 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:21.099009 16714 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.442s
I20260709 13:05:21.553354 17062 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:21.562100 14021 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:05:21.553567 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:21.556902 17009 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:21.572242 13871 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:21.585139 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.218988 17091 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.561815 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.585717 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:21.586390 13867 raft_consensus.cc:2410] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:05:21.586167 13947 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:21.585749 13871 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:21.587299 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:21.587529 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.587899 13867 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:21.587750 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:21.586407 14020 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:21.589463 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:21.589876 14020 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:21.590847 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:21.591228 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:21.591831 14020 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:21.581797 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.592998 14020 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:21.593714 13872 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:21.556800 16864 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:05:21.594591 13914 leader_election.cc:400] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8.
I20260709 13:05:21.594935 13914 leader_election.cc:403] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:05:21.595326 13871 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:21.566763 17142 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.558071 16958 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:05:21.608916 14020 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:21.611483 13868 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:21.612289 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.612809 13868 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:21.613430 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.613948 13868 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:21.601701 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.561693 17108 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.619573 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:21.617462 17071 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.768s	user 0.014s	sys 0.019s
I20260709 13:05:21.584161 17009 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:21.601814 14023 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.615525 13947 raft_consensus.cc:2368] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7.
I20260709 13:05:21.620955 13871 raft_consensus.cc:2410] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:21.620782 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.624634 14019 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:21.603950 17084 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:21.635926 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.637246 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.635841 13872 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:21.638029 13871 raft_consensus.cc:2410] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 13:05:21.638957 16927 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:05:21.634536 17009 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:21.640836 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:21.645181 14023 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:21.645903 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.646175 14019 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:05:21.646714 13868 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:21.641400 13874 raft_consensus.cc:2393] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:21.649209 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.650266 13872 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:21.650719 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.652019 13868 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 6 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:21.652693 13871 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:21.675288 17064 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:21.675793 13872 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:21.676404 13868 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:21.685578 16994 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.744s	user 0.004s	sys 0.005s
I20260709 13:05:21.686719 16918 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:21.721729 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.744592 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:21.782680 13874 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:21.653880 13871 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
W20260709 13:05:21.798453 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 267390us
I20260709 13:05:21.798877 13872 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:21.799307 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:21.804728 13866 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:21.806973 17062 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:21.809509 13868 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:21.809839 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:21.813957 13866 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:21.809237 17003 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:21.830636 17091 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:21.833552 13870 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:05:21.809058 16714 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:21.781392 16994 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.836845 17108 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:21.837354 16994 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:21.835505 17154 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:05:21.846894 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:21.852447 16918 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:21.848417 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:21.849736 17062 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:21.853916 13840 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:21.857323 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:21.835569 16714 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:21.862593 16714 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.867616 17084 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8.
I20260709 13:05:21.846365 17003 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:21.873144 17065 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:21.874879 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 439) took 1453 ms. Trace:
I20260709 13:05:21.876760 17118 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:21.875401 13873 rpcz_store.cc:276] 0709 13:05:20.421744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.521204 (+ 99460us) service_pool.cc:224] Handling call
0709 13:05:20.521520 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:20.521713 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:20.522006 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.861185 (+1339179us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b1800454840. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd323a
0709 13:05:21.874399 (+ 13214us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:21.874476 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:21.874536 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.874646 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:21.874807 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12913,"prepare.queue_time_us":1654,"prepare.run_cpu_time_us":4293,"prepare.run_wall_time_us":26570,"spinlock_wait_cycles":5465472,"thread_start_us":55721,"threads_started":1,"wal-append.queue_time_us":91895,"wal-append.run_cpu_time_us":1231,"wal-append.run_wall_time_us":1001250}
I20260709 13:05:21.877969 16918 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:21.878883 17031 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:21.881050 16930 raft_consensus.cc:3055] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:21.881477 16930 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:21.882368 16930 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:21.883306 16930 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:21.888908 16918 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:21.890686 13840 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:21.892551 14522 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:21.900979 13867 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
W20260709 13:05:21.889770 16866 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.850s	user 0.010s	sys 0.000s
I20260709 13:05:21.905764 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:21.889864 16929 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:21.908504 16966 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:21.909118 16966 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.909562 16966 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:21.911455 16983 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:21.912026 16983 raft_consensus.cc:697] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:21.912935 16983 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:21.903653 17062 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:21.920639 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:21.922678 13840 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:21.924718 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:21.928172 13840 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:21.917605 16866 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:21.929291 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:21.936045 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 190943us
I20260709 13:05:21.936955 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:21.937770 13945 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:05:21.918071 17064 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:21.943222 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:21.944373 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:21.944566 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:21.945001 14023 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:21.945271 14554 raft_consensus.cc:2393] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:21.933938 17120 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:21.934435 16918 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:21.949700 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:21.947301 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:21.949741 17120 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:21.951419 17120 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:21.955293 14559 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:21.946655 17156 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:21.958847 17156 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:21.958873 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:21.959472 17156 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:21.960531 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:21.960638 17114 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:21.961457 16982 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:21.942832 17064 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:21.961946 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:21.962340 17064 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:21.962791 16982 raft_consensus.cc:2764] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:05:21.964124 17102 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:21.966668 14553 raft_consensus.cc:2368] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6.
I20260709 13:05:21.967104 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:21.967998 14552 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:21.861399 17142 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:21.981974 14023 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
W20260709 13:05:21.985283 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 177732us
I20260709 13:05:21.989761 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:22.028169 14561 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:22.037153 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47450.1us
I20260709 13:05:22.038375 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:22.039639 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:22.040400 14023 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:22.036820 17065 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:22.044385 17091 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:22.042882 16994 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:05:22.078714 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 149504us
I20260709 13:05:22.079800 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:22.081290 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:22.083068 14018 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
W20260709 13:05:22.084460 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 120709us
I20260709 13:05:22.086031 13839 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:22.090505 14023 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
W20260709 13:05:22.091810 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 246991us
I20260709 13:05:22.040127 17102 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:22.091929 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:22.093853 13989 leader_election.cc:336] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.849s (SENT)
I20260709 13:05:22.078676 17091 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.096534 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:22.078980 16994 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:22.078662 17036 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:22.097004 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:22.051332 17109 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.101267 13915 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:22.084173 16918 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:22.072178 17108 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:05:22.108476 16714 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:22.099255 17155 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:05:22.108933 16714 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:22.109361 16714 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:22.109411 17155 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:22.109933 17155 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:22.110343 17155 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:22.111326 13989 connection.cc:256] RPC call timeout handler was delayed by -2.06651152500001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
I20260709 13:05:22.111989 17031 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:22.112244 13989 connection.cc:256] RPC call timeout handler was delayed by -1.906858884999906s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26520000000000005s for a response
I20260709 13:05:22.112495 17031 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:22.113536 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:22.114962 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:22.115391 17031 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:22.102284 16994 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:22.119782 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.120278 13989 connection.cc:256] RPC call timeout handler was delayed by -1.6296544849999464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2452s for a response
W20260709 13:05:22.120639 13989 connection.cc:256] RPC call timeout handler was delayed by -1.5868467569999893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2816s for a response
W20260709 13:05:22.120960 13989 connection.cc:256] RPC call timeout handler was delayed by -1.5828107219999765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
W20260709 13:05:22.121248 13989 connection.cc:256] RPC call timeout handler was delayed by -1.5162588849998428s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3086s for a response
W20260709 13:05:22.121555 13989 connection.cc:256] RPC call timeout handler was delayed by -1.2393964049999795s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
W20260709 13:05:22.121843 13989 connection.cc:256] RPC call timeout handler was delayed by -1.1463107219999529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2251s for a response
I20260709 13:05:22.094427 17084 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:05:22.122924 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:22.084715 17006 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:05:22.115608 14023 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:22.123306 17084 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:22.099634 17091 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.099804 16864 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:22.130016 16864 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.130463 16864 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:22.130956 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:22.131048 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.131754 13948 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:22.134593 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:22.134883 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 511) took 2661 ms. Trace:
I20260709 13:05:22.135299 13950 rpcz_store.cc:276] 0709 13:05:19.473245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.880912 (+407667us) service_pool.cc:224] Handling call
0709 13:05:19.900350 (+ 19438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:20.168179 (+267829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:20.168216 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:20.168287 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:20.168321 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.134636 (+1966315us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b5400091d54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:22.134692 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.134812 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605617920}
I20260709 13:05:22.136646 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:22.136873 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:22.139766 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.140843 13988 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:22.142155 16918 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:22.143903 16990 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:22.137800 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:22.145448 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1622) took 1373 ms. Trace:
I20260709 13:05:22.146209 14023 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:22.147902 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:22.148249 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:22.145854 14024 rpcz_store.cc:276] 0709 13:05:20.771973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.842418 (+ 70445us) service_pool.cc:224] Handling call
0709 13:05:22.145222 (+1302804us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54002eac54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:22.145387 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511374720}
I20260709 13:05:22.127343 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:22.129505 17115 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:22.072862 17009 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:22.157328 16990 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:22.105386 17036 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:22.159653 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2298) took 1144 ms. Trace:
I20260709 13:05:22.084288 17065 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:22.152089 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:22.167101 13945 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:22.167683 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:22.152791 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:22.168977 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:22.152058 14524 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:22.148615 14559 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
W20260709 13:05:22.154706 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 4dd265d61fc141a48ca9d3d5c8f52921 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:22.157462 17009 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:22.160091 13871 rpcz_store.cc:276] 0709 13:05:21.015000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.792778 (+777778us) service_pool.cc:224] Handling call
0709 13:05:21.793310 (+   532us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:21.801935 (+  8625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.801968 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:21.813000 (+ 11032us) log.cc:844] Serialized 52 byte log entry
0709 13:05:22.045612 (+232612us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:22.045671 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.045772 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.159393 (+113621us) raft_consensus.cc:1685] finished
0709 13:05:22.159419 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.159591 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15280,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":4754,"thread_start_us":126187,"threads_started":1,"wal-append.queue_time_us":245731}
W20260709 13:05:22.161815 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.158748 16966 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.159260 17109 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.127815 17084 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:22.193053 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.173791 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:22.153568 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:22.176256 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.184178 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 100194us
I20260709 13:05:22.168659 14554 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 3.
W20260709 13:05:22.210958 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 57473.2us
I20260709 13:05:22.185875 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:22.211390 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:22.211827 13989 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:22.186637 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 100688us
I20260709 13:05:22.212651 13840 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:22.126864 17031 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:22.202574 17142 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:22.117918 17064 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:22.146536 17166 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:22.216295 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:22.190466 16866 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:22.231928 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 91176.9us
I20260709 13:05:22.220633 16990 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:22.233145 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:22.218643 17102 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:22.244535 13988 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:22.222730 16979 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:22.252179 17174 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:22.252632 16979 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:22.253290 16979 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:22.254344 16979 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.257207 14016 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
W20260709 13:05:22.258203 17134 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.413s	user 0.007s	sys 0.012s
I20260709 13:05:22.258658 17084 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.259981 17118 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:22.260499 17118 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:22.260555 17084 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:22.223155 17068 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:22.235534 16990 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.265555 16990 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:22.267822 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:22.270180 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:22.271075 14558 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:22.234462 16930 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:22.272482 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2299) took 1253 ms. Trace:
I20260709 13:05:22.272665 16930 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:22.273591 14018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 326) took 2288 ms (client timeout 1967 ms). Trace:
W20260709 13:05:22.274386 14018 rpcz_store.cc:269] 0709 13:05:19.985167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.985416 (+   249us) service_pool.cc:224] Handling call
0709 13:05:22.273393 (+2287977us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b540023ad54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:22.273514 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4798662912}
I20260709 13:05:22.272912 13872 rpcz_store.cc:276] 0709 13:05:21.019922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.799925 (+780003us) service_pool.cc:224] Handling call
0709 13:05:21.815273 (+ 15348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:21.817347 (+  2074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.817388 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:21.846093 (+ 28705us) log.cc:844] Serialized 26 byte log entry
0709 13:05:22.089751 (+243658us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:22.089799 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.089898 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.272256 (+182358us) raft_consensus.cc:1685] finished
0709 13:05:22.272276 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.272422 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1619,"raft.queue_time_us":6546,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":159805,"threads_started":1,"wal-append.queue_time_us":346716}
I20260709 13:05:22.277711 17064 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:22.285938 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.286537 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.286845 13871 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:22.286729 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 70521.1us
I20260709 13:05:22.288208 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:22.288436 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 187273us
I20260709 13:05:22.289557 13989 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:22.291461 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:22.292546 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.285611 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:22.225036 17166 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:22.297885 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.300051 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.299690 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:22.230540 16866 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:22.305879 13867 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:22.300452 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:22.307552 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.312824 13872 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:22.315073 13988 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:22.318492 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.287734 17031 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:22.342235 13867 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:05:22.333384 13839 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:22.331930 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 119340us
I20260709 13:05:22.318492 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.294217 17064 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:22.345636 13868 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:22.345175 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:22.349370 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:22.351596 13838 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:22.321597 16866 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:22.354319 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.355129 13942 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:05:22.320369 17171 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:22.354902 13948 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:22.358814 14555 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:22.303452 17009 consensus_queue.cc:1059] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:05:22.358028 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.303772 17109 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:05:22.329411 16987 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:05:22.351871 17174 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:22.378767 17177 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:22.379076 17174 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:22.379503 17177 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:22.380133 17174 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:22.382180 17166 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:22.394613 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.398036 13948 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:22.400553 14522 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:22.360103 17180 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:22.414197 16832 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:22.354161 17064 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.362936 16994 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.402366 16966 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.374735 17183 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:22.425171 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2335) took 1419 ms. Trace:
I20260709 13:05:22.436996 17188 raft_consensus.cc:3055] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:22.408046 17178 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:22.451718 17188 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:22.452000 17178 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.452453 17178 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:22.452702 17188 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.454212 17188 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:22.454303 17169 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:05:22.456101 16994 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.418830 16832 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.456791 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 336) took 1923 ms. Trace:
I20260709 13:05:22.420241 17171 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:22.458472 17171 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:22.457186 14017 rpcz_store.cc:276] 0709 13:05:20.533059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.533313 (+   254us) service_pool.cc:224] Handling call
0709 13:05:21.916072 (+1382759us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:05:21.916272 (+   200us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:21.918441 (+  2169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.918478 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:21.921656 (+  3178us) log.cc:844] Serialized 26 byte log entry
0709 13:05:22.086187 (+164531us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:22.126464 (+ 40277us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.127675 (+  1211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.127770 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.456483 (+328713us) raft_consensus.cc:1685] finished
0709 13:05:22.456584 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.456727 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":142018,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":12364,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":181,"spinlock_wait_cycles":9711360,"thread_start_us":189400,"threads_started":2,"wal-append.queue_time_us":321186}
I20260709 13:05:22.458858 16990 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.459415 17171 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:22.448143 17102 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:22.461342 14016 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:22.418895 17184 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:22.461424 14015 rpcz_store.cc:276] 0709 13:05:21.005906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.006982 (+  1076us) service_pool.cc:224] Handling call
0709 13:05:22.424943 (+1417961us) spinlock_profiling.cc:243] Waited 57.2 ms on lock 0x7b54001220d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:22.425102 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131519744}
I20260709 13:05:22.431335 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.376413 16995 consensus_queue.cc:1059] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:05:22.468720 13875 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:22.454487 17173 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:05:22.403656 17182 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:22.457110 13945 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:22.426815 17036 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:22.369470 17175 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:05:22.494609 17031 raft_consensus.cc:697] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:22.495774 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 185265us
W20260709 13:05:22.496531 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 178297us
W20260709 13:05:22.461025 17156 consensus_meta.cc:287] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.000s
I20260709 13:05:22.512123 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:22.513720 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.456808 16832 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:22.515565 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.519174 13948 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:22.415906 17091 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:22.503880 17120 consensus_meta.cc:287] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.004s
I20260709 13:05:22.527302 17118 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:22.530230 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.531183 17031 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:22.518285 17156 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.509192 16990 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:22.572077 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 61040us
W20260709 13:05:22.575270 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 212611us
I20260709 13:05:22.578183 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.579085 13873 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:22.525048 17166 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.517838 17182 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:22.558482 17181 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.538928 17120 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.583568 17177 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:22.623262 17091 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:22.645030 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 353517us
I20260709 13:05:22.649320 13948 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:22.570457 16714 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.649281 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:22.651254 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.656072 13942 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:05:22.656764 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 522) took 1168 ms. Trace:
I20260709 13:05:22.659490 13942 rpcz_store.cc:276] 0709 13:05:21.488139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.350975 (+862836us) service_pool.cc:224] Handling call
0709 13:05:22.656683 (+305708us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:22.655529 17182 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.661948 13873 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:22.649986 16866 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:22.662233 17091 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:22.657519 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.663357 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:22.657469 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:22.665789 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.666576 13942 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:22.597925 17166 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:22.657692 17155 consensus_meta.cc:287] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.547s	user 0.014s	sys 0.004s
I20260709 13:05:22.675215 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.676123 13949 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:22.675761 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.614800 17181 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.662855 17091 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.677793 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1685) took 1628 ms. Trace:
I20260709 13:05:22.702155 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:22.679960 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:22.680047 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2355) took 1045 ms. Trace:
I20260709 13:05:22.681247 13872 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:22.703998 14022 rpcz_store.cc:276] 0709 13:05:21.635167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.676771 (+1041604us) service_pool.cc:224] Handling call
0709 13:05:22.679513 (+  2742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.679697 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:22.679722 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:22.679780 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:22.679827 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.679882 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.679986 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:22.692011 14020 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
W20260709 13:05:22.686173 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.707465 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2356) took 1057 ms. Trace:
I20260709 13:05:22.707891 14020 rpcz_store.cc:276] 0709 13:05:21.650313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.691398 (+1041085us) service_pool.cc:224] Handling call
0709 13:05:22.691745 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:22.705902 (+ 14157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.707407 (+  1505us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:22.680408 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:22.678795 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.710466 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:22.711117 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1683) took 1662 ms. Trace:
I20260709 13:05:22.711486 13942 rpcz_store.cc:276] 0709 13:05:21.048378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.710115 (+1661737us) service_pool.cc:224] Handling call
0709 13:05:22.711061 (+   946us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:22.712514 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.701980 13949 rpcz_store.cc:276] 0709 13:05:21.048923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.665036 (+1616113us) service_pool.cc:224] Handling call
0709 13:05:22.677712 (+ 12676us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:22.713585 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.694693 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.715901 13948 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:22.691776 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.716921 13873 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:22.723201 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.625520 17178 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:22.723559 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 518) took 1834 ms. Trace:
I20260709 13:05:22.759135 13948 rpcz_store.cc:276] 0709 13:05:20.889411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.672520 (+1783109us) service_pool.cc:224] Handling call
0709 13:05:22.723496 (+ 50976us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:22.761726 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:22.762629 13949 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:22.760202 13873 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:22.681361 17181 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.693760 16987 raft_consensus.cc:3055] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:22.689708 17166 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.699270 17064 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.789074 16987 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:22.678323 17155 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:22.756031 17197 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.688242 17118 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:22.785568 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:22.687813 16866 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.686789 17091 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.682040 16714 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.790983 13950 raft_consensus.cc:2393] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 5: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:22.760188 16979 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:22.791186 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:22.793995 13948 raft_consensus.cc:2393] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:22.698936 17156 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.804397 13949 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:22.804294 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 130442us
I20260709 13:05:22.691941 16832 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:22.812290 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.814699 16979 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.818290 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 517) took 1929 ms. Trace:
I20260709 13:05:22.826179 13949 rpcz_store.cc:276] 0709 13:05:20.889108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.768281 (+1879173us) service_pool.cc:224] Handling call
0709 13:05:22.803968 (+ 35687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.817971 (+ 14003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.818221 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:22.818773 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1681) took 1771 ms. Trace:
I20260709 13:05:22.827364 13948 rpcz_store.cc:276] 0709 13:05:21.047817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.759935 (+1712118us) service_pool.cc:224] Handling call
0709 13:05:22.818688 (+ 58753us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:22.791157 17182 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:22.824829 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:22.818141 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:22.831490 13873 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:22.833099 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:22.790129 16987 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.790377 17197 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:22.820847 16864 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.686s	user 0.010s	sys 0.003s
I20260709 13:05:22.838964 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:22.842566 13949 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:22.843076 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:22.845343 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2321) took 1048 ms. Trace:
I20260709 13:05:22.846478 13949 rpcz_store.cc:276] 0709 13:05:21.797204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.835162 (+1037958us) service_pool.cc:224] Handling call
0709 13:05:22.842386 (+  7224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.844986 (+  2600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.845268 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:22.835464 17091 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:22.840710 16930 consensus_meta.cc:287] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.568s	user 0.014s	sys 0.001s
I20260709 13:05:22.860816 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:22.835996 16714 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:05:22.880443 13873 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:22.844523 17177 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:22.888763 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2353) took 1269 ms. Trace:
I20260709 13:05:22.889351 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.889195 14017 rpcz_store.cc:276] 0709 13:05:21.619568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.459372 (+839804us) service_pool.cc:224] Handling call
0709 13:05:22.459778 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:22.460099 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:22.460128 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:22.460605 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:05:22.571518 (+110913us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:22.586159 (+ 14641us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.619847 (+ 33688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.619976 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.888591 (+268615us) raft_consensus.cc:1685] finished
0709 13:05:22.888607 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.888713 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20239,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":35040,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":61,"thread_start_us":110416,"threads_started":1,"wal-append.queue_time_us":245324}
I20260709 13:05:22.836293 17182 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.849474 17091 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:22.895190 13915 leader_election.cc:400] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8.
I20260709 13:05:22.879868 16930 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:22.883101 16987 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:22.883262 17155 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:22.843935 16864 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:22.906440 16979 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.907030 13943 raft_consensus.cc:2393] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:22.907949 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.908115 16864 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.908591 13875 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:22.910528 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.900432 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1625) took 1104 ms. Trace:
I20260709 13:05:22.912005 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:22.913257 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:22.911298 14017 rpcz_store.cc:276] 0709 13:05:21.794071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.891190 (+1097119us) service_pool.cc:224] Handling call
0709 13:05:22.897782 (+  6592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.897974 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:22.898000 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:22.898044 (+    44us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:22.898099 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.898157 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.898258 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:22.914731 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.915809 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:22.901400 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.887921 17120 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.890038 17205 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:22.896909 17091 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:22.927850 13947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1680) took 1880 ms (client timeout 2138 ms). Trace:
W20260709 13:05:22.928309 13947 rpcz_store.cc:269] 0709 13:05:21.047536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.587238 (+539702us) service_pool.cc:224] Handling call
0709 13:05:22.927614 (+1340376us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b5400090954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:22.927775 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59271552}
I20260709 13:05:22.929764 13947 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:22.916501 13873 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:22.914005 13868 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:22.950651 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2328) took 1144 ms. Trace:
W20260709 13:05:22.920940 17084 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.660s	user 0.011s	sys 0.002s
I20260709 13:05:22.951184 13947 rpcz_store.cc:276] 0709 13:05:21.806421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.929183 (+1122762us) service_pool.cc:224] Handling call
0709 13:05:22.929560 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.950252 (+ 20692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.950567 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:22.921788 17197 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.922423 17178 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:22.923902 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:22.924031 17182 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:22.925172 16832 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:22.939760 16994 raft_consensus.cc:3055] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:22.887741 17177 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:22.946516 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:22.947085 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 65137us
W20260709 13:05:22.919315 17203 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:05:22.979552 13870 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:22.977681 13875 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:22.984803 13874 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:05:22.983922 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.986416 13870 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:22.987488 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:22.988598 13870 raft_consensus.cc:2410] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:05:22.990701 13947 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:22.992551 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1688) took 1118 ms. Trace:
I20260709 13:05:22.992892 13947 rpcz_store.cc:276] 0709 13:05:21.901538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.952270 (+1050732us) service_pool.cc:224] Handling call
0709 13:05:22.990419 (+ 38149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.991304 (+   885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.992494 (+  1190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:22.994395 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2322) took 1189 ms. Trace:
I20260709 13:05:22.983182 17177 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:23.003876 13870 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:05:22.982929 16994 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.005164 13873 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:23.006734 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2331) took 1099 ms. Trace:
I20260709 13:05:23.007093 13947 rpcz_store.cc:276] 0709 13:05:21.906979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.993624 (+1086645us) service_pool.cc:224] Handling call
0709 13:05:23.006305 (+ 12681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.006481 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:23.006496 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:23.006527 (+    31us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:23.006553 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.006587 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.006659 (+    72us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:23.010900 13868 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:05:23.012595 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 115511us
I20260709 13:05:23.014298 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:05:23.002871 17180 consensus_meta.cc:287] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.567s	user 0.012s	sys 0.001s
I20260709 13:05:23.015859 13872 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:23.017132 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59630 (request call id 459) took 1069 ms. Trace:
I20260709 13:05:23.026305 14551 rpcz_store.cc:276] 0709 13:05:21.945858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.946410 (+   552us) service_pool.cc:224] Handling call
0709 13:05:22.923707 (+977297us) spinlock_profiling.cc:243] Waited 8.46 ms on lock 0x7b5800004898. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:23.015355 (+ 91648us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19449728}
I20260709 13:05:23.037667 16994 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.039315 16994 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:23.046543 17207 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:23.054747 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2324) took 1249 ms. Trace:
I20260709 13:05:23.055349 13945 rpcz_store.cc:276] 0709 13:05:21.805280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.168647 (+363367us) service_pool.cc:224] Handling call
0709 13:05:23.054422 (+885775us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b5400229a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:23.054653 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":660365568}
I20260709 13:05:23.057165 13945 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:23.057977 13947 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:23.021306 17180 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:23.060470 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1692) took 1139 ms. Trace:
I20260709 13:05:23.061029 13945 rpcz_store.cc:276] 0709 13:05:21.920736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.056472 (+1135736us) service_pool.cc:224] Handling call
0709 13:05:23.056932 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.060135 (+  3203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.060390 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:23.049192 17208 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:05:23.073027 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:23.076280 13945 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:23.077286 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 524) took 1142 ms. Trace:
I20260709 13:05:23.077826 13945 rpcz_store.cc:276] 0709 13:05:21.934928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.062182 (+1127254us) service_pool.cc:224] Handling call
0709 13:05:23.076098 (+ 13916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.076901 (+   803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.077204 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:23.079977 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:23.081602 17065 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:23.082316 17006 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:23.059101 17009 consensus_queue.cc:1059] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:05:23.087431 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:23.087185 17091 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.089051 13948 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:05:23.091145 14556 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:23.099601 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 349) took 1299 ms. Trace:
I20260709 13:05:23.100036 14024 rpcz_store.cc:276] 0709 13:05:21.800335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.098050 (+1297715us) service_pool.cc:224] Handling call
0709 13:05:23.099171 (+  1121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.099356 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:23.099372 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:23.099408 (+    36us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:23.099432 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.099467 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.099550 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:23.101217 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:23.101600 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:23.104321 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 525) took 1160 ms. Trace:
I20260709 13:05:23.104972 13945 rpcz_store.cc:276] 0709 13:05:21.944337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.078951 (+1134614us) service_pool.cc:224] Handling call
0709 13:05:23.103784 (+ 24833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.103990 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:23.104019 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:23.104064 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:23.104094 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.104149 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.104262 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:23.107352 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:23.107569 13914 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:23.108912 14024 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:23.095499 17091 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:23.109473 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2359) took 1301 ms. Trace:
I20260709 13:05:23.115509 14024 rpcz_store.cc:276] 0709 13:05:21.808324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.100779 (+1292455us) service_pool.cc:224] Handling call
0709 13:05:23.108616 (+  7837us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.109250 (+   634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.109428 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:23.106243 16987 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:23.110632 17009 raft_consensus.cc:3055] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:23.112227 17213 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:23.112919 14551 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:23.112972 13914 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:23.113893 16864 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:23.119040 13945 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:05:23.117770 17009 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.126473 16864 raft_consensus.cc:3055] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:23.126384 17009 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.127938 17009 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:23.123497 17006 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:23.129772 17188 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.127061 16864 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.133517 17084 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:23.134794 17084 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:23.135164 17084 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.136456 17084 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.137472 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 351) took 1040 ms. Trace:
I20260709 13:05:23.137903 14023 rpcz_store.cc:276] 0709 13:05:22.096642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.136474 (+1039832us) service_pool.cc:224] Handling call
0709 13:05:23.136881 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.137083 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:23.137116 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:23.137162 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:23.137198 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.137262 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.137380 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:23.130630 17188 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.145051 17177 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.149233 17177 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:23.149389 17210 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 13:05:23.139189 17084 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:23.180851 17092 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.331s	user 0.014s	sys 0.022s
I20260709 13:05:21.840626 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:23.184255 17092 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.335s	user 0.014s	sys 0.022s
I20260709 13:05:23.190214 17092 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.166307 16864 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.201769 13869 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 6 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:23.204679 13945 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:23.205598 17092 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:23.210157 13950 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2320) took 1418 ms (client timeout 1632 ms). Trace:
W20260709 13:05:23.210664 13950 rpcz_store.cc:269] 0709 13:05:21.792214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.136607 (+344393us) service_pool.cc:224] Handling call
0709 13:05:23.209897 (+1073290us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b5400091d54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:23.210086 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1502249344}
I20260709 13:05:23.212756 17188 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:23.213742 16864 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:23.217329 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:23.218307 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:23.220196 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:23.220559 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:23.221426 13873 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:23.222879 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:23.223796 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:23.223692 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:22.994681 13949 rpcz_store.cc:276] 0709 13:05:21.804687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.847664 (+1042977us) service_pool.cc:224] Handling call
0709 13:05:22.883899 (+ 36235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.884115 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:22.884510 (+   395us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.994118 (+109608us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:22.994176 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:22.994217 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.994265 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.994350 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3331,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":128,"thread_start_us":109251,"threads_started":1}
I20260709 13:05:23.228269 13873 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:23.230546 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:23.231604 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:23.232885 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:23.234669 17223 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:23.236388 17177 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:23.236804 17177 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:23.237231 17177 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:23.238811 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:23.239494 14560 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:23.242997 13868 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:05:23.243712 17192 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:23.244182 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:23.244318 17192 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.245013 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1631) took 1425 ms. Trace:
I20260709 13:05:23.245398 17108 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:23.246259 16994 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:23.245465 17192 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.247102 16994 consensus_queue.cc:1243] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:23.247277 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1690) took 1329 ms. Trace:
I20260709 13:05:23.247673 16994 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:23.248199 16994 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.247774 13947 rpcz_store.cc:276] 0709 13:05:21.917615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.007752 (+1090137us) service_pool.cc:224] Handling call
0709 13:05:23.057767 (+ 50015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.059555 (+  1788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.247075 (+187520us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b5400229a50. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:23.247214 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93129984}
I20260709 13:05:23.245363 14018 rpcz_store.cc:276] 0709 13:05:21.820051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.243899 (+1423848us) service_pool.cc:224] Handling call
0709 13:05:23.244945 (+  1046us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:23.246438 13914 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:23.251617 13870 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:23.259797 14018 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:23.261404 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2309) took 2498 ms. Trace:
I20260709 13:05:23.262091 13941 rpcz_store.cc:276] 0709 13:05:20.763361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.764114 (+   753us) service_pool.cc:224] Handling call
0709 13:05:20.764438 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:20.774683 (+ 10245us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:20.789626 (+ 14943us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.878483 (+ 88857us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.878717 (+   234us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:20.878786 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:20.878857 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.057944 (+179087us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b18003dbfa0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:21.997591 (+939647us) spinlock_profiling.cc:243] Waited 9.38 ms on lock 0x7b18003dbfa0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419
0709 13:05:23.261093 (+1263502us) spinlock_profiling.cc:243] Waited 9.98 ms on lock 0x7b5400091854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:23.261165 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.261324 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22314,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":4157,"raft.queue_time_us":9758,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":147,"spinlock_wait_cycles":68179968,"thread_start_us":80001,"threads_started":1,"wal-append.queue_time_us":160953,"wal-append.run_cpu_time_us":2046,"wal-append.run_wall_time_us":1114513}
I20260709 13:05:23.226280 17052 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.226941 16918 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:23.230286 17188 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:23.274328 17223 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:05:23.274567 17091 consensus_queue.cc:1059] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:05:23.276427 17006 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:23.264353 17207 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:23.284632 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.287077 17207 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.281607 17052 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.288733 17207 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:23.290720 17091 raft_consensus.cc:3055] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:23.291113 17091 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.291983 17091 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.293242 17213 raft_consensus.cc:3055] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:23.293637 17213 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.282578 16918 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.295151 17213 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.295819 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2295) took 2286 ms. Trace:
I20260709 13:05:23.296558 17213 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:23.296602 16918 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:23.296250 13874 rpcz_store.cc:276] 0709 13:05:21.009178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.792271 (+783093us) service_pool.cc:224] Handling call
0709 13:05:21.814699 (+ 22428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:22.992076 (+1177377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.295539 (+303463us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540026c5d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:23.295744 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2690288000}
I20260709 13:05:23.306958 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 523) took 1812 ms. Trace:
I20260709 13:05:23.307405 13948 rpcz_store.cc:276] 0709 13:05:21.494152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.833611 (+1339459us) service_pool.cc:224] Handling call
0709 13:05:22.834074 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:23.089571 (+255497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.306679 (+217108us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b5400229f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:23.306884 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":585608960}
I20260709 13:05:23.308163 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:23.310441 17222 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:05:23.313721 17091 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:23.315960 17175 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.316566 17175 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.319118 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1626) took 1524 ms. Trace:
I20260709 13:05:23.319545 14016 rpcz_store.cc:276] 0709 13:05:21.794404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.300197 (+505793us) service_pool.cc:224] Handling call
0709 13:05:23.318850 (+1018653us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400298bd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:23.319046 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343001344}
W20260709 13:05:23.320070 17224 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:05:23.323668 14020 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:05:23.337947 14553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59630 (request call id 457) took 1395 ms (client timeout 1507 ms). Trace:
W20260709 13:05:23.338563 14553 rpcz_store.cc:269] 0709 13:05:21.942806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.942950 (+   144us) service_pool.cc:224] Handling call
0709 13:05:23.337646 (+1394696us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b5400530f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:23.337867 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52007808}
I20260709 13:05:23.320541 17052 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:23.288086 17188 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.367583 13868 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7. Ops: []
W20260709 13:05:23.368670 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47464.2us
I20260709 13:05:23.372176 17177 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:23.401482 13943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2326) took 1595 ms (client timeout 1540 ms). Trace:
W20260709 13:05:23.402086 13943 rpcz_store.cc:269] 0709 13:05:21.805782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.130635 (+324853us) service_pool.cc:224] Handling call
0709 13:05:23.401151 (+1270516us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b54001482d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:23.401380 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1782613248}
I20260709 13:05:23.403829 13942 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:05:23.411880 17175 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:23.414089 17177 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:23.415308 17109 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:23.415781 17109 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.415388 17207 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.416669 17109 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.417939 17109 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:23.418295 17207 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:23.418979 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:23.419615 14553 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:23.381297 17188 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.426985 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2354) took 1803 ms. Trace:
W20260709 13:05:23.430044 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 780779us
I20260709 13:05:23.431051 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:23.432140 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:23.427659 14015 rpcz_store.cc:276] 0709 13:05:21.623630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.462513 (+838883us) service_pool.cc:224] Handling call
0709 13:05:22.509598 (+ 47085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.509912 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:22.559681 (+ 49769us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.590414 (+ 30733us) log.cc:844] Serialized 14 byte log entry
0709 13:05:23.426197 (+835783us) spinlock_profiling.cc:243] Waited 5.93 ms on lock 0x7b1800436540. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c0348a40 00007fb2c03434e4 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287 00007fb2c344e208
0709 13:05:23.426430 (+   233us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:23.426508 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:23.426589 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.426701 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.426891 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36769,"prepare.queue_time_us":114148,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":95,"raft.queue_time_us":113,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":127,"spinlock_wait_cycles":13643520,"thread_start_us":22396,"threads_started":1,"wal-append.queue_time_us":54128}
I20260709 13:05:23.439853 14561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 663) took 1865 ms. Trace:
I20260709 13:05:23.440446 14561 rpcz_store.cc:276] 0709 13:05:21.574722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.574967 (+   245us) service_pool.cc:224] Handling call
0709 13:05:21.594960 (+ 19993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.028870 (+433910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.439475 (+1410605us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b54003d72d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:23.439760 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":995608832}
I20260709 13:05:23.433147 17184 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:23.438476 16927 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:05:23.451576 17220 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.302s	user 0.004s	sys 0.000s
W20260709 13:05:23.503139 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.508380 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2357) took 1761 ms. Trace:
I20260709 13:05:23.509238 14022 rpcz_store.cc:276] 0709 13:05:21.746948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.753132 (+1006184us) service_pool.cc:224] Handling call
0709 13:05:22.761289 (+  8157us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:23.431870 (+670581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:23.431903 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:23.432446 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 13:05:23.507422 (+ 74976us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:23.507552 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.507703 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:23.507775 (+    72us) raft_consensus.cc:1685] finished
0709 13:05:23.507790 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.508030 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":670128,"raft.queue_time_us":671941,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":3444,"thread_start_us":73646,"threads_started":1,"wal-append.queue_time_us":23019}
I20260709 13:05:23.511804 13838 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:23.515010 14522 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:23.518339 14522 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:23.523624 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:23.525419 16771 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:23.525750 17174 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:23.515471 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:23.537360 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.527307 17174 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.528206 16809 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:23.543244 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:23.544687 14016 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:23.545629 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 345) took 1746 ms. Trace:
I20260709 13:05:23.543080 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:23.548207 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 350) took 1457 ms. Trace:
I20260709 13:05:23.551390 17178 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.552608 17178 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.548775 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 353) took 1421 ms. Trace:
I20260709 13:05:23.557972 17065 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.560678 14015 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:23.561306 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.563006 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:23.567147 16929 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:23.568794 17171 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:23.569278 17121 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:23.573975 17174 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:23.574596 17192 consensus_queue.cc:1059] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:05:23.576077 17178 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:23.576485 17192 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:23.577081 17192 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.577248 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2343) took 2308 ms. Trace:
I20260709 13:05:23.548683 14024 rpcz_store.cc:276] 0709 13:05:22.090512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.116267 (+1025755us) service_pool.cc:224] Handling call
0709 13:05:23.122357 (+  6090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.122593 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:23.132282 (+  9689us) log.cc:844] Serialized 14 byte log entry
0709 13:05:23.179683 (+ 47401us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b1800285160. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd323a 00007fb2c343da2a
0709 13:05:23.546701 (+367018us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:23.546850 (+   149us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:23.547628 (+   778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.547759 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.547977 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":349548,"prepare.queue_time_us":9547,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"spinlock_wait_cycles":13399936,"thread_start_us":16653,"threads_started":1,"wal-append.queue_time_us":72245}
W20260709 13:05:23.582800 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 150535us
W20260709 13:05:23.584029 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.584759 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:23.586540 14022 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:23.587913 16983 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:23.588052 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:23.581144 17192 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.588814 16983 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:23.590273 17192 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.597865 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:23.598549 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:23.599037 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 83896.8us
I20260709 13:05:23.599272 13948 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:23.599601 14019 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:23.600755 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:23.601195 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:23.601882 17174 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:23.547210 14017 rpcz_store.cc:276] 0709 13:05:21.799419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.914470 (+1115051us) service_pool.cc:224] Handling call
0709 13:05:22.914880 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.915547 (+   667us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:22.915895 (+   348us) log.cc:844] Serialized 14 byte log entry
0709 13:05:23.544743 (+628848us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:23.544869 (+   126us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:23.544981 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.545084 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.545382 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":524532,"prepare.queue_time_us":8713,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":41,"raft.queue_time_us":532361,"raft.run_cpu_time_us":390,"raft.run_wall_time_us":389,"thread_start_us":103381,"threads_started":1,"wal-append.queue_time_us":133323}
I20260709 13:05:23.603405 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:23.556828 14023 rpcz_store.cc:276] 0709 13:05:22.127015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.140092 (+1013077us) service_pool.cc:224] Handling call
0709 13:05:23.144257 (+  4165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.547793 (+403536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:23.547904 (+   111us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:23.547984 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:23.548100 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.548237 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.548535 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":402124,"raft.queue_time_us":408524,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":265}
I20260709 13:05:23.560034 17065 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.608814 13943 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:23.612069 17109 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.557376 16771 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:23.611240 17065 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:23.613179 17109 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.620071 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:23.622967 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:23.623241 16983 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:23.626367 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.577883 14021 rpcz_store.cc:276] 0709 13:05:21.269119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.591671 (+322552us) service_pool.cc:224] Handling call
0709 13:05:21.592002 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:21.595400 (+  3398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.595446 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:21.607681 (+ 12235us) log.cc:844] Serialized 52 byte log entry
0709 13:05:22.803136 (+1195455us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b1800209920. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:05:22.835446 (+ 32310us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:22.848659 (+ 13213us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.991366 (+142707us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b1800209920. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:23.575869 (+584503us) log.cc:844] Serialized 14 byte log entry
0709 13:05:23.576391 (+   522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.576551 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:23.576648 (+    97us) raft_consensus.cc:1685] finished
0709 13:05:23.576667 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.576957 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":700202,"prepare.queue_time_us":15264,"prepare.run_cpu_time_us":158,"prepare.run_wall_time_us":4082,"raft.queue_time_us":703975,"raft.run_cpu_time_us":475,"raft.run_wall_time_us":16790,"spinlock_wait_cycles":9662080,"thread_start_us":32039,"threads_started":1,"wal-append.queue_time_us":1277257}
I20260709 13:05:23.627857 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:23.629191 13838 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:23.629361 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2358) took 1848 ms. Trace:
I20260709 13:05:23.633322 14020 rpcz_store.cc:276] 0709 13:05:21.781439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.753354 (+971915us) service_pool.cc:224] Handling call
0709 13:05:23.323326 (+569972us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:23.324230 (+   904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.628952 (+304722us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b5400298bd0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:23.629180 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1309818752}
I20260709 13:05:23.636667 13871 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:23.637018 13873 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:23.645973 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:23.633105 16994 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:23.648305 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.649621 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2496) took 1062 ms. Trace:
I20260709 13:05:23.651975 13867 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
W20260709 13:05:23.652100 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.654467 13943 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:23.655696 17109 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:23.664426 17174 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:23.667824 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:23.668310 17240 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:23.672145 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1679) took 2624 ms. Trace:
I20260709 13:05:23.672791 13942 rpcz_store.cc:276] 0709 13:05:21.047166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.772432 (+1725266us) service_pool.cc:224] Handling call
0709 13:05:23.403559 (+631127us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:23.404331 (+   772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.671649 (+267318us) spinlock_profiling.cc:243] Waited 607 ms on lock 0x7b54001482d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:23.671878 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1396152064}
I20260709 13:05:23.673550 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:23.675587 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:23.676885 16990 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:23.678427 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.660257 13872 rpcz_store.cc:276] 0709 13:05:22.587258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.587368 (+   110us) service_pool.cc:224] Handling call
0709 13:05:23.649032 (+1061664us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b54003acfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:23.649309 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40526208}
W20260709 13:05:23.678973 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.679493 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 95010.9us
I20260709 13:05:23.680104 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:23.680932 14561 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:23.685472 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:23.685523 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:23.687362 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.677695 16990 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:23.689946 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 60955.6us
I20260709 13:05:23.691610 17174 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:23.692126 16809 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:23.692945 17174 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:23.693500 17174 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:23.694079 17175 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:23.688515 17065 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:23.694618 17175 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:23.694993 17174 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.696135 17175 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.697104 17174 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:23.698215 17175 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:23.699239 13941 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
W20260709 13:05:23.702541 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 179260us
W20260709 13:05:23.698473 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.703557 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.703955 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:23.706547 17138 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:23.714033 14020 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:23.716423 16864 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:23.717589 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:23.718134 17207 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:23.695031 17065 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.718570 17207 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.719316 16994 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:23.720096 17207 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.722523 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:23.723565 17065 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:23.669202 17240 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.724790 17121 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:23.725629 17121 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.729710 14560 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:05:23.733839 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51672.3us
I20260709 13:05:23.722357 16771 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.729143 17138 consensus_queue.cc:1059] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:05:23.733731 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:23.732775 13839 leader_election.cc:336] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.507s (SENT)
I20260709 13:05:23.772071 14016 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:23.772071 14019 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:23.774546 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:23.775101 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:23.775497 17184 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:23.785679 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:23.733337 17139 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.043s
W20260709 13:05:23.778856 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.781741 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.790555 14022 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:23.793788 14553 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:23.783070 16990 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:23.796097 13773 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:23.804912 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:23.792115 17065 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:23.811766 14021 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:23.805809 13942 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:23.809222 17121 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:23.809516 13914 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:05:23.810190 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 57546.4us
I20260709 13:05:23.810720 17031 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:23.819751 14019 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.825726 17109 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:23.826177 13946 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:23.826442 17109 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:23.827935 17109 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.830370 17139 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:23.830564 17239 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:05:23.831264 17139 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:23.831805 17239 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:23.832197 17239 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:23.821696 16809 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:23.833418 17239 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:23.844189 13947 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:23.850346 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 119664us
I20260709 13:05:23.869590 13944 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
W20260709 13:05:23.834437 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 60172.1us
W20260709 13:05:23.915072 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.844494 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:23.884519 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 59903.3us
I20260709 13:05:23.896288 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:23.921902 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.845280 17207 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:23.938145 13773 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:23.837131 17184 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.843082 16809 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:23.882020 17213 consensus_meta.cc:287] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.581s	user 0.015s	sys 0.003s
I20260709 13:05:23.834035 17139 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:23.942451 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:23.942472 13839 leader_election.cc:336] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.538s (SENT)
I20260709 13:05:23.977581 13839 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:23.847798 17241 raft_consensus.cc:3055] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:23.881520 17240 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:23.944213 13941 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:05:23.948273 16990 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:23.950230 17036 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:23.954557 16918 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:23.956811 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:23.957787 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:23.959468 13989 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:23.970736 14561 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:05:23.980224 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.981400 13941 pending_rounds.cc:85] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 3
I20260709 13:05:23.981154 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:23.986330 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 142007us
I20260709 13:05:23.884045 17138 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 13:05:23.994741 13948 raft_consensus.cc:2410] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:23.988162 13941 pending_rounds.cc:107] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:05:23.978358 17121 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.995465 13941 raft_consensus.cc:2889] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:23.999894 14019 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:05:23.981757 17241 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:24.003173 13942 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
W20260709 13:05:24.003472 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.004185 17184 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:24.004694 17139 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.006104 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:24.006459 17180 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:24.006997 17180 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:24.006676 17139 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:24.008919 13943 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:24.009629 17180 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:24.011729 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.011876 16809 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:24.013545 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.992439 16994 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:23.882545 17102 consensus_queue.cc:1059] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.113s
I20260709 13:05:24.024277 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:24.025418 17241 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:24.026793 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:24.027442 17241 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:24.027621 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:24.028373 14023 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:24.001044 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:24.042397 14019 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:05:24.042454 13946 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:24.046818 13989 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:24.047256 14018 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:24.047930 17052 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:24.048368 17052 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:24.048964 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:24.050645 17036 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:24.051487 17036 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:24.004084 17171 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:05:24.038830 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:24.031508 17102 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:24.084920 13942 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:05:24.086556 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.013411 16918 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:24.090456 13943 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:24.036512 17138 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:24.095795 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 46940.2us
I20260709 13:05:24.055377 17184 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:24.109941 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 167538us
I20260709 13:05:24.116431 13949 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:24.117801 13838 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:24.128199 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:24.113202 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.163434 14020 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
W20260709 13:05:24.114293 17244 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.165s	user 0.004s	sys 0.000s
W20260709 13:05:24.112106 17192 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.521s	user 0.013s	sys 0.002s
W20260709 13:05:24.170755 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.132288 17246 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:05:24.173403 14021 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
W20260709 13:05:24.102859 17243 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 13:05:24.174903 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 57231.4us
I20260709 13:05:24.133618 17239 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:24.176633 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.119346 13773 catalog_manager.cc:5697] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 5 to 7, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 7 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:24.106900 16771 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:24.131702 17139 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:24.183037 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:24.184242 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 133805us
I20260709 13:05:24.049513 17052 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:24.106523 17249 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:24.198530 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.202028 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.134291 17121 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:24.181380 16983 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:24.125654 17174 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 13:05:24.249152 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 58061.6us
I20260709 13:05:24.117781 17250 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
W20260709 13:05:24.244151 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.255825 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 65480.7us
W20260709 13:05:24.256313 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.186342 16771 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:24.185134 16994 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:24.258315 17241 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.258955 17241 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:24.259353 16994 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:24.261127 17241 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:24.262775 17240 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:24.263702 17174 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:24.264235 17174 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:24.265247 17174 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:24.245954 17249 consensus_queue.cc:1059] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:05:24.266036 17240 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 13:05:24.267323 17175 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.009s
I20260709 13:05:24.267737 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:24.193149 16809 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:24.268615 14015 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:24.269089 14019 raft_consensus.cc:3055] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:24.269428 14018 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:24.294011 14021 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:24.268501 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:24.307957 14019 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:24.270426 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.271190 14016 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:24.271628 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:24.273063 14561 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:24.273415 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:24.275032 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:24.277251 13838 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:24.288585 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.274312 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:24.285302 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:24.286520 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:24.311120 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.368352 13869 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
W20260709 13:05:24.282091 13769 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57642 (request call id 272) took 20481 ms (client timeout 15000 ms). Trace:
I20260709 13:05:24.210446 14023 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:24.279656 13773 catalog_manager.cc:5697] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:24.422250 14553 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:24.426867 14017 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:24.276324 17065 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:24.431638 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 145227us
W20260709 13:05:24.436121 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.436758 14024 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:24.431959 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.277513 17052 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:24.294977 16983 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:24.307315 16809 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:24.438946 14023 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:24.440339 14018 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:05:24.184487 17138 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:24.277611 17184 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.163s
W20260709 13:05:24.434293 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 157124us
I20260709 13:05:24.287896 17241 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.146s
I20260709 13:05:24.447957 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:24.281627 17249 consensus_queue.cc:1243] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:24.433358 17207 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.434363 13773 catalog_manager.cc:5697] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 7 to 8, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 8 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:24.439747 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:24.454910 14015 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:24.439977 17036 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:24.456485 14020 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:24.461436 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.281246 17174 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:24.451558 16809 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:24.464073 14016 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:24.469791 14023 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
W20260709 13:05:24.475612 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.476861 13866 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:24.468786 14019 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:24.483043 14019 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:24.468766 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:24.491943 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.492627 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:24.494402 16918 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:24.462280 17139 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:24.495160 17139 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.495637 17139 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:24.498730 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.498761 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:24.504022 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 235671us
I20260709 13:05:24.281102 16771 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:24.464557 17031 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:24.514797 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.508653 14016 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:24.499676 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.501034 14023 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:05:24.493396 14017 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:24.511256 14019 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:24.515585 14020 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:24.539820 13773 catalog_manager.cc:5697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:24.545405 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:24.551613 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.555994 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:24.557397 14019 raft_consensus.cc:2368] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7.
W20260709 13:05:24.560160 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 91680.9us
I20260709 13:05:24.565188 17259 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.102s
I20260709 13:05:24.565467 13838 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:24.539924 17255 raft_consensus.cc:3055] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:24.570788 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.570823 17255 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:24.571892 14020 raft_consensus.cc:2410] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:24.572444 17255 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:24.570220 16918 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:24.574431 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.575205 14017 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:24.576025 17184 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:24.565510 13773 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 5 to 8, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 8 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:24.578228 17092 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:24.578799 17259 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:24.579257 17259 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:24.578950 17092 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:24.579501 13773 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:24.580317 17259 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:24.581669 17259 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:24.581037 13773 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 5 to 8, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 8 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:24.585981 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.589103 14023 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:24.594846 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.588090 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.587971 17031 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:24.539820 17121 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:24.597522 17257 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.583005 17255 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:24.608863 14021 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:24.517367 17207 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:24.610836 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:24.613657 14018 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:24.587404 17180 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.047s	sys 0.037s
W20260709 13:05:24.611018 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 359525us
W20260709 13:05:24.615962 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.624349 14017 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:24.582007 17092 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:24.623981 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:24.628787 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 66725us
W20260709 13:05:24.633318 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.638576 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 452486us
I20260709 13:05:24.618556 13773 catalog_manager.cc:5697] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 3 to 4, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 4 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:24.646098 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.604205 16918 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:24.611471 17249 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:05:24.374531 13769 rpcz_store.cc:269] 0709 13:05:03.829613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.848481 (+ 18868us) service_pool.cc:224] Handling call
0709 13:05:24.281766 (+20433285us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:19.849813 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:19.876939 (+ 27126us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:19.876989 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:05:19.877016 (+    27us) tablet.cc:662] Decoding operations
0709 13:05:19.878854 (+  1838us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:19.878933 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:19.878970 (+    37us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:05:19.924769 (+ 45799us) tablet.cc:701] Row locks acquired
0709 13:05:19.924792 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:05:19.924971 (+   179us) write_op.cc:270] Start()
0709 13:05:19.925096 (+   125us) write_op.cc:276] Timestamp: P: 1783602319924915 usec, L: 0
0709 13:05:19.925119 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:05:19.984106 (+ 58987us) log.cc:844] Serialized 6061 byte log entry
0709 13:05:20.693222 (+709116us) op_driver.cc:464] REPLICATION: finished
0709 13:05:20.793437 (+100215us) write_op.cc:301] APPLY: starting
0709 13:05:20.793543 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:20.809800 (+ 16257us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:20.810004 (+   204us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:21.101548 (+291544us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:21.131570 (+ 30022us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 13:05:21.131637 (+    67us) write_op.cc:312] APPLY: finished
0709 13:05:21.132096 (+   459us) log.cc:844] Serialized 107 byte log entry
0709 13:05:21.132329 (+   233us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:21.132528 (+   199us) write_op.cc:454] Released schema lock
0709 13:05:21.132594 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:21.133110 (+   516us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:21.133142 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2155440,"reported_tablets":39,"spinlock_wait_cycles":48256,"child_traces":[["op",{"apply.queue_time_us":81835,"apply.run_cpu_time_us":7859,"apply.run_wall_time_us":339660,"num_ops":15,"prepare.queue_time_us":377506,"prepare.run_cpu_time_us":30419,"prepare.run_wall_time_us":370178,"raft.queue_time_us":139594,"raft.run_cpu_time_us":9611,"raft.run_wall_time_us":92567,"replication_time_us":767827,"thread_start_us":461758,"threads_started":4,"wal-append.queue_time_us":247207,"wal-append.run_cpu_time_us":22144,"wal-append.run_wall_time_us":2054342}]]}
W20260709 13:05:24.652133 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.653795 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:05:24.625718 17180 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.049s	sys 0.040s
I20260709 13:05:24.664086 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.668376 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:24.672169 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.672863 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.582679 16983 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:24.683162 13773 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 2 to 4, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 4 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:24.686517 17031 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:24.696686 17256 log.cc:927] Time spent T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:05:24.691594 17188 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.693841 16918 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:24.695370 17091 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.695546 13875 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:24.695793 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:24.696857 13867 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7. Ops: []
W20260709 13:05:24.700482 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 407147us
I20260709 13:05:24.709659 13773 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:24.704557 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:24.706161 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 262922us
I20260709 13:05:24.710402 14023 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:24.713493 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.720469 13946 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:24.721216 14017 raft_consensus.cc:2410] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:24.688670 17207 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:24.723042 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:24.724661 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:24.705467 17102 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:24.676918 17249 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:24.730903 14016 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:24.739214 13942 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:24.744062 13838 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:24.746378 13869 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:24.751637 13948 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:24.746039 17102 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:24.758761 14017 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:24.758529 17102 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:24.759676 17091 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:24.760610 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.762707 13867 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:24.677567 17257 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:24.763845 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:24.764569 13867 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:24.765884 17138 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:24.768877 16983 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:24.769109 17249 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:24.771183 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.771339 13943 raft_consensus.cc:2410] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:24.776470 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:24.777510 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.713286 17121 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
W20260709 13:05:24.784380 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.787777 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.785059 14024 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:24.794368 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:24.795481 14553 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:05:24.793931 14023 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:24.799160 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:24.802300 14016 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:24.824558 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.777292 17091 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:24.849380 17175 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:24.776129 17188 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:24.914119 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.852306 17207 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.811111 17257 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:24.912467 17175 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:24.810693 14017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49624
I20260709 13:05:24.959898 14017 raft_consensus.cc:606] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Received request to transfer leadership
W20260709 13:05:24.925112 16994 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.018s	sys 0.033s
I20260709 13:05:24.917336 17009 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:24.966989 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.967163 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:24.933568 17092 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.977125 13866 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
W20260709 13:05:24.962992 16994 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.018s	sys 0.033s
I20260709 13:05:24.960964 17109 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.947194 17207 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:24.999873 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 255693us
W20260709 13:05:24.991034 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.978473 17257 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.986258 17091 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.035749 14016 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:24.978976 16809 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:24.977149 17009 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:25.032241 13868 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:25.123920 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:25.107266 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.130995 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.131719 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.037161 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:25.134516 14561 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:05:25.003446 17267 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:25.135512 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:24.995986 17139 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:25.045418 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.150195 14016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49624
I20260709 13:05:25.123628 17084 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.011571 16994 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:25.153215 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.150887 14016 raft_consensus.cc:606] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Received request to transfer leadership
I20260709 13:05:25.147069 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.108776 17121 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:25.092231 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:25.004294 17109 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:25.149569 17184 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:25.173168 17184 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:25.173637 17258 log.cc:927] Time spent T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.373s	user 0.002s	sys 0.000s
I20260709 13:05:25.139231 17277 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:25.175251 13868 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:05:25.038692 17091 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:25.037992 17257 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:25.176065 13868 pending_rounds.cc:85] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 6
I20260709 13:05:25.186244 13868 pending_rounds.cc:107] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:05:25.186676 13868 raft_consensus.cc:2889] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:25.009905 17175 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:25.133119 17092 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:25.185194 13915 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.651s (SENT) [suppressed 45 similar messages]
W20260709 13:05:25.215587 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.167111 17121 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.220432 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:25.179231 17277 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:25.182605 17283 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:25.236416 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:25.239423 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.213691 17263 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 13:05:25.187263 17266 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:25.176419 17267 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:25.185547 16994 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:25.219750 17121 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:25.250876 17052 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.229529 17188 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:25.230006 17277 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:25.186589 17084 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:25.251995 17261 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.359s	user 0.001s	sys 0.001s
I20260709 13:05:25.257601 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.274726 13872 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
W20260709 13:05:25.279731 13839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.001s	sys 0.015s
W20260709 13:05:25.280241 13839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.001s	sys 0.016s
I20260709 13:05:25.289565 13866 raft_consensus.cc:2368] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7.
I20260709 13:05:25.262216 17052 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:25.275316 17259 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.689s	user 0.007s	sys 0.008s
I20260709 13:05:25.284837 17291 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.305663 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.312891 17286 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:05:25.294533 17207 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:25.317306 13869 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.000s
I20260709 13:05:25.259017 17283 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:25.318887 17031 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.326385 17031 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:25.328074 17171 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:25.328516 17171 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:25.328850 17031 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:25.330204 17264 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
I20260709 13:05:25.331547 17254 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:25.335564 17279 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:05:25.279683 17266 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:25.338155 17288 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:05:25.347345 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:25.348347 14551 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:25.349925 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:25.351150 17281 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:05:25.352161 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.324074 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2380) took 1298 ms. Trace:
I20260709 13:05:25.353075 14022 rpcz_store.cc:276] 0709 13:05:24.025784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.030189 (+  4405us) service_pool.cc:224] Handling call
0709 13:05:24.033526 (+  3337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:24.033868 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:24.033898 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:24.055436 (+ 21538us) log.cc:844] Serialized 26 byte log entry
0709 13:05:24.605062 (+549626us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:24.605102 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.605191 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:25.323749 (+718558us) raft_consensus.cc:1685] finished
0709 13:05:25.323781 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:25.323984 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":498103,"threads_started":1,"wal-append.queue_time_us":747964}
I20260709 13:05:25.308750 16809 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:25.356869 17291 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:25.359114 17296 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:05:25.359942 17270 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:05:25.360662 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 4dd265d61fc141a48ca9d3d5c8f52921 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:05:25.361443 13758 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 116713us
I20260709 13:05:25.329440 17171 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:25.366564 17009 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:25.391021 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.391746 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.324867 13869 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:25.325663 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:25.423388 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.429366 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.433616 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.435496 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:25.436405 13869 raft_consensus.cc:2376] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5. Re-sending same reply.
I20260709 13:05:25.437161 13875 raft_consensus.cc:2376] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7. Re-sending same reply.
I20260709 13:05:25.444058 13944 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:25.440689 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:25.433406 17255 consensus_meta.cc:287] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.819s	user 0.008s	sys 0.007s
I20260709 13:05:25.447702 13941 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:05:25.424454 17139 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:25.451162 13946 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:25.455651 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.460078 13875 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:25.448478 17249 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.461730 13872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:05:25.464533 17052 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:25.466261 13759 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 196260us
W20260709 13:05:25.467993 17262 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.613s	user 0.002s	sys 0.000s
I20260709 13:05:25.444079 17184 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:25.478138 17286 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.474082 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.486770 13867 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
W20260709 13:05:25.487920 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.478750 13916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.013s	sys 0.005s
W20260709 13:05:25.481531 13873 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.014s	sys 0.005s
W20260709 13:05:25.478530 17276 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.376s	user 0.003s	sys 0.002s
W20260709 13:05:25.489806 13873 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.014s	sys 0.006s
W20260709 13:05:25.490152 13916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.014s	sys 0.005s
I20260709 13:05:25.487915 14522 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:25.507247 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:25.511123 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.512703 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:25.508317 13866 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:25.513481 13950 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:25.513764 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.518178 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:25.534530 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:25.524302 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.501323 17092 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:25.493170 17255 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:05:25.520013 17283 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:25.498627 17286 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:25.516110 17091 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:25.563294 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.568761 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:25.545315 14551 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:25.504518 17241 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:05:25.573858 13866 raft_consensus.cc:2376] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5. Re-sending same reply.
I20260709 13:05:25.544930 17084 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:25.553893 17052 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.561404 17292 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:25.504539 17249 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:25.583065 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.571656 17267 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:25.586777 17265 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:05:25.588227 17174 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.120s	user 0.019s	sys 0.002s
W20260709 13:05:25.602964 17302 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:05:25.603613 17084 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:25.605135 17282 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.101s
I20260709 13:05:25.605705 14022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49624
I20260709 13:05:25.622174 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.607836 17239 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
W20260709 13:05:25.609767 17297 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:05:25.578552 17291 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:25.629225 13944 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:25.616731 17171 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:25.606102 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:25.624330 14022 raft_consensus.cc:606] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Received request to transfer leadership
I20260709 13:05:25.620222 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.624683 13948 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:25.625898 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.589556 17254 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:25.620421 16983 consensus_meta.cc:287] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.851s	user 0.015s	sys 0.000s
I20260709 13:05:25.598533 17265 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.585877 17052 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:25.638736 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.625108 17299 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:05:25.643755 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.644546 13944 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:25.645567 13873 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:25.647847 17265 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:25.650141 17254 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:25.598536 17292 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:25.655929 17292 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:25.622779 17121 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:25.657007 13950 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:25.658358 17121 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:25.665628 14553 raft_consensus.cc:2410] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 8 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:05:25.667599 17286 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:25.668871 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.669804 13945 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:25.669626 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:25.670442 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.670876 13873 raft_consensus.cc:2368] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9.
I20260709 13:05:25.671242 13866 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:25.671361 17249 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:25.671761 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:25.672113 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:25.672364 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.672683 13866 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:25.672941 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.673076 13867 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:05:25.670491 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.675146 13869 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:25.629860 17138 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.676885 17138 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:25.678143 17302 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:05:25.680217 17138 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:25.688274 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.681785 17277 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:25.682359 13987 connection.cc:256] RPC call timeout handler was delayed by -1.6258282539998845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
I20260709 13:05:25.659278 17282 consensus_queue.cc:1243] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:25.684852 17052 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:25.691675 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:25.692550 13987 connection.cc:256] RPC call timeout handler was delayed by -1.5916282539999429s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
W20260709 13:05:25.695127 13987 connection.cc:256] RPC call timeout handler was delayed by -1.5754282539999167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
W20260709 13:05:25.695519 13987 connection.cc:256] RPC call timeout handler was delayed by -1.5088282539999227s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
I20260709 13:05:25.695290 13950 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:25.697119 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:25.695993 13987 connection.cc:256] RPC call timeout handler was delayed by -1.4692282539998587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
I20260709 13:05:25.699617 13871 raft_consensus.cc:2368] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9.
W20260709 13:05:25.700187 13987 connection.cc:256] RPC call timeout handler was delayed by -1.0750282539999034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
I20260709 13:05:25.705665 13866 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:25.705732 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.705817 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:25.708180 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.710100 14018 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:05:25.710918 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1657) took 1051 ms. Trace:
I20260709 13:05:25.711570 16771 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:25.711328 14018 rpcz_store.cc:276] 0709 13:05:24.659092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.659259 (+   167us) service_pool.cc:224] Handling call
0709 13:05:25.705598 (+1046339us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:25.710798 (+  5200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9103104}
I20260709 13:05:25.715883 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.716619 13946 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
W20260709 13:05:25.717720 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.706090 17184 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.718778 17184 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:25.719547 17277 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:25.723243 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.702868 17138 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.724174 13871 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:05:25.725605 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.726367 13949 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:05:25.728461 13839 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:25.731184 17138 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:25.733184 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.733680 17138 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:25.735023 13945 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:25.734915 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:25.736184 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.740700 17184 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:25.741746 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.742267 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.744161 14560 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:25.751814 17175 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:25.752486 14560 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:25.753686 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.754472 13869 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:25.758988 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:25.761516 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.767745 13949 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:25.737636 13871 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 4.
I20260709 13:05:25.769484 13871 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:05:25.784315 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.785403 13867 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:25.796033 17031 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:25.796995 17311 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.797605 17311 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:25.806545 17311 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:25.812493 13867 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:05:25.816193 13875 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:05:25.818214 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.829491 17084 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:25.830835 17084 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:25.832887 17084 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:25.833402 17084 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:25.833690 17109 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:25.829859 17307 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:25.836522 17266 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:25.770922 16918 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:25.839568 13868 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:25.829952 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:25.775272 17257 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:25.807062 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.733246 17254 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:25.863255 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.866990 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:25.868697 17254 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:25.869508 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 123487us
I20260709 13:05:25.844722 16918 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.870731 17092 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:25.871847 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:25.872056 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:25.873821 13872 raft_consensus.cc:606] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:05:25.909811 13872 raft_consensus.cc:614] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:25.873970 13838 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:25.911113 13838 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:25.912367 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:25.874747 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.877184 17265 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:25.878744 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.878872 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:25.866226 17307 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:25.864506 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:25.892205 17184 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.894363 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:25.914590 14553 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:05:25.915172 14561 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:25.895169 13989 leader_election.cc:400] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7.
I20260709 13:05:25.863224 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:25.917758 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.918514 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.921376 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.906931 13840 connection.cc:256] RPC call timeout handler was delayed by -1.1089998220001007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.325s for a response
W20260709 13:05:25.922523 13840 connection.cc:256] RPC call timeout handler was delayed by -1.0545010290001073s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
I20260709 13:05:25.931697 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.932780 13949 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
W20260709 13:05:25.936457 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 66441.3us
I20260709 13:05:25.955695 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:25.908847 16918 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:25.907845 16983 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.963730 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.975687 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:25.960989 17295 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.287s	user 0.000s	sys 0.005s
I20260709 13:05:25.956473 17282 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:25.973717 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.913877 17307 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:25.968044 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.963121 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:25.924388 16809 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:25.938643 13873 raft_consensus.cc:2393] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:25.982613 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.982672 14555 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:25.936303 17266 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.934418 17250 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:25.964248 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.981488 13947 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
W20260709 13:05:25.984328 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 89350.4us
I20260709 13:05:25.980593 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:25.985628 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52694.4us
I20260709 13:05:25.965912 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:25.987211 13948 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:25.960978 17249 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:25.990504 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.961598 17031 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:25.983927 13941 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:26.000604 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.970178 17291 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:25.999337 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:26.001273 13866 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
W20260709 13:05:26.002182 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.936252 17184 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:26.004487 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.960542 17052 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:05:26.006358 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:26.051049 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 87059.7us
W20260709 13:05:26.051633 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71155.4us
W20260709 13:05:26.034824 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.029311 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.053757 17138 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.003113 16809 raft_consensus.cc:697] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:26.054659 17138 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.055651 16809 consensus_queue.cc:237] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.052377 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:25.981382 17091 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:25.984185 17282 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.978394 17121 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.963675 17292 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:05:25.983626 17266 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:25.973215 17084 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:25.984998 17307 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:26.073081 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:25.999462 17250 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:26.070961 17315 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.187s	user 0.000s	sys 0.002s
I20260709 13:05:26.081974 17031 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.085181 17174 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:26.085665 17174 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:25.999575 17320 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.051162 17109 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:26.082861 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:26.067059 17091 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.986413 16983 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:26.090392 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 113641us
I20260709 13:05:26.062655 17291 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:26.097182 17174 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.076082 17282 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:26.070075 17138 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:26.101435 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:26.107805 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 101535us
I20260709 13:05:26.113112 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:26.115046 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:26.115753 13867 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:26.067924 17249 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.099062 17109 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.110410 17091 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:26.132797 13987 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:26.082499 17121 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.137678 13866 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:26.121073 17267 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:26.131451 17109 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:26.128005 17092 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:26.147521 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:26.152750 17326 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:05:26.153249 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:26.154124 13867 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:05:26.154816 17317 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:05:26.153162 17332 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:26.159369 17265 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:26.160192 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:26.161048 13870 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:26.162194 17325 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:05:26.164031 17325 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.164623 17325 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.167615 17325 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:26.168215 17325 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:26.116992 17320 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.172151 17320 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:26.173460 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.173527 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.175100 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.199007 17332 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:05:26.201074 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.143353 17267 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:26.210872 17318 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.204s	user 0.001s	sys 0.001s
I20260709 13:05:26.201457 14551 raft_consensus.cc:2410] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:26.149106 17092 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.232321 17269 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:26.232945 17269 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.233381 17269 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:26.237900 17331 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:26.237841 17138 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.238729 17331 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.136181 17213 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 13:05:26.201851 17265 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.241066 17257 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:26.301585 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.302258 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 317887us
I20260709 13:05:26.316092 13867 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:26.214716 17267 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:26.338755 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.239056 17138 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.243779 17092 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:26.380950 17328 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:26.381721 17328 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:26.385476 17316 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.328s	user 0.002s	sys 0.000s
I20260709 13:05:26.387086 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 539) took 1032 ms. Trace:
I20260709 13:05:26.387512 13942 rpcz_store.cc:276] 0709 13:05:25.355218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.426614 (+ 71396us) service_pool.cc:224] Handling call
0709 13:05:25.441563 (+ 14949us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:25.452712 (+ 11149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:25.452738 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:25.472420 (+ 19682us) log.cc:844] Serialized 26 byte log entry
0709 13:05:25.736518 (+264098us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:25.736600 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.736681 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:25.820547 (+ 83866us) spinlock_profiling.cc:243] Waited 5.36 ms on lock 0x7b480012c3d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:05:26.386719 (+566172us) raft_consensus.cc:1685] finished
0709 13:05:26.386750 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.387018 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":955,"raft.run_cpu_time_us":4641,"raft.run_wall_time_us":368448,"spinlock_wait_cycles":12330240,"thread_start_us":226775,"threads_started":1,"wal-append.queue_time_us":358968}
I20260709 13:05:26.389205 17184 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:26.390205 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.395985 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 272771us
I20260709 13:05:26.157541 17250 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.406242 17138 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:26.250793 17291 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.363189 17265 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.405035 17254 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:26.376400 17257 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.454129 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57836 (request call id 13) took 1226 ms. Trace:
I20260709 13:05:26.454716 13872 rpcz_store.cc:276] 0709 13:05:25.223608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.440240 (+216632us) service_pool.cc:224] Handling call
0709 13:05:26.449258 (+1009018us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b54002e68d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c342d853 00007fb2c83fb60a 00007fb2c19817d4 00007fb2c1981762 00007fb2c1981691 00007fb2c198160c 00007fb2c19808c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:26.449393 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":925650688}
W20260709 13:05:26.459360 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 474227us
W20260709 13:05:26.449509 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 316820us
I20260709 13:05:26.440936 17266 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:26.465710 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:26.446866 17328 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:26.471688 17298 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:26.472635 17249 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:26.473261 17257 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:26.473809 17333 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.458910 17254 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.448217 16994 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:26.482182 17266 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.467507 17331 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:26.483475 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:26.486440 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.532660 14556 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:26.525673 17266 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.539860 17266 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:26.540035 17342 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:26.540349 17307 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.541292 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:26.542119 13946 raft_consensus.cc:2393] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:26.543879 17269 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.544318 17254 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:26.545239 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 84830us
I20260709 13:05:26.547005 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:26.497020 17298 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.556869 17307 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:26.556555 17328 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:26.566576 17265 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:26.567591 17325 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:26.567516 17341 consensus_queue.cc:1059] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:26.513932 17333 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.578922 17336 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:05:26.536069 16918 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.001s
W20260709 13:05:26.599220 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 49131.7us
W20260709 13:05:26.613299 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.618461 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 164669us
I20260709 13:05:26.627094 17266 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:05:26.636299 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.636482 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 167921us
W20260709 13:05:26.646462 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 99582us
I20260709 13:05:26.649138 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:26.652256 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:26.661778 14556 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:26.622763 17344 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:05:26.666842 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 73759.8us
I20260709 13:05:26.635733 17328 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:26.613101 17109 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.694025 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:26.633775 16918 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.672732 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:05:26.710543 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.656680 17174 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:26.691366 17344 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.671581 17328 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:26.752835 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:26.758570 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:26.697410 17282 consensus_meta.cc:287] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.571s	user 0.013s	sys 0.001s
W20260709 13:05:26.760135 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 90271.3us
I20260709 13:05:26.758859 14558 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:26.765174 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:26.672060 17267 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.768042 13873 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:26.783146 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.785200 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:26.668526 17249 consensus_queue.cc:1059] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
W20260709 13:05:26.795399 14524 leader_election.cc:336] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.645s (SENT)
I20260709 13:05:26.754737 17139 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:26.751675 17338 raft_consensus.cc:3055] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:26.788249 17325 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:26.789984 17091 consensus_meta.cc:287] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.647s	user 0.009s	sys 0.004s
I20260709 13:05:26.769517 17300 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:26.762599 17344 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:26.768101 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 74198.6us
I20260709 13:05:26.764590 17354 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.784794 17031 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:05:26.820354 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 59467.6us
I20260709 13:05:26.802184 17091 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.765396 17282 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.835691 14560 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:26.801784 17338 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:26.807616 17336 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:26.837594 16918 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:26.806180 17300 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.816651 17354 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.840071 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:26.842840 17139 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.814595 17342 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:05:26.850596 17300 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:26.868752 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:26.838378 17328 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:26.872498 17350 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:05:26.873026 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.850855 17336 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:26.900893 17336 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:26.902318 17357 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:05:26.875792 17139 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.878134 17338 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:26.904378 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:26.905375 14552 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:26.905465 17139 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:26.907478 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 45657.7us
I20260709 13:05:26.883526 17267 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:26.853266 17250 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:05:26.886508 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:26.889005 17282 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:26.853617 17333 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:26.853852 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:26.889310 17325 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:26.857955 17031 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:05:26.892004 17109 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:26.926812 13838 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:26.853693 17269 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:26.928280 13838 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:26.928108 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:26.894313 14560 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:26.931658 14556 pending_rounds.cc:85] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting all ops after (but not including) 8
I20260709 13:05:26.875283 17344 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:26.860154 17298 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:26.917464 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.919080 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.923796 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:26.924175 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 107857us
I20260709 13:05:26.908264 17091 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:26.932139 14556 pending_rounds.cc:107] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting uncommitted NO_OP operation due to leader change: 6.9
I20260709 13:05:26.976229 14556 raft_consensus.cc:2889] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:26.929292 16809 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.011s	sys 0.001s
W20260709 13:05:26.932964 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 111641us
I20260709 13:05:26.894754 17092 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:26.971714 16918 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:26.932477 17333 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:26.932557 17139 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:26.986753 17139 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.967891 14551 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:26.987255 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:26.968959 17269 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:26.968174 13839 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:26.978717 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:26.981772 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:26.973594 17328 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:26.993680 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:26.994318 13948 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:26.979549 17213 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:26.987200 17139 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:26.990224 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:26.990339 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:26.986225 17092 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:26.992178 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 344782us
W20260709 13:05:26.992686 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 69091.2us
I20260709 13:05:26.987331 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.002022 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:26.981134 16809 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.013s	sys 0.008s
I20260709 13:05:26.995774 14552 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:26.996637 13866 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:26.979549 16918 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.975881 17109 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:27.038347 16918 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.038489 17109 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.041347 17109 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:27.003237 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.042433 13840 leader_election.cc:400] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7.
I20260709 13:05:27.042773 13840 leader_election.cc:403] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:05:27.004737 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:27.044320 13942 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 6: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:27.044579 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:27.045361 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:27.045873 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 64152.4us
I20260709 13:05:27.046088 13942 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.047245 13989 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:27.047658 17249 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7.
I20260709 13:05:27.048538 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:27.048729 17249 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:26.996333 17298 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:27.002750 13945 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:27.002661 13947 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:27.053081 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.053709 13947 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 4.
I20260709 13:05:27.054607 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.055160 13947 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:27.055956 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:27.056455 13947 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:26.984220 17282 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:27.058363 17282 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:27.058948 17282 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:27.059468 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:27.061902 17341 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:27.062523 17341 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:27.063050 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.009604 13915 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:26.990908 17102 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.003533 17213 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.012387 17354 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:27.014741 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.015965 16983 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:27.073822 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1727) took 1147 ms. Trace:
I20260709 13:05:27.016860 17266 raft_consensus.cc:3055] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:26.990782 17269 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.023483 13943 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:27.027338 13948 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:27.030407 17319 consensus_queue.cc:1059] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:26.997700 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:27.049612 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.981556 17333 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:27.073304 13944 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:05:27.075090 14522 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:27.074429 13946 rpcz_store.cc:276] 0709 13:05:25.917074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.918360 (+  1286us) service_pool.cc:224] Handling call
0709 13:05:27.064581 (+1146221us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b54003309d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:27.064731 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1306019584}
I20260709 13:05:27.064167 17298 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:27.065291 17331 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:27.098870 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 50398.6us
W20260709 13:05:27.090044 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 80587us
I20260709 13:05:27.078258 17269 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:27.083832 14522 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:27.075857 17266 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:27.105667 14522 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:27.081230 17333 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.083487 17298 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:27.063482 17341 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:27.082165 17354 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:27.098129 17031 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:27.119510 14522 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:27.118273 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:27.119175 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 121570us
I20260709 13:05:27.123693 13987 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:27.075157 17102 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.127456 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:27.131227 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.113765 17239 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:27.138893 13942 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:27.116120 17353 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:27.113714 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:27.127241 17349 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:27.160599 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:27.162318 17254 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:27.162925 17254 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.163378 17254 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:27.139971 17319 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:27.136389 17331 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:27.215019 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:27.142773 17239 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:27.146202 17353 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.140894 17344 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:27.220325 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 120489us
W20260709 13:05:27.229264 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 101911us
W20260709 13:05:27.242545 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.247236 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 202738us
I20260709 13:05:27.260310 13989 leader_election.cc:400] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9.
I20260709 13:05:27.264243 13989 leader_election.cc:403] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:05:27.183393 17349 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.266309 17349 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:27.231308 17239 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:27.258517 17353 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.270571 17344 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:27.219067 16983 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:27.272797 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:27.292266 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 165060us
I20260709 13:05:27.299988 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:27.269786 17352 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.302s	user 0.000s	sys 0.003s
I20260709 13:05:27.302392 13987 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:27.309093 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:27.223014 17254 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.294800 17031 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.325742 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:05:27.316265 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 135705us
I20260709 13:05:27.328284 14022 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:05:27.345409 14524 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.038s (SENT)
I20260709 13:05:27.179883 17266 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.301064 17139 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:27.335479 14018 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:05:27.347586 14524 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.047s (SENT)
I20260709 13:05:27.333786 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:27.359627 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.307813 17184 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:27.305665 17320 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:27.364217 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:27.371616 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:27.368530 17349 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.375033 17379 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:05:27.376670 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 76787.4us
I20260709 13:05:27.370452 17213 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:27.348672 17342 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:27.381644 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:27.391103 13947 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:05:27.365265 17184 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.391891 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:27.391906 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:27.318868 17300 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.399334 14018 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:27.401466 17102 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:27.412895 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:27.413564 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 106024us
I20260709 13:05:27.402031 17349 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:27.415329 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:27.419330 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:27.424494 13947 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:27.425012 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.420130 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:05:27.434502 17376 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.136s	user 0.000s	sys 0.002s
I20260709 13:05:27.437613 13947 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:27.439710 14522 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:27.362363 17031 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:27.443148 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:27.447810 14024 raft_consensus.cc:2376] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 6. Re-sending same reply.
I20260709 13:05:27.398523 17184 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:27.398221 17109 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:27.451223 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:27.461737 14023 raft_consensus.cc:2376] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 7. Re-sending same reply.
I20260709 13:05:27.458384 13989 leader_election.cc:400] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9.
I20260709 13:05:27.416803 17325 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:27.468914 16918 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.471719 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:27.475725 17300 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:27.486995 17139 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:27.476737 17378 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:27.445785 17342 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:27.445099 17102 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:27.488468 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:27.495088 14024 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:27.469466 17282 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:27.454134 17331 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:27.492175 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:27.464398 17349 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:27.501685 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:27.499259 13867 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:27.500610 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.502584 14018 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:27.506157 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:27.501062 17325 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:27.513803 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:27.514539 14018 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:27.517714 17031 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:27.517781 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.517786 17332 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:27.518488 17379 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:05:27.519098 17332 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:27.519562 17379 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:27.519922 17379 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:27.520227 17266 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:27.520129 17332 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.520779 17379 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:27.523134 17084 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:27.523857 17282 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:27.532797 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:27.533934 17109 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:27.534873 16983 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:27.535482 16983 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:27.546311 17338 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:27.548199 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:27.549062 14024 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:27.509855 17320 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:27.572939 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 225362us
I20260709 13:05:27.579721 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:27.574039 17332 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:27.500448 17378 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:27.587546 17353 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.588493 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:27.588563 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 75990.9us
I20260709 13:05:27.589574 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.592320 17269 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.593046 17379 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:27.594285 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.623551 13946 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:27.624248 13947 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:27.583967 17254 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:27.597563 17344 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9.
I20260709 13:05:27.598598 17239 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.603021 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:27.603680 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:27.605142 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.614121 14018 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:27.512486 17349 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:27.625061 13838 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:27.677099 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52148.3us
I20260709 13:05:27.657694 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.658634 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.661424 13943 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:27.667068 14018 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:05:27.670311 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.677951 13944 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:27.674759 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.679875 13950 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:27.683781 13949 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 6: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:27.678611 13941 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:27.683637 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:27.684446 13946 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:27.690582 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:27.691376 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.691496 14024 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:27.701012 14022 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
W20260709 13:05:27.694504 17333 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.558s	user 0.015s	sys 0.005s
I20260709 13:05:27.705327 14556 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:27.708336 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 87360.8us
I20260709 13:05:27.673897 17121 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:27.710675 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:27.687570 17348 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:27.711323 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 49363.1us
I20260709 13:05:27.712865 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:27.717283 13867 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:27.714150 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:27.689819 17250 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:27.717177 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:27.720124 14018 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
W20260709 13:05:27.722137 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.721012 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 212246us
I20260709 13:05:27.723438 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:27.724103 14023 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:27.715448 17319 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:27.731012 13944 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:27.733415 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:27.736601 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.738196 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:27.738976 13947 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:27.747493 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.754684 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.721629 17250 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.766575 13946 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:27.761319 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:27.766621 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:27.767663 14023 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 3.
I20260709 13:05:27.764143 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.772725 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.778151 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.783829 13838 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:27.785209 13838 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:27.786304 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:27.709973 17333 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.793289 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.769348 17362 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
W20260709 13:05:27.797652 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 358010us
I20260709 13:05:27.801137 13838 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:27.803407 13867 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:27.804085 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:27.804692 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:27.806583 14017 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:05:27.808497 13943 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:05:27.777204 17328 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.777s	user 0.015s	sys 0.001s
I20260709 13:05:27.811986 16918 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:27.812574 17171 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:27.812388 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:27.813514 13946 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:27.807472 17291 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:27.770135 17250 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:27.765962 17326 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.769906 17109 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
W20260709 13:05:27.827973 14524 leader_election.cc:336] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.630s (SENT)
I20260709 13:05:27.774853 17139 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:27.829752 13872 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:27.737979 17341 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 13:05:27.846514 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:27.796617 17052 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:27.789695 17282 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:27.857009 13989 connection.cc:256] RPC call timeout handler was delayed by -1.2711163100000249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20920000000000002s for a response
I20260709 13:05:27.796883 17249 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:27.839771 13874 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
W20260709 13:05:27.842113 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 108785us
W20260709 13:05:27.842618 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 76111.3us
I20260709 13:05:27.847087 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.847339 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:27.862761 13944 raft_consensus.cc:2410] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:27.863188 13915 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:27.848063 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.725771 17348 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.726878 17378 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.825372 13947 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:27.789258 16771 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:05:27.828855 17369 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:27.862484 13941 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:05:27.846194 17298 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.721s	user 0.009s	sys 0.003s
W20260709 13:05:27.815408 13819 connection.cc:573] client connection to 127.12.109.67:46599 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 13:05:27.855618 14556 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:27.841988 17171 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:27.826761 17366 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:05:27.851337 17326 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:27.908948 17298 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:27.857892 17052 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.796161 17349 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:27.922031 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 73502.3us
I20260709 13:05:27.866847 17249 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.857241 17282 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.912169 13840 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:27.976965 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 64899.2us
I20260709 13:05:27.978686 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:27.950650 17368 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:05:27.978858 13873 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:27.911151 17300 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:27.933249 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71280.3us
I20260709 13:05:27.892462 14551 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:27.924674 17370 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:05:27.942960 17319 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:05:27.989774 17389 log.cc:927] Time spent T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:05:27.925956 17369 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:27.974001 13821 auto_rebalancer.cc:893] Removing replace marker failed: recv error from unknown peer: Transport endpoint is not connected
W20260709 13:05:28.001551 13821 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 14bba4e2cef04a418c6f660869c7e6e5 already in progress
W20260709 13:05:28.001957 13821 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 14bba4e2cef04a418c6f660869c7e6e5 already in progress
I20260709 13:05:28.002769 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.003348 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.004114 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.004743 14015 raft_consensus.cc:2376] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9. Re-sending same reply.
I20260709 13:05:28.004817 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.004114 14021 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:27.942075 17052 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:27.875785 17386 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.990206 17269 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:28.006865 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.007290 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.007700 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:27.988866 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:28.005435 14024 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:28.008870 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.009334 17386 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.010680 14021 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:27.980614 17213 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:27.980098 17249 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:27.979929 17282 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:27.977340 17352 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:05:27.977818 17341 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:27.996073 17239 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.029657 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:27.991178 17139 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:27.992844 17174 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:28.028934 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:27.999502 17353 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:28.022409 17325 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.996244 17291 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:28.029876 17384 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:05:28.044132 14023 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:05:28.087795 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.088479 14023 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:28.089972 17404 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.090595 17404 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.093248 17351 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:27.982182 17184 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.095863 17184 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:28.097792 14016 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:28.099148 16983 consensus_meta.cc:287] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.001s
I20260709 13:05:28.099632 16983 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.103816 17386 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:28.101802 16983 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.095924 17404 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:28.054675 17387 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 13:05:28.054994 17326 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:28.058341 14018 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:28.059144 17393 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:05:28.011065 17109 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:05:28.061224 17369 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.062819 14019 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:28.132689 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.030303 17171 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.135542 13941 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:28.070345 17298 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:28.073326 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:28.072525 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:28.023470 17213 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:28.085887 17291 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.045444 17341 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.103842 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.052966 17348 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.119745 17401 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:28.120374 17383 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:05:28.039474 17121 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:28.120247 14023 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:28.172788 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:27.996534 17300 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.022727 17319 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.104660 17362 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:28.194569 17300 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.194779 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.195492 13873 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:05:28.199293 17333 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:28.124840 14015 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:28.125876 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:28.127084 13872 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
W20260709 13:05:28.201047 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 75313.7us
I20260709 13:05:28.201308 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 479) took 1226 ms. Trace:
I20260709 13:05:28.201761 13872 rpcz_store.cc:276] 0709 13:05:26.975094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.975232 (+   138us) service_pool.cc:224] Handling call
0709 13:05:28.201136 (+1225904us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b540026bbd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:28.201249 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1935190912}
I20260709 13:05:28.135510 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:28.132107 17344 consensus_queue.cc:1059] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:05:28.203297 14552 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:28.203347 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:28.203954 17344 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:28.204329 17395 raft_consensus.cc:3055] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:28.204684 17344 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:28.204751 17395 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:28.205220 17344 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.205331 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.205684 17344 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:28.136366 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.206355 14018 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:28.205866 17395 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.206622 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.207284 17395 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:28.207736 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.146759 14556 raft_consensus.cc:2368] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10.
W20260709 13:05:28.209102 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.172312 17401 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.209945 14017 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:28.211261 17401 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:28.211865 17401 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:05:28.148692 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 76260.4us
I20260709 13:05:28.212950 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:28.214231 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:28.159839 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.160808 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.163288 13941 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:28.164722 13838 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:28.167661 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:28.217139 13950 raft_consensus.cc:2410] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:28.169529 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:28.171032 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.171490 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:28.218858 13949 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:05:28.171805 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:28.219749 13942 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:28.173780 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.220521 14023 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 4: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:28.151129 17213 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:28.224478 17282 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.150921 17336 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.181370 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:28.226449 17266 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.007s
I20260709 13:05:28.172883 17121 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:28.189520 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:28.227823 13948 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:28.190320 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:28.227946 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 63318.1us
I20260709 13:05:28.228436 13945 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:28.190412 14551 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:28.190829 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:28.234611 13867 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:28.191551 16983 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:28.192852 17319 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:28.122743 17325 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:28.149700 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 160969us
I20260709 13:05:28.247745 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:28.190495 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.249155 14016 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:05:28.214812 17341 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:28.153196 17109 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:28.215015 17400 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:05:28.177030 17084 consensus_meta.cc:287] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.653s	user 0.014s	sys 0.000s
I20260709 13:05:28.183473 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:28.231043 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.236294 13874 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:28.261775 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:28.265564 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.185714 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:28.268486 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 82898.9us
I20260709 13:05:28.232194 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:28.226616 17249 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.271569 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:28.273415 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:28.240819 17052 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.274467 13948 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:28.245018 17266 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:28.275949 17282 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.276289 17052 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.253849 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:28.255646 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:28.227169 17121 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:28.271538 17400 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:28.279958 17400 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.282269 16994 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.247349 17362 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:28.226336 17336 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.282899 16994 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:28.284510 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52445.5us
I20260709 13:05:28.284674 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.285319 14019 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:28.285471 17282 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:28.286121 17282 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:28.287094 17282 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.289752 13773 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54072 (request call id 137) took 10256 ms. Trace:
I20260709 13:05:28.290041 13942 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:28.295012 13943 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: []
W20260709 13:05:28.298447 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 84283.4us
I20260709 13:05:28.298220 13772 catalog_manager.cc:5697] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 3 to 6, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 6 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:28.299757 17325 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:28.256381 17298 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:28.282991 17362 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:28.302278 14021 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:28.302686 17362 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.305564 17249 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.307065 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:28.310199 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62558.8us
I20260709 13:05:28.311234 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:28.314195 14017 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:28.312484 14019 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:28.314998 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:28.313625 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:28.307960 14018 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:28.311728 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:28.256176 17338 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.709s	user 0.009s	sys 0.005s
I20260709 13:05:28.317759 13874 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:28.319981 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:28.329504 13943 raft_consensus.cc:2368] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11.
I20260709 13:05:28.329595 13941 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:28.320518 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.330778 13872 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:28.323446 14522 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:28.323727 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.326726 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:28.328088 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:28.312858 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52211.5us
I20260709 13:05:28.333022 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:28.333925 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:28.340201 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 424587us
I20260709 13:05:28.340835 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.300449 13772 catalog_manager.cc:5697] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 5 to 6, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 6 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:28.348762 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:28.348953 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.350137 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.308929 17349 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:28.357612 13838 leader_election.cc:400] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11.
I20260709 13:05:28.354069 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.360545 13838 leader_election.cc:403] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:05:28.353148 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:28.350298 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.349321 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:28.354419 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.363951 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:28.364548 17171 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:28.326313 17319 consensus_queue.cc:1059] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:05:28.365717 16771 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:28.364508 17341 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.366340 16771 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:28.366593 17320 raft_consensus.cc:3055] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:28.367027 17320 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:28.367358 16771 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.367431 13772 catalog_manager.cc:5697] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.368031 17320 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.367595 17319 consensus_queue.cc:1059] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:05:28.369275 17320 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:28.369760 17319 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:28.370143 17319 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:28.370127 13772 catalog_manager.cc:5697] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:28.371253 17319 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.366365 17341 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.372202 13772 catalog_manager.cc:5697] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 5 to 7, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 7 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:28.372718 17319 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:28.374311 13772 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:28.375515 17341 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:28.379762 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.380558 13873 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:28.380482 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.381131 14016 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:28.381657 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:28.382951 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:28.384255 14018 raft_consensus.cc:2368] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10.
I20260709 13:05:28.380226 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:28.380226 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.387794 13867 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:28.389395 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.390239 14021 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:28.290189 13773 rpcz_store.cc:276] 0709 13:05:18.033430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.063049 (+ 29619us) service_pool.cc:224] Handling call
0709 13:05:28.289640 (+10226591us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:25.602282 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:25.603190 (+   908us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:25.603248 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:05:25.603274 (+    26us) tablet.cc:662] Decoding operations
0709 13:05:25.604540 (+  1266us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:25.604651 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:25.604679 (+    28us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:05:25.637158 (+ 32479us) tablet.cc:701] Row locks acquired
0709 13:05:25.650306 (+ 13148us) write_op.cc:260] PREPARE: finished
0709 13:05:25.650549 (+   243us) write_op.cc:270] Start()
0709 13:05:25.650715 (+   166us) write_op.cc:276] Timestamp: P: 1783602325650473 usec, L: 0
0709 13:05:25.650738 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:05:25.651597 (+   859us) log.cc:844] Serialized 4939 byte log entry
0709 13:05:26.155148 (+503551us) op_driver.cc:464] REPLICATION: finished
0709 13:05:26.516192 (+361044us) write_op.cc:301] APPLY: starting
0709 13:05:26.516336 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:26.516734 (+   398us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:26.516925 (+   191us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:26.532211 (+ 15286us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:26.532726 (+   515us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:05:26.532800 (+    74us) write_op.cc:312] APPLY: finished
0709 13:05:26.533347 (+   547us) log.cc:844] Serialized 89 byte log entry
0709 13:05:26.533556 (+   209us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:26.533717 (+   161us) write_op.cc:454] Released schema lock
0709 13:05:26.533795 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:26.557322 (+ 23527us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:26.557360 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":318179,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":360658,"apply.run_cpu_time_us":8856,"apply.run_wall_time_us":38222,"num_ops":12,"prepare.queue_time_us":303946,"prepare.run_cpu_time_us":22230,"prepare.run_wall_time_us":123518,"raft.queue_time_us":69851,"raft.run_cpu_time_us":103671,"raft.run_wall_time_us":273777,"replication_time_us":504143,"thread_start_us":555243,"threads_started":4,"wal-append.queue_time_us":155009,"wal-append.run_cpu_time_us":39482,"wal-append.run_wall_time_us":1691488}]]}
I20260709 13:05:28.395553 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.396236 14022 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
W20260709 13:05:28.395566 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 46905.1us
I20260709 13:05:28.397168 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.398175 14017 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:28.399777 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.332674 16983 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:28.349759 17267 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:28.379268 13772 catalog_manager.cc:5697] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 4, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 4 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.355229 17300 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:28.416983 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.418299 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.418129 13772 catalog_manager.cc:5697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.419064 14022 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:28.419987 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.420598 14016 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:05:28.421458 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.420609 13772 catalog_manager.cc:5697] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 1 to 4, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 4 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.360008 17349 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:28.398986 17250 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.000s
I20260709 13:05:28.405826 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:28.349639 17298 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:28.423179 13772 catalog_manager.cc:5697] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 1 to 2, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.425043 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:28.382001 17266 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:28.441200 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:28.395093 17171 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:28.444250 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:28.348403 17249 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 13:05:28.411113 14018 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:28.451206 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:28.454335 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.457824 14022 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:28.458176 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.403074 17052 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11.
I20260709 13:05:28.451442 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:28.405081 16983 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.464798 13944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "617d39d9397c4724be821e5efacbde17"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:05:28.467731 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.475944 14018 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:28.469077 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.476465 13772 catalog_manager.cc:5697] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:28.430665 17378 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:28.480975 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:28.439606 17349 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:28.483084 14017 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:28.484329 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.417296 17300 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:28.482816 13867 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:28.467113 16918 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:28.455353 17139 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:28.475634 13944 raft_consensus.cc:606] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:28.446853 17366 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:28.487068 13772 catalog_manager.cc:5697] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.465744 17348 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:28.494467 13868 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:28.441809 17266 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:28.512574 17381 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 13:05:28.512982 17266 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:28.443614 17298 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:28.514340 17298 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:28.514618 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.515414 14016 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:05:28.515563 17391 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:05:28.516363 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.516973 14016 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:28.488008 17336 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.495528 16994 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.485821 16983 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:28.518409 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.519196 14018 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:28.520381 14018 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:28.499199 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:28.524031 17341 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.524524 17341 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.524966 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.509133 17052 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.526912 17341 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:28.526646 17052 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.475106 17267 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:28.490792 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:28.404070 17213 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.456452 17171 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:28.500069 16918 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:28.470028 17325 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:28.485773 13945 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:28.511866 17406 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 13:05:28.517761 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.531163 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.497756 17265 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:28.551373 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 60651us
I20260709 13:05:28.502952 17139 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:28.553174 13871 raft_consensus.cc:2393] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:28.491853 17338 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:28.446389 17250 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.558696 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.562940 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:28.565104 14020 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:28.570081 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:28.501355 17300 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:28.572909 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:28.573741 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:28.552955 17171 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:28.506739 17378 raft_consensus.cc:697] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:28.588299 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:28.553349 16918 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:28.590080 14021 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:28.583391 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.619168 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:28.624264 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.625448 14021 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:28.626556 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:28.621564 14016 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:28.589534 17139 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:28.591842 17348 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:28.633121 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:28.633441 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:28.639571 13944 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:28.640028 13915 leader_election.cc:400] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10.
I20260709 13:05:28.640413 13915 leader_election.cc:403] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:05:28.599184 17396 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:28.645295 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:28.644950 13945 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:28.645471 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:28.615070 17342 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:28.656991 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:28.657406 13941 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:28.624717 17052 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.622970 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:28.649259 17174 consensus_meta.cc:287] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.578s	user 0.011s	sys 0.000s
I20260709 13:05:28.665169 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:28.668073 13867 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:05:28.650756 17410 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:05:28.641299 17291 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:28.641409 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:28.657334 17338 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:28.661755 17109 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:28.686179 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:28.660012 17336 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:28.656884 17348 raft_consensus.cc:697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:28.659837 16994 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10.
I20260709 13:05:28.695091 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:28.664582 17084 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:28.696727 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.703557 13867 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:28.704252 13838 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:28.670110 17174 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:28.710409 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:28.716447 13871 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:28.716630 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.717885 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.718943 13941 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:28.720079 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:28.722141 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.691308 17052 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.741217 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:28.693972 17109 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.737363 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:28.738605 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.734969 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:28.757328 13873 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:28.656409 17267 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.758715 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.705135 17328 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:28.700826 17336 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:28.762072 17414 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:05:28.705490 17362 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:28.714391 17282 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:28.714761 16983 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.792244 17282 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:28.762197 17121 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.795043 16983 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.795821 17121 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.744303 17396 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:28.796562 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:28.797101 13870 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:28.755854 17342 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:28.797719 17362 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.765133 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:28.766183 17415 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:05:28.782899 17336 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:28.798204 17342 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.800997 17342 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:28.793392 17282 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.805189 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.650512 17378 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.745738 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.768792 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.705752 17084 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:28.780102 17291 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:28.783792 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.822911 13839 leader_election.cc:400] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10.
I20260709 13:05:28.787108 17213 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:28.766978 17348 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:28.798489 17408 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:05:28.759858 17109 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:28.833673 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:28.828248 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.798256 17362 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:28.829207 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.836071 14018 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:28.836220 13989 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:28.828447 13871 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:28.807950 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:28.808730 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.828338 13839 leader_election.cc:403] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:05:28.847034 13866 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:28.848219 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:28.831416 17395 consensus_meta.cc:287] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.624s	user 0.013s	sys 0.000s
I20260709 13:05:28.851259 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:28.886919 13873 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:28.855597 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.856781 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:28.894006 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.894914 14016 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:28.889045 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.897490 13873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:05:28.898289 13873 raft_consensus.cc:606] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:28.806735 17250 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.760967 17052 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.830014 17084 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:28.850731 17325 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:28.837492 17396 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.858657 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.911083 13867 raft_consensus.cc:2410] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:28.864123 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.844375 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:28.865095 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.891842 17121 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10.
I20260709 13:05:28.920372 13870 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:28.867014 17109 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.868708 13942 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:05:28.869177 13866 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:28.922416 13866 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:05:28.869582 17344 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.628s	user 0.007s	sys 0.004s
W20260709 13:05:28.870766 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 99365.9us
I20260709 13:05:28.927120 13866 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:28.927205 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:28.874825 17139 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.877475 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:28.878008 14018 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:28.857775 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:28.905102 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 48427.1us
I20260709 13:05:28.929631 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.931154 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.931783 14018 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:28.905390 13949 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:28.905584 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.911026 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:28.912590 13871 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:28.883736 13988 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:28.919518 13869 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:28.933279 13989 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:28.805956 17266 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.935395 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:28.936120 13872 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:28.938712 17266 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.939929 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:28.909514 17319 consensus_meta.cc:287] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.000s
I20260709 13:05:28.942984 13868 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:28.947726 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:28.949738 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 92028.9us
W20260709 13:05:28.950915 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 67273.5us
I20260709 13:05:28.957192 16771 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:28.958158 16771 raft_consensus.cc:3055] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:28.958519 16771 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:28.958887 17269 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:28.959460 16771 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.960695 16771 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:28.962411 13870 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:28.963806 13870 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:28.960352 17300 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.926401 17396 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.966818 14024 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:28.966199 17300 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:28.967988 17396 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:28.969144 13867 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:28.906615 17366 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:28.970975 17109 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.975607 14016 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:28.975696 14018 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:28.954655 17344 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:28.971014 13868 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:28.942247 17298 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:28.982371 17344 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:28.983168 17325 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:28.984975 17298 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:28.985090 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.981681 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:28.969076 17139 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:28.993459 14019 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:29.004119 14018 raft_consensus.cc:2410] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:28.995141 13948 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:29.005125 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:28.999590 14015 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
W20260709 13:05:29.002385 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 47713.2us
W20260709 13:05:29.002592 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 54983.9us
I20260709 13:05:28.993983 13875 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:05:29.007243 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:29.012815 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:29.015692 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.015882 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.016829 14016 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:29.016820 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:29.023775 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:29.001194 17031 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:29.004434 17320 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:29.017664 14023 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
W20260709 13:05:29.014991 17349 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.000s
I20260709 13:05:29.004719 17319 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:28.946751 17250 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:05:29.046797 17338 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:29.037626 17421 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.349s	user 0.000s	sys 0.002s
I20260709 13:05:29.012843 17265 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:29.051560 17349 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:29.053056 14556 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:29.052954 14016 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
W20260709 13:05:29.053709 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 126611us
I20260709 13:05:29.056427 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:29.064703 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:29.018348 17379 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:29.022324 17400 raft_consensus.cc:2764] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:05:29.071169 17400 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:29.067641 17328 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:29.076566 16983 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:29.078183 17109 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:29.078666 17109 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:29.079047 17109 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:29.081825 17291 raft_consensus.cc:993] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: : Instructing follower d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:05:29.081893 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:29.082474 17291 raft_consensus.cc:1081] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Signalling peer d1b4fd04200247d99ba9ad1fbeb42b42 to start an election
I20260709 13:05:29.083750 17052 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:29.084290 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:29.086040 13948 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:29.086510 17084 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.087105 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:29.087610 13945 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:29.088711 13944 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:29.089226 17342 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:29.094892 13944 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:05:29.104075 13868 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:05:29.077800 16983 raft_consensus.cc:3055] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:29.106388 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:29.109050 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.036763 17320 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.018538 17344 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:05:29.054461 17395 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.115078 14016 raft_consensus.cc:2376] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 7. Re-sending same reply.
I20260709 13:05:29.040763 17325 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:29.040557 17326 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:29.107213 13942 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
W20260709 13:05:29.138636 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 319548us
I20260709 13:05:29.139922 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:29.157128 13868 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:29.143095 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:29.158432 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.159242 14016 raft_consensus.cc:2368] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8.
I20260709 13:05:29.143790 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.163692 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:29.146513 13871 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:29.143213 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.165007 14023 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:29.164294 14019 raft_consensus.cc:2368] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10.
I20260709 13:05:29.113049 16983 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:29.172796 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:29.069029 17396 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 13:05:29.141114 17395 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.174768 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.196033 14016 raft_consensus.cc:2376] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9. Re-sending same reply.
I20260709 13:05:29.178507 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:29.197178 14019 raft_consensus.cc:2410] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:05:29.181103 17366 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:29.182667 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:29.184932 13868 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
W20260709 13:05:29.187433 17171 consensus_meta.cc:287] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.554s	user 0.005s	sys 0.006s
I20260709 13:05:29.124338 17319 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:29.172925 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.206496 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:29.207500 14023 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:29.190627 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 106407us
I20260709 13:05:29.209615 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:05:29.206358 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 139717us
I20260709 13:05:29.212615 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:29.214354 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:29.215462 13868 raft_consensus.cc:2393] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:29.134548 17349 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:29.133688 17320 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:29.199611 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.123366 17239 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:05:29.141940 17174 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:05:29.203657 14558 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:29.233588 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.203740 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:29.204600 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.208436 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.234973 14019 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:29.142529 17052 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
W20260709 13:05:29.222612 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 216234us
I20260709 13:05:29.235371 14023 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:29.231509 14017 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:29.239246 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:29.165704 17338 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:05:29.237911 14024 raft_consensus.cc:2376] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 7. Re-sending same reply.
I20260709 13:05:29.243269 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:29.243270 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 151610us
I20260709 13:05:29.248193 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:29.242270 17213 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:29.251991 17213 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:29.252938 17213 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.237208 17396 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:29.254360 17342 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:29.254701 17325 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:29.254786 17396 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.257128 17396 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:29.253348 17084 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:29.242520 17319 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:29.237710 14551 raft_consensus.cc:2368] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14.
W20260709 13:05:29.258868 16918 consensus_meta.cc:287] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.628s	user 0.012s	sys 0.000s
W20260709 13:05:29.260246 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.244431 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:29.236066 17341 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:29.265522 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:29.266197 13871 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:29.267555 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:29.263202 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:29.270968 13839 leader_election.cc:400] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14.
I20260709 13:05:29.271309 13839 leader_election.cc:403] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:05:29.271893 13868 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:29.273345 14556 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:29.275847 13868 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:29.276065 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:29.282867 13942 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:29.217773 17171 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:29.275219 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:29.288549 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:29.258899 17400 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:29.299063 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:29.301708 13875 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:29.302867 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 382) took 1229 ms. Trace:
I20260709 13:05:29.237589 17336 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:29.294315 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 46224.4us
I20260709 13:05:29.303274 14015 rpcz_store.cc:276] 0709 13:05:28.073134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.207504 (+134370us) service_pool.cc:224] Handling call
0709 13:05:29.302678 (+1095174us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b54003bbfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:29.302794 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1600679808}
I20260709 13:05:29.308660 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:29.298332 13872 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:29.300122 13874 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:29.265178 17031 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:29.302271 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:29.265661 17341 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:29.309336 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:29.295588 13871 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:29.311014 13873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:05:29.345659 13873 raft_consensus.cc:606] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:05:29.279762 17250 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14.
W20260709 13:05:29.348726 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55333.9us
I20260709 13:05:29.329950 14560 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:29.259375 16918 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.273013 17453 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:29.261991 17362 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:29.305488 17423 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:29.368417 17267 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:05:29.297709 17406 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:29.303360 17401 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.299975 17400 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:29.347183 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46234us
I20260709 13:05:29.292764 16983 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:29.398650 16983 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:29.293385 17265 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:05:29.403564 17348 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:29.405417 17395 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:29.407943 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.357879 17336 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:29.366746 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:29.358477 17453 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:29.377538 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 68406.3us
I20260709 13:05:29.412631 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:29.293699 17441 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:05:29.379678 13866 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:29.416200 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:29.386445 13941 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 14: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:29.346370 13873 raft_consensus.cc:614] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:29.414381 14020 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:29.400223 17438 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.183s	user 0.003s	sys 0.000s
I20260709 13:05:29.417768 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.418601 14020 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:29.419461 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:29.419523 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.420168 14020 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:29.420714 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.420961 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.421468 14020 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:29.421789 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.422314 14021 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:29.384687 17338 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:29.369120 17378 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:05:29.425500 13949 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:05:29.427073 17440 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:29.427863 17031 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.442744 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:29.408088 17400 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:29.414849 17423 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:05:29.461179 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 45047.9us
I20260709 13:05:29.423204 17265 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:29.415907 17453 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:29.463239 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:29.464388 14020 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:05:29.454237 17333 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:29.468629 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 13:05:29.471716 16771 consensus_meta.cc:287] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.003s
I20260709 13:05:29.471022 17423 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:29.474215 17437 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.232s	user 0.006s	sys 0.000s
I20260709 13:05:29.434335 17440 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:29.493719 14021 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:29.410952 17401 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.485500 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:29.409878 17341 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.499603 13840 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:29.497468 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.477533 17320 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.434489 17338 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.503031 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.515312 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:29.496301 17265 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:29.523731 14021 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:29.467928 17241 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 13:05:29.523269 17464 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:05:29.521420 17052 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:29.536773 13840 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:29.539011 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:29.529703 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:29.534919 14018 raft_consensus.cc:2368] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7.
I20260709 13:05:29.512089 16918 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:29.507694 17250 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:29.545624 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:29.524755 17341 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:29.526666 17139 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:29.566895 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:05:29.565029 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 257231us
I20260709 13:05:29.577037 14024 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:29.577356 14021 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:29.498672 17109 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:29.535432 17366 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:29.582721 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:29.588682 17463 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:05:29.579854 17084 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:29.589540 17109 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:29.590668 13950 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:29.591841 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:05:29.592731 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 182378us
W20260709 13:05:29.527611 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:05:29.544847 17052 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:29.622730 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.580018 17344 raft_consensus.cc:3055] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:29.637197 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:29.566617 17442 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:29.660272 14021 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:29.574204 16918 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:05:29.650172 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 67570.2us
I20260709 13:05:29.690618 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:29.580860 17139 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:29.598827 17213 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:29.679955 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:29.656709 17396 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:29.656287 17458 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.657114 17366 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:29.662952 17052 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:29.710033 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:29.737674 14021 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:29.738608 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1695) took 1004 ms. Trace:
I20260709 13:05:29.739053 14021 rpcz_store.cc:276] 0709 13:05:28.733974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.700245 (+966271us) service_pool.cc:224] Handling call
0709 13:05:29.737194 (+ 36949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.738247 (+  1053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.738529 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:29.662550 13758 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 136471us
I20260709 13:05:29.663556 17344 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:29.742360 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 205700us
I20260709 13:05:29.687783 17109 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:29.748426 13840 leader_election.cc:400] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8.
I20260709 13:05:29.749387 13840 leader_election.cc:403] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:05:29.665820 17454 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.729143 17319 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:29.736467 17458 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:29.758904 17465 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:05:29.760079 17401 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:29.764199 17467 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.335s	user 0.001s	sys 0.000s
W20260709 13:05:29.746927 17457 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.166s	user 0.001s	sys 0.001s
I20260709 13:05:29.706897 17265 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:29.799036 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2430) took 1281 ms. Trace:
I20260709 13:05:29.765971 17139 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.659245 17440 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.770530 17366 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.741809 17472 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:29.726615 16918 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:29.727463 17362 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:29.774915 17462 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:05:29.775600 17344 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.749236 17473 consensus_queue.cc:1059] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:05:29.746351 17171 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:29.780665 17333 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8.
I20260709 13:05:29.781409 13840 leader_election.cc:400] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10.
W20260709 13:05:29.783375 17459 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:05:29.784123 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1675) took 1056 ms. Trace:
I20260709 13:05:29.862476 14018 rpcz_store.cc:276] 0709 13:05:28.727889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.015632 (+287743us) service_pool.cc:224] Handling call
0709 13:05:29.783919 (+768287us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b540020d0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:29.784060 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191597824}
I20260709 13:05:29.774997 17320 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:29.788537 13875 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:29.793547 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2432) took 1275 ms. Trace:
I20260709 13:05:29.665880 17328 consensus_queue.cc:1059] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:05:29.883431 17328 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:29.883816 17328 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:29.884676 17328 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.751397 17291 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:29.886718 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2433) took 1330 ms. Trace:
I20260709 13:05:29.887023 13945 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:05:29.887115 14022 rpcz_store.cc:276] 0709 13:05:28.556150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.248253 (+692103us) service_pool.cc:224] Handling call
0709 13:05:29.248592 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.248881 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:29.248914 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.249393 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.384332 (+134939us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:29.384378 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.384459 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.886499 (+502040us) raft_consensus.cc:1685] finished
0709 13:05:29.886522 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.886660 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":134497,"threads_started":1,"wal-append.queue_time_us":288482}
I20260709 13:05:29.889358 14022 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:05:29.885993 17328 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:29.891361 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:29.889889 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1698) took 1035 ms. Trace:
I20260709 13:05:29.894342 14022 rpcz_store.cc:276] 0709 13:05:28.854552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.888671 (+1034119us) service_pool.cc:224] Handling call
0709 13:05:29.889007 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.889684 (+   677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.889837 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:29.895368 14018 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:05:29.895812 14022 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:05:29.819263 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.861184 13840 leader_election.cc:403] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:05:29.874647 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:29.877053 16918 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.880478 17458 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:29.914454 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:29.896345 14023 rpcz_store.cc:276] 0709 13:05:28.555757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.261018 (+705261us) service_pool.cc:224] Handling call
0709 13:05:29.261613 (+   595us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.265680 (+  4067us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:29.265727 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.266334 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.580308 (+313974us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:29.697230 (+116922us) log.cc:844] Serialized 14 byte log entry
0709 13:05:29.703222 (+  5992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.703338 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.793308 (+ 89970us) raft_consensus.cc:1685] finished
0709 13:05:29.793331 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.793485 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19405,"prepare.queue_time_us":116557,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":11677,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":40648,"thread_start_us":271239,"threads_started":1,"wal-append.queue_time_us":458000}
I20260709 13:05:29.861088 17139 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:29.816732 17451 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:29.907936 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:29.909652 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.915485 13946 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:29.799464 14019 rpcz_store.cc:276] 0709 13:05:28.517649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.240904 (+723255us) service_pool.cc:224] Handling call
0709 13:05:29.241263 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.246607 (+  5344us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:29.246638 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.247166 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.377118 (+129952us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:29.377174 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.377449 (+   275us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.798772 (+421323us) raft_consensus.cc:1685] finished
0709 13:05:29.798795 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.798949 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4890,"raft.queue_time_us":9640,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":72613,"threads_started":1,"wal-append.queue_time_us":280332}
I20260709 13:05:29.802322 17454 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:29.924575 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 143391us
I20260709 13:05:29.896179 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1697) took 1042 ms. Trace:
I20260709 13:05:29.897603 17400 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:29.922642 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 457637us
I20260709 13:05:29.925410 14018 rpcz_store.cc:276] 0709 13:05:28.853828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.894622 (+1040794us) service_pool.cc:224] Handling call
0709 13:05:29.895063 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.895862 (+   799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.896112 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:29.926102 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:29.926276 14023 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:29.927269 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2431) took 1409 ms. Trace:
I20260709 13:05:29.931234 14017 rpcz_store.cc:276] 0709 13:05:28.517925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.260275 (+742350us) service_pool.cc:224] Handling call
0709 13:05:29.260593 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.262712 (+  2119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:29.262757 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.268657 (+  5900us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.327779 (+ 59122us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:29.327925 (+   146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.328035 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.927047 (+599012us) raft_consensus.cc:1685] finished
0709 13:05:29.927071 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.927211 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18866,"raft.queue_time_us":4358,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":4231,"thread_start_us":23786,"threads_started":1,"wal-append.queue_time_us":226106}
I20260709 13:05:29.909606 17333 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:29.916743 17344 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:29.948766 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:29.950626 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:29.953732 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:29.952925 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 208039us
I20260709 13:05:29.955606 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:29.921878 17174 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10.
I20260709 13:05:29.958491 13840 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:29.960915 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:29.929737 14022 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "617d39d9397c4724be821e5efacbde17"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:49644
I20260709 13:05:29.965358 14022 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:29.957648 17282 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:29.969050 16771 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:29.969604 16771 raft_consensus.cc:697] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:29.970253 17369 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:29.965829 14022 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:29.966723 17084 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:29.973023 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:29.975522 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:29.977526 13840 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:29.976749 13944 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:29.972056 17476 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:29.981253 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:29.982949 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:29.955576 17250 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:29.955438 17441 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:29.984294 17171 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:30.008666 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:30.011737 13874 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:29.924751 17451 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.072387 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 580) took 1162 ms. Trace:
I20260709 13:05:30.035418 14017 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:05:29.985489 17369 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:30.035912 17453 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.540s	user 0.012s	sys 0.000s
W20260709 13:05:30.067673 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 59152.5us
W20260709 13:05:30.068310 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 354251us
W20260709 13:05:30.064837 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 102064us
I20260709 13:05:30.028820 17458 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:30.073012 13941 rpcz_store.cc:276] 0709 13:05:28.909880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.910782 (+   902us) service_pool.cc:224] Handling call
0709 13:05:30.072109 (+1161327us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b540004bfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:30.072306 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026463488}
I20260709 13:05:30.006276 17423 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:30.081861 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:30.027925 17450 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:29.964362 17333 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:29.970574 16771 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.092520 14022 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.012173 17341 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:30.087090 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:30.088737 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:30.034665 17174 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:30.006979 17298 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:30.080534 13872 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:05:30.063284 17326 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:30.069412 17250 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:30.106726 14015 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:05:30.085801 17458 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.005103 17452 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:05:30.087042 17171 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:30.107537 17381 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:05:30.122660 17441 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:05:30.132580 17447 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.111243 17174 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.115379 17476 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:30.122825 17031 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:30.118960 13840 leader_election.cc:400] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7.
I20260709 13:05:30.145223 13840 leader_election.cc:403] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:05:30.132957 17171 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.139882 17381 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.029069 17267 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:05:30.133265 17447 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.143380 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:30.169446 13941 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:30.169767 13941 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:30.089289 17453 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.153237 17441 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:30.153692 17440 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:30.175509 17440 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.175757 17453 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.176028 17440 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.156752 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:30.179472 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:30.116921 17458 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:30.161131 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1696) took 1307 ms. Trace:
I20260709 13:05:30.186326 14021 rpcz_store.cc:276] 0709 13:05:28.853467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.750862 (+897395us) service_pool.cc:224] Handling call
0709 13:05:29.801052 (+ 50190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.801458 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:29.814022 (+ 12564us) log.cc:844] Serialized 14 byte log entry
0709 13:05:30.059970 (+245948us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:30.076554 (+ 16584us) log.cc:844] Serialized 26 byte log entry
0709 13:05:30.076771 (+   217us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:30.080955 (+  4184us) log.cc:844] Serialized 14 byte log entry
0709 13:05:30.081288 (+   333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.081372 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:30.160867 (+ 79495us) raft_consensus.cc:1685] finished
0709 13:05:30.160894 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.161062 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28419,"prepare.queue_time_us":13976,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2908,"raft.queue_time_us":29190,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":24456,"spinlock_wait_cycles":28371584,"thread_start_us":216485,"threads_started":1,"wal-append.queue_time_us":316114}
I20260709 13:05:30.112025 17362 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:30.110572 17369 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.191351 17369 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:30.194857 17319 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:30.195863 17319 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:30.207707 17401 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:30.208570 17401 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.219255 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:30.222168 17401 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:30.225948 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:30.189654 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:30.227324 14552 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:30.229072 13839 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:30.225827 17440 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.240139 17440 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:30.240792 17440 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.241251 17440 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:30.170527 13941 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.244774 13941 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:30.245056 17451 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.247743 17333 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.248558 17476 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.249164 17476 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.115518 17298 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.172072 17250 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 13:05:30.105945 17341 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.134127 17326 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.118782 17450 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:05:30.166756 17291 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:30.170644 17381 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.149843 17348 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.113s
I20260709 13:05:30.165814 17031 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.259156 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2571) took 1536 ms. Trace:
I20260709 13:05:30.285290 13866 rpcz_store.cc:276] 0709 13:05:28.722919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.944704 (+221785us) service_pool.cc:224] Handling call
0709 13:05:28.944959 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.416322 (+471363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.258853 (+842531us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b54001aa754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:30.259058 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":994948864}
I20260709 13:05:30.286633 13941 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
W20260709 13:05:30.260205 17052 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.000s
I20260709 13:05:30.260653 17291 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.290320 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:30.291036 17052 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.278604 13870 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:30.294081 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:30.281263 14017 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:30.281665 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.301420 13873 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
W20260709 13:05:30.283424 17479 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.298s	user 0.000s	sys 0.004s
I20260709 13:05:30.251667 17476 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.283746 14021 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:30.300055 13942 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:30.302613 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.301154 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.305070 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:30.308467 13874 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:30.306684 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.337276 13869 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:30.278458 17450 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.340590 17450 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.256138 17298 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.256577 17400 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.343338 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.260028 17326 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:30.344403 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:30.349269 13874 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:30.346843 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:30.349504 14021 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:30.310115 13867 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
W20260709 13:05:30.260764 17495 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:05:30.351588 13873 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:30.352782 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.352874 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.275106 17174 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.354221 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.365001 13874 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:30.365325 13867 raft_consensus.cc:2393] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 4: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:30.273489 17282 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:30.366838 17482 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.202s	user 0.001s	sys 0.003s
W20260709 13:05:30.370251 17485 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.215s	user 0.001s	sys 0.003s
W20260709 13:05:30.370682 17489 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.159s	user 0.000s	sys 0.004s
I20260709 13:05:30.380383 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1686) took 1649 ms. Trace:
I20260709 13:05:30.380825 14024 rpcz_store.cc:276] 0709 13:05:28.731232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.651385 (+920153us) service_pool.cc:224] Handling call
0709 13:05:29.651972 (+   587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.688281 (+ 36309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:29.688323 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.700045 (+ 11722us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.938926 (+238881us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:29.964988 (+ 26062us) log.cc:844] Serialized 14 byte log entry
0709 13:05:29.965216 (+   228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.965295 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:30.380167 (+414872us) raft_consensus.cc:1685] finished
0709 13:05:30.380187 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.380317 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35699,"prepare.queue_time_us":28857,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":44850,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":24699,"thread_start_us":146096,"threads_started":1,"wal-append.queue_time_us":240460}
I20260709 13:05:30.303483 17250 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:30.370605 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 392) took 1647 ms. Trace:
I20260709 13:05:30.344380 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.340581 17381 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.348541 17031 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:30.383347 14020 rpcz_store.cc:276] 0709 13:05:28.723573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.561007 (+837434us) service_pool.cc:224] Handling call
0709 13:05:29.565819 (+  4812us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.599750 (+ 33931us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:29.599799 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.654294 (+ 54495us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.854992 (+200698us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:29.855054 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.855155 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:30.370275 (+515120us) raft_consensus.cc:1685] finished
0709 13:05:30.370310 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.370517 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33430,"raft.queue_time_us":81563,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":3441,"thread_start_us":146074,"threads_started":1,"wal-append.queue_time_us":275142}
I20260709 13:05:30.305436 17341 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:30.359510 16983 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:30.359146 17451 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.359545 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:30.396062 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 314305us
I20260709 13:05:30.309733 17452 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:05:30.396211 16983 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.387105 17441 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7.
I20260709 13:05:30.440506 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2439) took 1133 ms. Trace:
I20260709 13:05:30.441015 14018 rpcz_store.cc:276] 0709 13:05:29.306927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.928977 (+622050us) service_pool.cc:224] Handling call
0709 13:05:29.929282 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.929626 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:29.929659 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.946729 (+ 17070us) log.cc:844] Serialized 26 byte log entry
0709 13:05:30.040764 (+ 94035us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:30.040827 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.040941 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:30.440211 (+399270us) raft_consensus.cc:1685] finished
0709 13:05:30.440242 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.440426 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1700,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":19145,"thread_start_us":55709,"threads_started":1,"wal-append.queue_time_us":197216}
W20260709 13:05:30.446374 17396 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.705s	user 0.006s	sys 0.005s
I20260709 13:05:30.447129 17396 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:30.426342 17400 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:05:30.452555 17400 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:30.453248 17400 consensus_queue.cc:1243] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:30.425917 14552 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:30.457050 17440 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.458310 17454 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.426385 17084 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:05:30.459966 14019 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:30.461783 13869 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 5: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:30.467747 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.459030 17291 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.468655 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:30.398658 17250 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.433935 16983 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:30.480909 13871 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:30.485656 13870 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:30.486505 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:30.419775 17451 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.467157 17084 consensus_queue.cc:1243] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:30.504609 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2429) took 1987 ms. Trace:
I20260709 13:05:30.474368 17454 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:30.514581 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:30.505167 14016 rpcz_store.cc:276] 0709 13:05:28.517403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.241333 (+723930us) service_pool.cc:224] Handling call
0709 13:05:29.250759 (+  9426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.259926 (+  9167us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:29.274862 (+ 14936us) log.cc:844] Serialized 14 byte log entry
0709 13:05:29.553173 (+278311us) spinlock_profiling.cc:243] Waited 6.15 ms on lock 0x7b18002d2260. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c0348a40 00007fb2c03434e4 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287 00007fb2c344e208
0709 13:05:29.560716 (+  7543us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:29.560774 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:29.560832 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.877173 (+316341us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b48002115d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:05:30.504337 (+627164us) spinlock_profiling.cc:243] Waited 8.92 ms on lock 0x7b5400411c54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:30.504407 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.504548 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25677,"prepare.queue_time_us":16102,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":4302,"raft.queue_time_us":253718,"raft.run_cpu_time_us":8930,"raft.run_wall_time_us":303952,"spinlock_wait_cycles":47790080,"thread_start_us":35408,"threads_started":1,"wal-append.queue_time_us":162755,"wal-append.run_cpu_time_us":1444,"wal-append.run_wall_time_us":1045921}
I20260709 13:05:30.515659 13873 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:30.474524 17496 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:30.439759 17369 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.462663 17328 consensus_queue.cc:1059] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:05:30.502476 17441 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.466506 17458 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.524777 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:30.507340 17480 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:05:30.510743 17341 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.529400 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:30.516765 17450 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.517230 17396 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:05:30.532810 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 91415.5us
I20260709 13:05:30.517416 17454 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:30.526908 14020 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:30.535210 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:30.547933 14017 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:30.536448 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:30.533890 14552 raft_consensus.cc:2410] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:30.528357 17031 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:30.550185 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:30.551126 17458 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:30.552778 17250 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.544442 17052 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.555348 17451 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:30.556247 17480 consensus_queue.cc:1243] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:30.579959 17362 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:30.580374 17362 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:30.566422 17031 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:30.581869 17031 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:30.556416 17396 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.556967 17341 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.533063 17441 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.526069 17496 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.562363 16983 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.565816 17506 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.534150 17440 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.571110 17505 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:30.572077 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 47398.6us
I20260709 13:05:30.581267 17362 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.603354 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:30.573167 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:30.604461 14017 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:30.575062 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:30.564563 17454 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:30.601257 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:30.601716 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:30.606468 14024 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:30.607599 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:30.609454 13941 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:30.591087 17319 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:30.611618 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:30.618949 17509 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:05:30.619925 17509 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:30.620227 17509 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:30.620985 17509 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.621585 17400 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:30.622746 17400 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.623234 17396 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:30.625265 17400 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.625927 17400 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:30.626526 17400 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.627329 17341 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.628930 17400 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.622232 17453 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.629709 17453 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.631292 16983 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.632570 17453 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.614805 17451 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:30.640167 17441 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.617046 17084 consensus_queue.cc:1059] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:05:30.647225 17084 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:30.647632 17084 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:30.648448 17084 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.649123 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:30.649672 17084 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:30.650161 17504 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:05:30.653364 13870 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:30.649796 14021 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:30.658041 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:30.661239 13941 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:30.614632 17480 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:30.628499 17507 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:30.668965 17509 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:30.682876 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:30.683630 14017 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:30.686965 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:30.687855 14021 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:30.688596 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:30.689255 13867 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:30.689515 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:30.691609 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:30.665853 17341 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:30.605432 17506 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.695712 17505 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:30.723091 14017 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
W20260709 13:05:30.726552 14522 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.66:43953 timed out after 1.822s (SENT) [suppressed 22 similar messages]
I20260709 13:05:30.691118 17400 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.688827 17480 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:30.750159 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 54374.5us
I20260709 13:05:30.690923 17453 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:30.705618 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:30.720111 17442 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:30.691391 16983 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
W20260709 13:05:30.768903 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 157339us
I20260709 13:05:30.767948 13867 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:30.705943 17171 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:05:30.766752 17341 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:30.777633 17504 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:30.769927 17319 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:30.757272 17031 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.766209 17453 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.777346 17521 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:30.759507 17400 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:30.803320 17513 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:05:30.803498 17336 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.782516 17348 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
W20260709 13:05:30.807988 17512 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
W20260709 13:05:30.823060 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 96558.7us
I20260709 13:05:30.791543 17341 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.807691 17504 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.831164 17496 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.834401 17031 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:30.821847 17336 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:30.846271 17517 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:05:30.849815 17523 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:05:30.852608 17453 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:30.812716 17480 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.863039 17336 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.854866 17496 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:30.863765 17496 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.816726 17521 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.858424 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:30.871146 13875 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:30.844676 17319 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.871897 17265 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:30.866032 17496 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.889184 17265 raft_consensus.cc:697] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:30.890694 17265 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.881826 17400 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.901366 17504 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.902339 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:30.867880 17521 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.880218 17506 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.882042 17239 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
W20260709 13:05:30.928943 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 158272us
I20260709 13:05:30.930042 13915 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:30.931769 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:30.932014 17341 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:30.925765 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:30.934288 14551 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:30.934244 17521 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:30.935916 17521 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:30.934892 17497 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:30.937850 17480 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:30.938872 17497 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:30.937132 17521 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.941347 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:30.939867 17497 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:30.945170 17497 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:30.946657 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:30.948379 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:30.949997 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:30.950765 17320 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.951351 17320 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.618351 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:30.963330 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:30.964272 13873 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:30.966079 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.967320 13875 raft_consensus.cc:2410] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:30.968685 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.970754 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.974380 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:30.974650 17320 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.974965 13874 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:30.975208 17338 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:30.976037 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:30.978219 17440 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:30.978780 17440 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:30.980989 17440 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:30.984493 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:30.984819 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.985450 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:30.985917 13866 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:30.986158 13942 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:30.988889 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.989531 13866 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:30.989722 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:30.990480 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:30.990875 13868 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:30.993305 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:30.993557 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:30.993932 14016 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:30.996738 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:30.997972 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.999727 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:31.000633 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:31.000537 17052 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:05:31.001956 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.003101 13874 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:31.006002 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.006222 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.006467 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:31.008287 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:30.996716 17506 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.011315 17506 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.014549 17447 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:31.019049 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:31.020891 13941 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:31.021445 14017 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:31.000967 17441 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:31.027913 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:31.029433 13840 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:31.031397 17496 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:31.032051 17496 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:31.032595 17496 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:31.033278 17506 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:31.034327 17406 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:31.034897 17406 raft_consensus.cc:697] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:31.035864 17406 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.038937 17052 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:31.039425 17052 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.039768 17052 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:31.040380 17506 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.041143 17506 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.053592 13989 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:31.001467 17320 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:31.057178 14018 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
W20260709 13:05:31.061861 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.065398 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:31.067876 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:31.068994 13942 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:31.070847 17466 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:31.101742 17466 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.102142 14551 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:31.087038 17401 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:31.063892 17320 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:31.105674 17320 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:31.112588 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.064482 17441 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:31.070716 17341 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:31.122836 13944 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:05:31.123266 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 57991.9us
W20260709 13:05:31.123638 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.124457 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:31.086958 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:31.127385 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:31.142799 13867 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:05:31.142966 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 56103.7us
I20260709 13:05:31.128806 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:31.143996 13874 raft_consensus.cc:2410] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:31.099922 17440 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.121474 17400 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.133533 14016 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:31.156098 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:31.146793 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:31.157248 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:31.158931 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:31.167688 14016 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:31.165426 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:31.174278 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:31.174448 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:31.176519 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:31.178809 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:31.165781 17341 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:31.180605 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:31.181551 13946 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
I20260709 13:05:31.183071 17506 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:31.184995 17441 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.173439 17458 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:31.180758 17341 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:31.122289 17454 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.188196 13989 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:31.199566 17507 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:31.200040 17507 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:31.200438 17507 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:31.202133 17524 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:31.203347 13840 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:31.173743 17344 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:31.186867 17338 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:31.189123 17458 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:31.192488 17504 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:31.213641 14016 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:31.194880 17506 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:31.196448 17497 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:31.166307 17440 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.170789 17400 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.242951 13942 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:31.243680 17440 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:31.244084 17400 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:31.202677 17524 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.244853 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.245280 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:31.246317 17524 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:31.247570 13988 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:31.250165 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.192296 17521 consensus_queue.cc:1059] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:05:31.216159 17458 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:31.255357 17472 raft_consensus.cc:3055] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:31.255744 17472 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:31.256650 17472 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.230753 17250 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.261620 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.261540 17250 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.262437 14016 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:05:31.186579 17447 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:05:31.263397 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:31.264017 14016 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:31.266381 17250 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:31.269178 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.274593 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:31.275504 14017 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:31.276412 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.235024 13873 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:31.278225 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2440) took 1971 ms. Trace:
I20260709 13:05:31.278682 14019 rpcz_store.cc:276] 0709 13:05:29.307174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.937636 (+630462us) service_pool.cc:224] Handling call
0709 13:05:29.938488 (+   852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.466122 (+527634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.277959 (+811837us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b54000ab754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:31.278161 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1198627584}
I20260709 13:05:31.279946 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:31.280652 14019 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:31.241210 17506 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.281445 14022 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:31.282214 17504 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:31.283275 17504 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.284051 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.284690 14024 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:31.250249 17542 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.277707 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.285665 17542 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.286919 14017 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:31.208415 17466 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:31.222386 17341 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.285601 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.251505 17440 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:31.252959 17052 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.302389 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:31.355880 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:31.306630 16983 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.322726 17466 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:31.474318 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1709) took 1316 ms. Trace:
I20260709 13:05:31.260142 17362 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:31.380146 17319 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:31.442420 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:31.474974 14023 rpcz_store.cc:276] 0709 13:05:30.158220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.282544 (+124324us) service_pool.cc:224] Handling call
0709 13:05:30.282925 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:30.512325 (+229400us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:30.512707 (+   382us) log.cc:844] Serialized 14 byte log entry
0709 13:05:30.592576 (+ 79869us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:30.593105 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:05:30.593267 (+   162us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:30.593602 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 13:05:30.593779 (+   177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.593863 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:31.105723 (+511860us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b18003cfca0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:31.267511 (+161788us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b18003cfca0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1
0709 13:05:31.268904 (+  1393us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b18003cfca0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:31.474094 (+205190us) raft_consensus.cc:1685] finished
0709 13:05:31.474119 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:31.474244 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19586,"prepare.queue_time_us":2883,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":81,"raft.queue_time_us":41335,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":40672,"spinlock_wait_cycles":24801664,"thread_start_us":284647,"threads_started":2,"wal-append.queue_time_us":80280}
I20260709 13:05:31.474352 16983 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.306953 17400 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:31.491887 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.520584 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:31.430138 17540 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.230s	user 0.005s	sys 0.000s
I20260709 13:05:31.458554 17328 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:31.458175 17531 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:31.508577 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:31.573374 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.573720 14561 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:31.547693 14023 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:31.575832 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:31.351969 17541 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:31.350987 17395 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:31.360507 17440 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:31.281345 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.431856 17447 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.441929 17442 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:31.380810 17524 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:05:31.568995 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 321481us
I20260709 13:05:31.458514 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.469074 17250 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:31.600512 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:31.605664 14016 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:31.606379 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:49644 (request call id 2438) took 2268 ms. Trace:
I20260709 13:05:31.613116 14022 rpcz_store.cc:276] 0709 13:05:29.306623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.919250 (+612627us) service_pool.cc:224] Handling call
0709 13:05:31.281326 (+1362076us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b4c00388a40. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c16ffa20 00007fb2c1710a29 00007fb2c171113f 00007fb2c16d52ab 00007fb2c1702f3c 00007fb2c17033fe 00007fb2c199088b 00007fb2c338f495 00007fb2c33e030f 00007fb2c342aa55 00007fb2c83facd0
0709 13:05:31.571007 (+289681us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50772224}
I20260709 13:05:31.625669 14017 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:05:31.628955 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 425663us
I20260709 13:05:31.524294 17319 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.491588 17466 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.571043 17400 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.618119 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:31.616364 16771 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:31.643756 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 584) took 2558 ms. Trace:
I20260709 13:05:31.628222 17395 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.587831 17328 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.635571 14022 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:31.647265 13950 rpcz_store.cc:276] 0709 13:05:29.080862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.090899 (+ 10037us) service_pool.cc:224] Handling call
0709 13:05:29.091187 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.652930 (+561743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.639156 (+1986226us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400087354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:31.639357 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148190080}
I20260709 13:05:31.599020 17531 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.655225 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:31.617715 17524 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:31.659456 17543 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:05:31.664367 17458 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.665066 17338 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:31.622395 17250 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.665592 17338 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.667790 17250 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:31.668010 17338 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:31.651629 14551 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:31.674933 17542 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:31.675645 17440 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:31.677583 17481 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:05:31.652627 17506 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:31.679392 17319 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:31.680511 17524 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:31.617511 17442 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.689318 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:31.691515 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:31.714608 17458 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:31.715400 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:31.716279 13867 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:31.727731 17400 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:31.614269 17447 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.691825 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:31.745880 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:31.747416 13867 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:31.745208 13868 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
W20260709 13:05:31.739321 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 83433us
I20260709 13:05:31.758325 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:31.646378 17052 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:31.761897 14551 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:31.765518 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:31.767475 14022 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:31.695338 17546 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:31.778142 13987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.005s	sys 0.015s
W20260709 13:05:31.779125 13987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.005s	sys 0.015s
I20260709 13:05:31.780095 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:31.782032 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:31.606199 17541 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.751673 17481 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:31.765062 17496 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.732s	user 0.013s	sys 0.006s
I20260709 13:05:31.657303 17395 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:31.824542 13869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2404) took 1516 ms (client timeout 1719 ms). Trace:
W20260709 13:05:31.825161 13869 rpcz_store.cc:269] 0709 13:05:30.308195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.343993 (+ 35798us) service_pool.cc:224] Handling call
0709 13:05:31.824257 (+1480264us) spinlock_profiling.cc:243] Waited 76.9 ms on lock 0x7b54004510d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:31.824461 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176966272}
I20260709 13:05:31.827147 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2399) took 1522 ms. Trace:
I20260709 13:05:31.653066 16771 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.827629 13871 rpcz_store.cc:276] 0709 13:05:30.304614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.304761 (+   147us) service_pool.cc:224] Handling call
0709 13:05:31.826950 (+1522189us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54001aac54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:31.827074 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366639488}
I20260709 13:05:31.829152 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:05:31.811723 17540 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:05:31.791591 17319 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:31.794871 17506 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:31.795233 17458 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:31.791796 17362 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.872277 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:31.874233 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:31.872918 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 43868.4us
I20260709 13:05:31.881650 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:31.862084 17396 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:31.795215 17546 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.883036 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:31.876682 16994 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.022s	sys 0.025s
I20260709 13:05:31.864576 17319 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:31.904439 13870 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:31.868475 17506 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.829612 17496 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.830611 17481 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.909327 13988 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:31.909371 13867 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:31.878798 17109 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:05:31.905742 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.887082 17369 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:31.889204 17542 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:05:31.902730 16994 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.022s	sys 0.026s
W20260709 13:05:31.930394 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.935014 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.886855 17441 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.937574 14016 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:31.937790 17441 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:31.938912 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:31.940591 14016 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:31.940757 17441 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:31.942081 14016 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:31.879683 17362 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:31.942562 13869 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:31.910872 17319 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:31.915769 17506 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:31.929049 17401 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.812s	user 0.022s	sys 0.006s
I20260709 13:05:31.951181 14016 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:31.886829 17538 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:05:31.913343 16983 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:31.929582 17250 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:31.979478 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.960668 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:31.961345 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:31.936779 17542 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.976429 14016 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:31.932901 17458 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:31.993577 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 118256us
I20260709 13:05:31.995512 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:31.961400 17466 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:31.978744 17320 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.873s	user 0.006s	sys 0.020s
I20260709 13:05:31.975629 17400 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:31.977981 17401 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:31.990710 17542 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:32.017565 17549 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
W20260709 13:05:32.020781 17552 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:05:32.021272 14019 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:31.992036 17458 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:31.989562 17441 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:32.017055 17344 consensus_meta.cc:287] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.806s	user 0.017s	sys 0.004s
W20260709 13:05:32.050433 17507 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.824s	user 0.017s	sys 0.015s
I20260709 13:05:32.010203 16994 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.995697 17557 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:32.057327 13950 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:05:32.007534 17447 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:32.013958 17341 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:32.155519 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 169443us
I20260709 13:05:31.992548 17250 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.160743 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:32.025013 17400 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.017750 17320 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.239729 13866 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:32.164172 13874 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:32.455153 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:32.183944 13947 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:32.035254 17396 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.163697 13950 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:32.150889 17547 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:32.159019 17109 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:32.449395 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:32.189664 14024 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:32.464552 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:32.055313 17458 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:32.461642 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:32.495929 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:32.050979 17326 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:32.241485 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:32.522702 13943 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 14: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:32.163525 13868 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:32.523916 13866 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:32.506850 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 495842us
I20260709 13:05:32.524547 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:32.523283 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 445651us
I20260709 13:05:32.525856 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:32.525645 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.526643 13868 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:05:32.306141 16771 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:32.307729 17552 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:05:32.157295 17531 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.495236 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:32.456897 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.520217 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:32.531414 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:32.474493 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 516) took 2418 ms. Trace:
W20260709 13:05:32.264688 17550 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.397s	user 0.000s	sys 0.001s
W20260709 13:05:32.335186 17565 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.272s	user 0.000s	sys 0.000s
I20260709 13:05:32.541148 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:32.541177 14556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 698) took 2443 ms (client timeout 1928 ms). Trace:
I20260709 13:05:32.446763 17362 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.455665 17452 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.474442 17440 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.551873 13872 rpcz_store.cc:276] 0709 13:05:30.029647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.030069 (+   422us) service_pool.cc:224] Handling call
0709 13:05:30.030474 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.108254 (+ 77780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.448079 (+2339825us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b54000daa54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:32.448275 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113942784}
I20260709 13:05:32.455994 17546 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.550680 13947 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:32.554244 13875 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:32.570775 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.571514 13875 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11. Ops: [6.4-10.6]
I20260709 13:05:32.573148 13875 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:32.502211 17441 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.557130 13950 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 4.
I20260709 13:05:32.422075 17395 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:32.578027 13947 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:32.509500 17447 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.456637 17369 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.475242 17524 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:32.150349 17507 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:32.559229 14556 rpcz_store.cc:269] 0709 13:05:30.086529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.086762 (+   233us) service_pool.cc:224] Handling call
0709 13:05:32.529222 (+2442460us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54007f1554. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:32.529381 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093119744}
I20260709 13:05:32.586854 13868 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: [2.2-5.4]
I20260709 13:05:32.558358 17541 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.178414 17442 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.239595 17344 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:32.565239 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:32.529341 17052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.000s	sys 0.019s
I20260709 13:05:32.286317 17341 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.577731 13875 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:32.553396 17401 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.579831 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.264539 17557 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.244792 16994 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.586556 13874 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 8 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: [3.3-7.5]
W20260709 13:05:32.587054 13916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.025s	sys 0.019s
I20260709 13:05:32.588979 13868 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:32.563447 13866 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:32.495299 17481 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.538141 17326 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.510437 17396 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.578265 17452 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.587493 17447 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.583382 17546 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.707985 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 411) took 1681 ms. Trace:
I20260709 13:05:32.714429 17546 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.714381 14017 rpcz_store.cc:276] 0709 13:05:31.026878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.611049 (+584171us) service_pool.cc:224] Handling call
0709 13:05:31.611328 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.629312 (+ 17984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.707775 (+1078463us) spinlock_profiling.cc:243] Waited 7.24 ms on lock 0x7b5400411254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:32.707925 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16646400}
I20260709 13:05:32.716324 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1726) took 1520 ms. Trace:
I20260709 13:05:32.716816 14019 rpcz_store.cc:276] 0709 13:05:31.196419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.027350 (+830931us) service_pool.cc:224] Handling call
0709 13:05:32.134421 (+107071us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:32.142487 (+  8066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:32.142541 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:32.156969 (+ 14428us) log.cc:844] Serialized 26 byte log entry
0709 13:05:32.585992 (+429023us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:32.586024 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.586132 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:32.716065 (+129933us) raft_consensus.cc:1685] finished
0709 13:05:32.716096 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.716251 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13994,"raft.queue_time_us":8429,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":94095,"thread_start_us":300325,"threads_started":1,"wal-append.queue_time_us":556838}
W20260709 13:05:32.718328 17551 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.000s	sys 0.004s
I20260709 13:05:32.708320 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:32.710076 13916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.032s	sys 0.023s
I20260709 13:05:32.597420 13947 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:32.633805 14016 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:32.710592 13838 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:32.495355 17547 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.584458 17441 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:32.677891 14522 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Timed out: RequestConsensusVote RPC to 127.12.109.66:43953 timed out after 1.822s (SENT)
W20260709 13:05:32.738174 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 60347us
I20260709 13:05:32.720427 17401 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.740453 17496 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:32.741005 17506 consensus_meta.cc:287] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.758s	user 0.006s	sys 0.003s
I20260709 13:05:32.741840 17241 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:32.743242 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.743650 17452 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.743767 17241 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:05:32.744395 13942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2394) took 1570 ms (client timeout 1579 ms). Trace:
I20260709 13:05:32.744278 17452 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:05:32.744817 13942 rpcz_store.cc:269] 0709 13:05:31.173685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.173904 (+   219us) service_pool.cc:224] Handling call
0709 13:05:32.744203 (+1570299us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b540004c254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:32.744336 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155894528}
I20260709 13:05:32.730641 17507 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:32.724800 13950 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
W20260709 13:05:32.712491 17052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.000s	sys 0.030s
I20260709 13:05:32.590920 17109 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.726971 17574 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:32.730576 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.733426 17320 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:32.734126 17566 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:05:32.572325 17440 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.734459 17400 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.748365 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.741454 17506 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.749986 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:32.747685 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:32.676959 17442 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:32.722983 17567 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:05:32.763741 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.776194 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.777032 13943 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:32.713342 17341 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:32.597747 17541 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:32.771288 13950 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:32.778934 17541 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.767599 13942 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:32.589588 17369 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.740456 17401 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:32.781999 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:32.783725 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.785399 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.756877 17507 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:32.591571 17524 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.794097 13950 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:32.762534 17395 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:32.772279 17320 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.786979 17267 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:32.774760 17171 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:32.806321 17447 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:32.795481 17507 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.807637 17547 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:32.808465 17507 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:32.796438 17395 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.811110 17395 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:32.811715 17546 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.815208 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:32.815635 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:32.818946 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.819012 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.821656 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:32.822757 13838 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:32.825260 17539 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:32.823834 13838 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:32.829553 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.829527 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.832304 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.834532 13942 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:32.834610 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.835507 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:32.788033 17442 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.839766 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:32.842515 13942 raft_consensus.cc:2376] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 4. Re-sending same reply.
I20260709 13:05:32.843683 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.844403 13942 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:32.845360 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.805919 17267 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:32.848044 14522 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:32.849782 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 534) took 1001 ms. Trace:
I20260709 13:05:32.851336 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:32.830102 17319 consensus_meta.cc:287] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.853s	user 0.009s	sys 0.004s
I20260709 13:05:32.837586 17052 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:32.850750 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:32.846024 13942 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:32.848939 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.852638 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:32.811625 17320 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.851460 13871 rpcz_store.cc:276] 0709 13:05:31.848526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.848754 (+   228us) service_pool.cc:224] Handling call
0709 13:05:31.849241 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:31.860344 (+ 11103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:31.860394 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:31.869254 (+  8860us) log.cc:844] Serialized 26 byte log entry
0709 13:05:31.985249 (+115995us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:31.985293 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.985370 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:32.849544 (+864174us) raft_consensus.cc:1685] finished
0709 13:05:32.849565 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.849701 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10519,"raft.queue_time_us":12203,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":3674,"thread_start_us":81897,"threads_started":1,"wal-append.queue_time_us":682011}
I20260709 13:05:32.834378 17545 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:32.828873 17546 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:32.834649 17577 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:32.859577 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:32.861639 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:32.851068 17267 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:32.862303 13943 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:32.826166 17171 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:32.857663 17560 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.005s	sys 0.010s
I20260709 13:05:32.853137 17576 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.850752 17507 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:32.872944 17441 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.871640 17524 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:32.855983 17486 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:32.883514 17486 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.883924 17486 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:32.884553 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.885221 13947 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:32.886572 17396 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:32.887319 17353 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:32.887722 17353 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:32.888132 17442 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.888563 17353 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.891067 17353 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:32.866322 17542 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.815s	user 0.012s	sys 0.000s
I20260709 13:05:32.860035 17546 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:32.881703 17507 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.882457 17576 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.883052 17171 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.854329 17052 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.893186 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2462) took 1847 ms. Trace:
I20260709 13:05:32.943615 14016 rpcz_store.cc:276] 0709 13:05:31.045927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.991331 (+945404us) service_pool.cc:224] Handling call
0709 13:05:31.991727 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.742729 (+751002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.892918 (+150189us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54003bb0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:32.893125 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1475887872}
I20260709 13:05:32.893501 17320 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:32.861461 17401 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:32.878849 17326 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:32.895414 17561 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.556s	user 0.003s	sys 0.000s
I20260709 13:05:32.863124 17577 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.895910 13943 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:32.896498 16994 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:32.897514 17557 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:32.897710 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:32.901002 17486 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.856822 17319 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.907474 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:32.959028 13867 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:32.912181 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:32.913250 14018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1721) took 1918 ms (client timeout 2196 ms). Trace:
I20260709 13:05:32.919525 17458 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.960083 13949 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 4.
W20260709 13:05:32.960014 14018 rpcz_store.cc:269] 0709 13:05:30.994761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.994901 (+   140us) service_pool.cc:224] Handling call
0709 13:05:32.913005 (+1918104us) spinlock_profiling.cc:243] Waited 50.5 ms on lock 0x7b540023c154. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:32.913170 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116067712}
I20260709 13:05:32.961218 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.961460 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:32.961848 13949 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:32.962117 17458 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:32.962842 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:32.963603 13949 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:32.958612 17109 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:32.964411 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.965224 13949 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:32.965083 17109 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:32.860249 17547 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:32.966260 17547 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:32.966703 17547 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:32.968746 17454 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:32.968890 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:32.929173 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.971107 17109 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:32.876453 17560 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.005s	sys 0.010s
I20260709 13:05:32.945526 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:32.972105 14016 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:32.954669 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:32.941483 17546 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:32.946970 17052 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:32.965794 17472 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:32.962844 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:32.976908 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:32.977490 14018 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:05:32.948065 17401 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:32.974107 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.978034 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.979048 13943 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:05:32.980705 13838 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:32.943709 17542 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:32.959585 16994 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.956413 17442 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.988358 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:32.989015 13943 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:32.991003 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:32.994014 14019 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6. Ops: []
W20260709 13:05:32.995699 17548 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.103s	user 0.008s	sys 0.002s
I20260709 13:05:32.998325 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 529) took 1531 ms. Trace:
I20260709 13:05:32.986581 17585 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:32.999037 17585 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:32.999678 17585 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:32.998770 13873 rpcz_store.cc:276] 0709 13:05:31.467096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.467213 (+   117us) service_pool.cc:224] Handling call
0709 13:05:31.467612 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:31.468006 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:31.468046 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:31.468670 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 13:05:31.945497 (+476827us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:31.945557 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.945653 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:32.894761 (+949108us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:32.989969 (+ 95208us) raft_consensus.cc:1685] finished
0709 13:05:32.989994 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.998242 (+  8248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5596,"raft.queue_time_us":151880,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":17631,"spinlock_wait_cycles":478675328,"thread_start_us":421058,"threads_started":1,"wal-append.queue_time_us":666222}
I20260709 13:05:32.990355 16994 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.002202 17576 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:33.002857 17576 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.002734 17442 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.003407 17576 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.004961 17442 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.978490 17401 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:33.007546 17577 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:33.009708 17319 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:33.011415 17507 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:33.011950 17542 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:33.013109 17486 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:32.974758 17544 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:33.014591 17544 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:33.016153 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:33.016753 14018 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:33.016834 17544 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:33.019574 17328 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:33.005546 17576 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:33.024215 17441 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:33.026646 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.027343 13943 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:33.028393 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 596) took 1001 ms. Trace:
I20260709 13:05:33.028756 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:33.029403 13943 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:33.030104 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:33.003733 16994 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:33.031436 14016 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:05:33.032292 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:33.033399 14016 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:33.028774 13946 rpcz_store.cc:276] 0709 13:05:32.026915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.027492 (+   577us) service_pool.cc:224] Handling call
0709 13:05:32.746142 (+718650us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:32.746581 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:32.746615 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:32.747296 (+   681us) log.cc:844] Serialized 52 byte log entry
0709 13:05:32.940952 (+193656us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:32.941024 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.941134 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:33.025904 (+ 84770us) raft_consensus.cc:1685] finished
0709 13:05:33.025926 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.028074 (+  2148us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b540004c250. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:33.028335 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12265,"raft.queue_time_us":3506,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":4012,"spinlock_wait_cycles":671591168,"thread_start_us":193007,"threads_started":1,"wal-append.queue_time_us":212617}
I20260709 13:05:33.034698 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:33.035943 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.036701 13949 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:33.037768 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.037962 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:33.038990 14018 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:05:33.048427 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.049499 13946 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:33.051240 14016 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:33.051816 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2465) took 1854 ms. Trace:
I20260709 13:05:33.052088 14016 rpcz_store.cc:276] 0709 13:05:31.197264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.050782 (+1853518us) service_pool.cc:224] Handling call
0709 13:05:33.051099 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.051626 (+   527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.051770 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:33.054405 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.055089 13949 raft_consensus.cc:2410] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:05:33.055639 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.053892 14018 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:33.056820 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2466) took 1848 ms. Trace:
I20260709 13:05:33.057174 14018 rpcz_store.cc:276] 0709 13:05:31.208330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.052828 (+1844498us) service_pool.cc:224] Handling call
0709 13:05:33.053688 (+   860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.056631 (+  2943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.056775 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:33.060940 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:33.061146 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.062296 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.063248 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.064630 14016 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:33.064774 13949 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:05:33.065254 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2469) took 1199 ms. Trace:
I20260709 13:05:33.065600 14016 rpcz_store.cc:276] 0709 13:05:31.866159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.064036 (+1197877us) service_pool.cc:224] Handling call
0709 13:05:33.064318 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.065016 (+   698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.065201 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:33.065790 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.068621 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.070466 13946 raft_consensus.cc:2410] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:33.072196 13943 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:33.075460 14522 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:33.075688 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.076500 13946 raft_consensus.cc:2410] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:33.078423 14018 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:33.050196 17441 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:33.083794 13949 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:05:33.084941 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.086812 17441 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:33.087208 14024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 409) took 2079 ms (client timeout 2427 ms). Trace:
W20260709 13:05:33.087723 14024 rpcz_store.cc:269] 0709 13:05:31.007381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.302081 (+294700us) service_pool.cc:224] Handling call
0709 13:05:33.087023 (+1784942us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b54003255d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:33.087142 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1742551296}
I20260709 13:05:33.088833 17381 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:33.093353 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.087304 17441 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:33.094028 13947 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:33.097654 13947 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:33.079572 17171 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:33.079424 17544 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:33.109244 13949 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:33.111111 17545 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:33.104800 17542 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:05:33.114876 17454 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:33.115422 17454 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.117673 17454 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:33.118824 17454 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:33.119637 17454 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:33.098896 17559 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.007s	sys 0.038s
W20260709 13:05:33.129495 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.095263 17267 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:33.134827 17546 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:33.113168 17504 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:33.111150 13949 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:05:33.150594 13949 pending_rounds.cc:85] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 4
I20260709 13:05:33.150820 17551 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:33.155395 17551 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.132460 17171 raft_consensus.cc:697] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:33.158671 13949 pending_rounds.cc:107] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 6.5
W20260709 13:05:33.137636 17559 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.007s	sys 0.038s
I20260709 13:05:33.160612 13949 raft_consensus.cc:2889] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:33.167630 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.179343 17171 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:33.255327 14017 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:33.132717 17544 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:33.230647 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 597) took 1203 ms. Trace:
W20260709 13:05:33.217721 13941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2385) took 2230 ms (client timeout 2174 ms). Trace:
I20260709 13:05:33.255041 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2446) took 2222 ms. Trace:
I20260709 13:05:33.278621 14022 rpcz_store.cc:276] 0709 13:05:31.034745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.655277 (+620532us) service_pool.cc:224] Handling call
0709 13:05:31.655633 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.767961 (+112328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.183713 (+1415752us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540029a4d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:33.254958 (+ 71245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256722176}
I20260709 13:05:33.185295 14021 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:33.276845 13944 rpcz_store.cc:276] 0709 13:05:32.027211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.027595 (+   384us) service_pool.cc:224] Handling call
0709 13:05:32.156430 (+128835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.736712 (+580282us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:32.862140 (+125428us) log.cc:844] Serialized 14 byte log entry
0709 13:05:32.960274 (+ 98134us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:32.960338 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:32.960390 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.230305 (+269915us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b5400091d54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:33.230365 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.230539 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3475,"prepare.queue_time_us":240135,"raft.queue_time_us":92,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":1462,"spinlock_wait_cycles":1342779264,"thread_start_us":218697,"threads_started":2,"wal-append.queue_time_us":161344}
W20260709 13:05:33.277842 13941 rpcz_store.cc:269] 0709 13:05:30.983308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.984205 (+   897us) service_pool.cc:224] Handling call
0709 13:05:33.213382 (+2229177us) spinlock_profiling.cc:243] Waited 35.7 ms on lock 0x7b5400054954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:33.213564 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82041728}
I20260709 13:05:33.316094 13941 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:33.316361 14021 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:33.314658 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.317078 13941 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:33.319082 14022 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:33.319144 13941 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.320392 13941 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:33.257298 17546 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:33.183009 17563 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.912s	user 0.002s	sys 0.001s
I20260709 13:05:33.227092 17454 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:33.180974 17541 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:33.257836 17551 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:33.148229 17504 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:33.274941 17442 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.348032 13941 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:05:33.358441 13941 pending_rounds.cc:85] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 4
I20260709 13:05:33.358855 13941 pending_rounds.cc:107] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:05:33.359185 13941 raft_consensus.cc:2889] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:33.262037 17328 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.186834 17369 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:33.159660 17585 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:33.362337 13941 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:05:33.346537 17400 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.335000 17560 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:05:33.293726 17441 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.363061 17328 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.371060 17504 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:33.372460 17328 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:33.361352 17442 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:33.375069 17544 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:33.376312 17442 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:33.377496 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:33.378192 14558 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:05:33.378536 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:33.379535 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:33.379755 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:33.380177 13866 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:33.380319 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:33.380371 14024 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:33.381142 13871 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:33.381227 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:33.381690 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:33.343761 17547 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:33.381753 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:33.383299 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:33.430757 17536 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 2.230s	user 0.010s	sys 0.011s
W20260709 13:05:33.529287 17401 consensus_meta.cc:287] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.523s	user 0.007s	sys 0.006s
I20260709 13:05:33.572535 17401 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:33.566109 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:33.573872 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.520175 17541 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:33.574743 17541 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:33.426914 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:33.576081 17547 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:33.403766 17267 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:33.573833 14016 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:33.532899 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2464) took 2335 ms. Trace:
I20260709 13:05:33.577706 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:33.416107 17544 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.163s
I20260709 13:05:33.578595 13871 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:33.577870 14019 rpcz_store.cc:276] 0709 13:05:31.197014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.718875 (+1521861us) service_pool.cc:224] Handling call
0709 13:05:32.719268 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.994444 (+275176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.532590 (+538146us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54002981d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:33.532828 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631368448}
W20260709 13:05:33.577730 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.419334 17052 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:33.580209 13915 connection.cc:256] RPC call timeout handler was delayed by -1.3435564440000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 13:05:33.580228 17401 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:33.496418 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.383488 13914 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:33.384053 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1725) took 2379 ms. Trace:
I20260709 13:05:33.382591 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:33.587440 14024 raft_consensus.cc:2410] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:05:33.413750 13941 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:05:33.445129 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 402) took 3307 ms. Trace:
I20260709 13:05:33.590689 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.591542 13941 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:33.590277 14015 rpcz_store.cc:276] 0709 13:05:30.135176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.141208 (+  6032us) service_pool.cc:224] Handling call
0709 13:05:30.157543 (+ 16335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:30.270718 (+113175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:30.270769 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:30.598283 (+327514us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b1800185620. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a
0709 13:05:31.825181 (+1226898us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b2c000e8788. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:05:31.825776 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:05:31.920399 (+ 94623us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:31.920461 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.909693 (+989232us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b5400020dd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:32.909770 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:33.182293 (+272523us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:33.442075 (+259782us) raft_consensus.cc:1685] finished
0709 13:05:33.442109 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.442311 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13782,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":5899,"spinlock_wait_cycles":971654784,"thread_start_us":94237,"threads_started":1,"wal-append.queue_time_us":200967}
I20260709 13:05:33.500231 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:33.592511 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.459093 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1775) took 1037 ms. Trace:
I20260709 13:05:33.593184 14021 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:33.593413 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:33.594276 14015 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:33.594599 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:33.587190 14023 rpcz_store.cc:276] 0709 13:05:31.004630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.598326 (+593696us) service_pool.cc:224] Handling call
0709 13:05:31.598691 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.649217 (+ 50526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:31.649250 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:31.649295 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:31.649334 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.383732 (+1734398us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b540023b254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:33.383806 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.383981 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115656960}
I20260709 13:05:33.595497 14021 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:33.596081 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:33.596771 14019 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:33.597275 13872 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:33.579100 17544 consensus_queue.cc:1243] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:33.576716 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:33.598491 14019 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:33.598922 14021 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:33.602155 14018 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:33.390172 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:33.600788 13915 connection.cc:256] RPC call timeout handler was delayed by -1.2040564440000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17300000000000001s for a response
W20260709 13:05:33.603240 13915 connection.cc:256] RPC call timeout handler was delayed by -1.2040564440000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17300000000000001s for a response
W20260709 13:05:33.603582 13915 connection.cc:256] RPC call timeout handler was delayed by -1.1167564439999751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
W20260709 13:05:33.603878 13915 connection.cc:256] RPC call timeout handler was delayed by -1.1167564439999751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
W20260709 13:05:33.604164 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0852564440000378s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
W20260709 13:05:33.604425 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0258564440000555s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
W20260709 13:05:33.604679 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0258564440000555s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
I20260709 13:05:33.604736 14023 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:33.605368 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:33.594914 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:33.578169 13875 raft_consensus.cc:2410] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:33.607813 13867 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:33.607970 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:33.608554 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 225186us
I20260709 13:05:33.608858 13872 raft_consensus.cc:2410] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:33.609586 13946 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:05:33.419538 17535 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 2.220s	user 0.011s	sys 0.005s
I20260709 13:05:33.427742 17319 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.593402 13943 rpcz_store.cc:276] 0709 13:05:32.422984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.082553 (+659569us) service_pool.cc:224] Handling call
0709 13:05:33.082819 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.083017 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:33.157824 (+ 74807us) log.cc:844] Serialized 14 byte log entry
0709 13:05:33.458627 (+300803us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:33.458688 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:33.458747 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.458838 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.459005 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25727,"prepare.queue_time_us":279151,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":33437,"raft.queue_time_us":126806,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":16046,"thread_start_us":356994,"threads_started":2,"wal-append.queue_time_us":412308}
I20260709 13:05:33.617295 13941 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:33.623732 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:33.413748 17560 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:33.585618 17585 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:33.610601 17328 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:33.627719 17599 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.245s	user 0.001s	sys 0.001s
I20260709 13:05:33.615886 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.611943 17109 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.608124 17521 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.582023 17052 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:33.612982 16994 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:33.642113 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:33.413131 17400 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:33.645162 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.630889 17472 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:05:33.621639 17442 consensus_queue.cc:1059] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:05:33.639838 17328 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.640319 17109 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.662904 17328 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:33.647055 17521 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.633555 17319 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:33.637039 17560 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:33.665426 16994 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:33.688107 16994 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:33.688704 17319 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:33.689087 16994 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.691149 14022 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:33.692134 17250 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:33.694314 17109 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:33.694679 17506 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:33.679962 17442 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:33.695637 17442 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:33.696873 17605 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:05:33.696653 17442 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.664523 17441 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:33.683533 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1715) took 3134 ms. Trace:
W20260709 13:05:33.681803 17562 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.343s	user 0.002s	sys 0.002s
I20260709 13:05:33.702888 14020 rpcz_store.cc:276] 0709 13:05:30.549025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.695181 (+146156us) service_pool.cc:224] Handling call
0709 13:05:30.695541 (+   360us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:30.695972 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:30.696010 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:30.704900 (+  8890us) log.cc:844] Serialized 52 byte log entry
0709 13:05:30.980659 (+275759us) spinlock_profiling.cc:243] Waited 910 us on lock 0x7b180019a200. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:05:31.073206 (+ 92547us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:31.099852 (+ 26646us) log.cc:844] Serialized 14 byte log entry
0709 13:05:31.100443 (+   591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.100573 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:33.364500 (+2263927us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:33.683210 (+318710us) raft_consensus.cc:1685] finished
0709 13:05:33.683234 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.683449 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25978,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":4539,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":12986,"spinlock_wait_cycles":15789824,"thread_start_us":82705,"threads_started":1,"wal-append.queue_time_us":467260}
I20260709 13:05:33.687651 17521 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:33.699549 17401 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:33.706146 17401 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:33.707049 17606 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.137s	user 0.003s	sys 0.000s
I20260709 13:05:33.708190 17560 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:33.708629 17607 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:05:33.708848 17401 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:33.700107 17506 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:33.714073 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.717123 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:33.718497 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:33.719416 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:33.717285 17400 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:33.735438 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:33.742146 17442 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:33.711979 17506 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.799167 17506 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:33.797641 17440 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:33.801012 17611 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:05:33.799008 17604 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.231s	user 0.003s	sys 0.000s
I20260709 13:05:33.848634 13762 maintenance_manager.cc:419] P 91c63734839347c4aee340a59c8e3221: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033341
W20260709 13:05:33.867125 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:33.866922 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 253019us
I20260709 13:05:33.871542 13943 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:33.874421 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 613) took 1049 ms. Trace:
I20260709 13:05:33.875134 13949 rpcz_store.cc:276] 0709 13:05:32.824760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.451512 (+626752us) service_pool.cc:224] Handling call
0709 13:05:33.451980 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:33.584333 (+132353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:33.584378 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:33.610675 (+ 26297us) log.cc:844] Serialized 26 byte log entry
0709 13:05:33.679171 (+ 68496us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:33.679236 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.679367 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:33.701065 (+ 21698us) spinlock_profiling.cc:243] Waited 7.6 ms on lock 0x7b480012c3d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:05:33.873511 (+172446us) raft_consensus.cc:1685] finished
0709 13:05:33.873560 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.874144 (+   584us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131843,"raft.queue_time_us":132267,"raft.run_cpu_time_us":2198,"raft.run_wall_time_us":104109,"spinlock_wait_cycles":17469184,"thread_start_us":40561,"threads_started":1,"wal-append.queue_time_us":53156}
I20260709 13:05:33.877414 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:33.900499 13941 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:33.900980 13941 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:33.917250 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1727) took 2720 ms. Trace:
I20260709 13:05:33.930692 13941 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:33.933478 13941 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:33.937862 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:33.938818 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:33.942202 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 598) took 1914 ms. Trace:
I20260709 13:05:33.942802 13945 rpcz_store.cc:276] 0709 13:05:32.027501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.027614 (+   113us) service_pool.cc:224] Handling call
0709 13:05:33.890167 (+1862553us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b5800202e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030eeee 0000000000375939 00007fb2c83821e0 00007fb2c844b0c4 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:05:33.890759 (+   592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.919646 (+ 28887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:33.919682 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:33.935882 (+ 16200us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:33.941599 (+  5717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.941704 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.941901 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2361472}
I20260709 13:05:33.945129 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:33.918037 14017 rpcz_store.cc:276] 0709 13:05:31.196747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.717823 (+1521076us) service_pool.cc:224] Handling call
0709 13:05:33.254915 (+537092us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.270678 (+ 15763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.916502 (+645824us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b540029a4d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:33.916863 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1234241408}
I20260709 13:05:33.956871 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:33.981364 14556 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:33.941274 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:34.017078 13945 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:34.018848 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.020537 13914 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:05:34.072942 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 52831.6us
W20260709 13:05:34.092742 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.199856 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:34.200379 13941 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:34.201416 17440 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.070407 16994 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:34.202322 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:34.202651 17440 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:34.208678 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:34.209928 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:34.209642 17267 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:34.210915 14552 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:34.212689 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:34.214532 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:34.214514 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:34.201705 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:34.182469 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 312670us
I20260709 13:05:34.246290 17496 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:34.247450 17496 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:34.249154 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:34.214942 17442 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:34.214872 17440 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:34.262827 17547 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.263732 17547 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:34.257872 17610 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:05:34.255232 17442 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.283562 17480 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:34.283836 17506 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.284204 17480 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:34.284660 17506 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:34.281913 17442 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:34.290927 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.292855 14551 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:34.294945 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:34.284716 17480 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:34.301064 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.309485 17496 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:34.309911 13840 connection.cc:256] RPC call timeout handler was delayed by -1.8027510819999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
I20260709 13:05:34.319739 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:34.327091 17616 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.068s	user 0.001s	sys 0.001s
I20260709 13:05:34.328873 17267 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:34.330216 17267 raft_consensus.cc:3055] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:34.330623 17267 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:34.332104 17267 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:34.334560 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.339531 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 90637.4us
I20260709 13:05:34.341934 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:05:34.344069 13840 connection.cc:256] RPC call timeout handler was delayed by -1.4328510819999565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
I20260709 13:05:34.347201 17349 consensus_queue.cc:1059] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:34.348915 17171 raft_consensus.cc:3055] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:34.349443 17171 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:34.351008 17171 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:34.347215 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:34.353463 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 617) took 1262 ms. Trace:
I20260709 13:05:34.353868 13946 rpcz_store.cc:276] 0709 13:05:33.091106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.647450 (+556344us) service_pool.cc:224] Handling call
0709 13:05:33.647903 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:33.657506 (+  9603us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:33.664059 (+  6553us) log.cc:844] Serialized 14 byte log entry
0709 13:05:34.129344 (+465285us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:34.178036 (+ 48692us) log.cc:844] Serialized 26 byte log entry
0709 13:05:34.178355 (+   319us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:34.178411 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.352789 (+174378us) spinlock_profiling.cc:243] Waited 8.79 ms on lock 0x7b540004bfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:34.352843 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:34.352898 (+    55us) raft_consensus.cc:1685] finished
0709 13:05:34.353003 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.353235 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8095,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":49,"raft.queue_time_us":116709,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":11182,"spinlock_wait_cycles":20220672,"thread_start_us":41731,"threads_started":1,"wal-append.queue_time_us":147407}
I20260709 13:05:34.357533 13942 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:34.358635 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.362924 17496 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.363929 17496 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:34.365352 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2398) took 1639 ms. Trace:
W20260709 13:05:34.366437 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52060.6us
I20260709 13:05:34.363924 13949 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:34.371080 13941 raft_consensus.cc:2410] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:05:34.375968 13874 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
W20260709 13:05:34.342245 17328 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.638s	user 0.007s	sys 0.012s
I20260709 13:05:34.370850 17496 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:34.392729 17617 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.382345 17440 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:34.430193 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.388958 17506 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:34.365842 13944 rpcz_store.cc:276] 0709 13:05:32.726238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.318026 (+591788us) service_pool.cc:224] Handling call
0709 13:05:33.318363 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:33.379598 (+ 61235us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:33.444000 (+ 64402us) log.cc:844] Serialized 14 byte log entry
0709 13:05:33.644172 (+200172us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:33.644686 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:05:33.644831 (+   145us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:33.644867 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.364584 (+719717us) spinlock_profiling.cc:243] Waited 60.7 ms on lock 0x7b54003b01d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:34.364663 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:34.364752 (+    89us) raft_consensus.cc:1685] finished
0709 13:05:34.364872 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.365129 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55895,"prepare.queue_time_us":64909,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":11807,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":16333,"spinlock_wait_cycles":139666688,"thread_start_us":117704,"threads_started":1,"wal-append.queue_time_us":164187}
I20260709 13:05:34.413111 17267 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:34.443588 13840 connection.cc:256] RPC call timeout handler was delayed by -1.1385510819999354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2523s for a response
I20260709 13:05:34.444489 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:34.394013 17328 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:34.423705 17496 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:34.469866 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 118053us
W20260709 13:05:34.471550 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.471736 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.472364 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.470135 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 425152us
I20260709 13:05:34.479099 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:34.488322 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.437325 17349 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
W20260709 13:05:34.501060 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.465703 17452 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
W20260709 13:05:34.503796 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 48098.7us
I20260709 13:05:34.453694 17440 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:34.515689 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.524598 17328 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:34.527886 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:34.528333 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:34.528770 13868 raft_consensus.cc:2376] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7. Re-sending same reply.
I20260709 13:05:34.528997 14551 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:34.529088 17547 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:34.476660 17617 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:34.532341 17617 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:34.535036 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.536509 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:34.536728 17621 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:34.537192 13950 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:34.538578 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:34.543125 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 201410us
I20260709 13:05:34.507687 17541 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:34.541810 14551 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
W20260709 13:05:34.547804 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.549173 13914 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:34.550213 17627 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.551316 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.559789 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.561262 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.513728 17496 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:34.572229 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.573401 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:34.557636 17328 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:05:34.574884 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.575843 17623 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.580490 13941 raft_consensus.cc:2410] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:05:34.580509 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.584656 13868 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:34.588091 17171 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:34.588975 17267 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:34.591555 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:34.594869 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.595948 13871 raft_consensus.cc:2410] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:05:34.597787 14023 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:34.028287 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:34.602671 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:34.605281 13874 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:34.603632 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.611523 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:34.612424 14017 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:34.617090 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.618223 13868 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:34.618731 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.619364 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.620924 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.624045 13950 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:34.605175 14024 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:34.619442 13872 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:34.621815 13868 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:34.631747 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:34.613648 17440 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:34.634362 13948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1797) took 1985 ms (client timeout 2123 ms). Trace:
I20260709 13:05:34.584012 17328 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:34.642932 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:34.636879 13948 rpcz_store.cc:269] 0709 13:05:32.642047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.884196 (+242149us) service_pool.cc:224] Handling call
0709 13:05:34.543943 (+1659747us) spinlock_profiling.cc:243] Waited 45.8 ms on lock 0x7b5800202e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030eeee 0000000000375939 00007fb2c83821e0 00007fb2c844b0c4 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:34.616816 (+ 72873us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b5800202e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030ef8c 00007fb2c030eef6 0000000000375939 00007fb2c83821e0 00007fb2c844b0c4 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:34.618044 (+  1228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110291840}
I20260709 13:05:34.645592 13866 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:34.642799 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.646891 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:34.646450 13950 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:34.649711 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.652792 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:34.653649 14023 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:34.658172 17328 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:34.658901 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.659917 13868 raft_consensus.cc:2410] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:05:34.662350 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.663380 13870 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:34.663565 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:34.577579 17627 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:34.664723 17629 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:34.665701 17629 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:34.666638 17629 consensus_queue.cc:1243] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:34.667245 17629 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:34.667647 17629 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:34.577458 17623 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:34.668745 17629 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:34.670220 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.671283 13869 raft_consensus.cc:2393] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:34.671351 17623 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:34.671896 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2619) took 1263 ms. Trace:
I20260709 13:05:34.672325 13869 rpcz_store.cc:276] 0709 13:05:33.408292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.408443 (+   151us) service_pool.cc:224] Handling call
0709 13:05:34.623847 (+1215404us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b5800003698. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:34.671802 (+ 47955us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7481088}
I20260709 13:05:34.674870 17629 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:34.675048 17496 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:34.675455 13871 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:34.678699 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2617) took 1297 ms. Trace:
I20260709 13:05:34.679123 13875 rpcz_store.cc:276] 0709 13:05:33.381199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.381514 (+   315us) service_pool.cc:224] Handling call
0709 13:05:34.678257 (+1296743us) spinlock_profiling.cc:243] Waited 8.89 ms on lock 0x7b54002e6dd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:34.678497 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20445952}
I20260709 13:05:34.680943 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.681543 13868 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:34.681854 13873 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:34.682421 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.682672 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.682487 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2620) took 1120 ms. Trace:
I20260709 13:05:34.683293 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:34.677836 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.685935 13873 rpcz_store.cc:276] 0709 13:05:33.562122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.562270 (+   148us) service_pool.cc:224] Handling call
0709 13:05:34.680603 (+1118333us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b5800003698. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:34.682421 (+  1818us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12115200}
I20260709 13:05:34.695330 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:34.678655 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.694940 13944 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:34.701000 13945 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:34.703418 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:34.704322 13869 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:34.703418 13941 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:34.705878 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.710815 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:34.711766 13866 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:34.703567 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 116921us
I20260709 13:05:34.719099 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.719290 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:34.728648 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.729132 13868 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:34.730847 14023 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:34.706315 17627 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:34.738214 13875 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:34.739388 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.736559 13867 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:34.737459 14017 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:34.746277 13872 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:34.747514 13870 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:34.751569 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:34.753468 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:34.754679 16994 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:34.756021 17480 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:34.756575 16994 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:34.759176 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:34.766269 13871 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:34.754719 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:34.776809 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.782382 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.782577 13950 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:34.785221 13950 pending_rounds.cc:85] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 2
I20260709 13:05:34.786497 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.787426 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:34.789180 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.789101 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.775301 17440 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:34.810164 13866 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:05:34.785614 13950 pending_rounds.cc:107] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:05:34.811079 13950 raft_consensus.cc:2889] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:34.812752 13840 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:34.788591 13875 raft_consensus.cc:2410] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:34.815392 13950 pending_rounds.cc:107] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:05:34.815793 13950 raft_consensus.cc:2889] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:34.817181 14016 raft_consensus.cc:2410] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:05:34.788832 17328 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:34.827140 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:34.828251 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:34.828555 14020 raft_consensus.cc:2410] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:34.829907 17496 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:34.830238 14017 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:34.830507 17496 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:34.835026 17496 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:34.835705 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:05:34.835048 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52103.7us
I20260709 13:05:34.840147 17344 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.840865 17344 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:34.842640 17612 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:34.844373 17626 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:34.824082 13867 raft_consensus.cc:2376] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7. Re-sending same reply.
I20260709 13:05:34.845423 17626 consensus_queue.cc:1243] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:05:34.842443 17622 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:34.847486 17626 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:34.848063 17626 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:34.850553 13838 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:34.851563 17622 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:34.852128 17622 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:34.853379 17623 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.854046 17623 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:34.856078 17344 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:34.856339 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:34.861647 13873 raft_consensus.cc:2410] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:05:34.853297 17622 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:34.870952 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:34.871685 17353 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:05:34.872300 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 57209.5us
I20260709 13:05:34.872462 17328 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:34.872900 13948 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:34.849370 17626 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:34.873617 17267 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:34.874478 17636 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:34.875116 17636 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:34.875597 17636 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:34.874615 17626 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:34.810225 17440 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:34.878829 17440 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:34.812824 17480 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:34.881014 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2417) took 1987 ms. Trace:
I20260709 13:05:34.881493 13943 rpcz_store.cc:276] 0709 13:05:32.893994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.624992 (+730998us) service_pool.cc:224] Handling call
0709 13:05:33.625364 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.872572 (+247208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.880736 (+1008164us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400088754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:34.880952 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562815744}
I20260709 13:05:34.886596 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:34.887096 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.887159 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:34.886591 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:34.888334 13872 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:34.888581 14022 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:34.888753 14023 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:34.889685 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:34.890520 14015 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:34.891279 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:34.894966 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:34.899564 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:34.901924 17621 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:34.902508 17621 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.904654 13871 raft_consensus.cc:2393] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:34.914642 13869 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:05:34.830485 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47993.5us
W20260709 13:05:34.919826 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.957535 14023 raft_consensus.cc:2410] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:05:34.931983 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:34.902908 17621 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:34.964501 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:34.903939 17267 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.984894 17267 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:34.990559 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:34.856410 17623 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:34.984157 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:34.952397 17442 consensus_meta.cc:287] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.650s	user 0.015s	sys 0.000s
W20260709 13:05:34.991920 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.993952 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 137699us
I20260709 13:05:34.994629 14019 raft_consensus.cc:2410] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:05:35.002774 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.963644 14024 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:35.018904 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 50907.9us
I20260709 13:05:35.020421 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:35.021481 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:35.021862 14017 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:35.025741 14015 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:35.034705 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:35.042017 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.043231 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:35.047181 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:34.996400 17171 consensus_queue.cc:1059] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 13:05:35.040715 17610 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.016s	sys 0.000s
I20260709 13:05:35.000648 17442 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:34.873633 17328 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:35.058297 17610 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.017s	sys 0.000s
W20260709 13:05:35.080255 17639 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:05:35.090265 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:35.092139 17442 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:35.092787 13943 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:35.093382 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:35.117672 14551 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:35.104580 17349 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:05:35.108237 17638 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.273s	user 0.000s	sys 0.002s
I20260709 13:05:35.110380 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1828) took 1112 ms. Trace:
W20260709 13:05:35.096004 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.129665 13949 rpcz_store.cc:276] 0709 13:05:33.997633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.368442 (+370809us) service_pool.cc:224] Handling call
0709 13:05:34.424152 (+ 55710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:34.711898 (+287746us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:34.785860 (+ 73962us) log.cc:844] Serialized 14 byte log entry
0709 13:05:34.957589 (+171729us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:34.997246 (+ 39657us) log.cc:844] Serialized 26 byte log entry
0709 13:05:34.997467 (+   221us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:34.997523 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.997626 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:35.110001 (+112375us) raft_consensus.cc:1685] finished
0709 13:05:35.110035 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.110279 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":274395,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":24529,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":21129,"spinlock_wait_cycles":29187968,"thread_start_us":355233,"threads_started":3,"wal-append.queue_time_us":178386}
W20260709 13:05:35.124029 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 53124.2us
I20260709 13:05:35.136067 14024 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:35.140471 17440 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:35.166460 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 149705us
I20260709 13:05:35.163957 17496 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:35.175405 17440 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:35.176242 17440 raft_consensus.cc:2764] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:05:35.176954 17440 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.123541 17349 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:05:35.179561 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 162725us
I20260709 13:05:35.173769 17496 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:35.177481 17440 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.195175 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:35.191475 17623 raft_consensus.cc:2764] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:05:35.215387 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:35.216722 17650 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:35.217154 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:35.217370 17650 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:35.217960 17650 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:35.218171 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:35.219239 17344 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:35.219446 13987 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:35.221282 17645 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:35.222322 17629 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:35.193243 17610 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:05:35.227537 17610 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:35.228024 17610 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:35.243686 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.221776 17645 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:35.206516 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 185589us
W20260709 13:05:35.259657 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.260435 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.233857 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 71576.9us
W20260709 13:05:35.279141 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 59810.3us
I20260709 13:05:35.283110 14551 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:35.229305 17610 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.179061 17349 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:35.292672 13914 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:35.262723 17645 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:35.303966 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 86925.7us
I20260709 13:05:35.306167 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:35.303301 17328 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:35.315697 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1822) took 1320 ms. Trace:
W20260709 13:05:35.317024 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.316246 13946 rpcz_store.cc:276] 0709 13:05:33.995214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.357023 (+361809us) service_pool.cc:224] Handling call
0709 13:05:34.357552 (+   529us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:34.434179 (+ 76627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:34.434212 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:34.880376 (+446164us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b2c000e5d98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:05:34.920805 (+ 40429us) log.cc:844] Serialized 26 byte log entry
0709 13:05:35.065070 (+144265us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:35.065125 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.065213 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:35.312329 (+247116us) raft_consensus.cc:1685] finished
0709 13:05:35.312361 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.315600 (+  3239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":139269,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":13324,"spinlock_wait_cycles":2703744,"thread_start_us":120714,"threads_started":2,"wal-append.queue_time_us":169761}
I20260709 13:05:35.298117 17623 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:35.338529 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:35.314297 17349 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:35.409415 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:35.338587 17661 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:35.402566 17636 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.526s	user 0.014s	sys 0.011s
W20260709 13:05:35.418241 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 103352us
I20260709 13:05:35.423575 13869 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:35.399855 17328 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:35.426764 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 137037us
I20260709 13:05:35.408790 17612 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:35.431568 17648 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:05:35.427578 17653 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:35.434168 17653 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:35.435323 17653 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.431619 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:35.439502 17655 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:05:35.422472 17610 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:35.449942 13989 leader_election.cc:336] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.049s (SENT)
I20260709 13:05:35.447196 17629 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:35.450690 17267 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:35.462381 13989 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:35.419196 17661 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.464354 17267 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:35.464711 17661 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:35.453512 17665 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:35.454030 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:35.465353 17665 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.465811 17665 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:35.466235 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 152622us
I20260709 13:05:35.467664 17666 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:35.467137 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:35.468801 17663 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:35.469347 17629 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:35.469630 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:35.458285 17621 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.469836 17629 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.470443 17629 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:35.471005 17662 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:35.471975 17621 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:35.472312 17626 raft_consensus.cc:3055] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:35.472743 17626 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:35.473459 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:35.473559 17626 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.454541 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:35.424988 17636 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:35.476642 17636 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:35.481098 17650 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:35.481588 13870 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:35.483669 13874 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:35.485966 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:35.487087 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:35.487118 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:35.487329 17665 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.416298 17623 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:35.513550 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 46004.1us
I20260709 13:05:35.492703 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:35.515317 17632 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.507885 17621 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:35.501937 17440 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:35.522194 17645 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:35.512472 17664 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:35.531749 17664 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:35.532228 17664 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:35.533329 17645 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:35.518836 17629 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.536926 17629 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:35.530167 17632 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:35.543891 17632 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:35.537705 17650 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:35.538216 17328 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:35.540400 17623 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:35.541119 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 67743.1us
I20260709 13:05:35.524932 17665 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:35.516376 17636 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:35.558892 17670 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:35.559259 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:35.558995 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:35.517212 17666 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:35.568477 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:35.570638 17666 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.576678 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:35.578131 17623 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:35.578243 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.578694 17623 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.578974 14551 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:05:35.579094 17623 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:35.579419 13988 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:35.579986 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.582655 17663 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:05:35.594357 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:35.595882 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:35.596812 17665 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:35.597249 17665 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.597287 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:35.597690 17665 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:35.598352 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:35.599220 14556 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:05:35.600279 14561 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:35.601766 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.602042 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.602793 14558 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:05:35.603349 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.603479 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.603981 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.604529 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.605049 17667 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:35.605999 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:35.606127 17659 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:35.607023 17659 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:35.606828 17665 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.607589 17659 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:35.607920 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:35.608676 17659 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.609256 17665 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:35.604987 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.602883 14551 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:35.606122 17667 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:35.611148 17667 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:35.611572 17667 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:35.614693 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.617379 14556 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:05:35.618199 17496 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:35.618651 17496 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.619010 17496 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:35.622008 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 167587us
W20260709 13:05:35.630743 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62441.6us
I20260709 13:05:35.633178 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:35.634009 14556 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:05:35.586169 17650 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:35.642416 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 192494us
W20260709 13:05:35.643761 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.655153 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:35.655930 13944 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:35.657342 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:35.658602 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.659718 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:35.662750 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:35.664151 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:35.668522 17633 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.839s	user 0.009s	sys 0.000s
I20260709 13:05:35.670680 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 612) took 2846 ms. Trace:
W20260709 13:05:35.671067 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 91861.3us
I20260709 13:05:35.674867 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.672024 13988 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:35.677040 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 79818.6us
I20260709 13:05:35.648917 17636 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.679646 13941 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:35.681288 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:35.681866 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.670674 17675 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:35.671113 13947 rpcz_store.cc:276] 0709 13:05:32.824526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.429788 (+605262us) service_pool.cc:224] Handling call
0709 13:05:33.430199 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:33.442821 (+ 12622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:33.442874 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:33.566729 (+123855us) log.cc:844] Serialized 26 byte log entry
0709 13:05:34.539760 (+973031us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b180044c140. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:05:34.814079 (+274319us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:34.814142 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.824105 (+  9963us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:35.667746 (+843641us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:35.670229 (+  2483us) raft_consensus.cc:1685] finished
0709 13:05:35.670253 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.670434 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3181,"raft.queue_time_us":99,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":3649,"spinlock_wait_cycles":115068544,"thread_start_us":162497,"threads_started":1,"wal-append.queue_time_us":1242970}
I20260709 13:05:35.694182 17636 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.695640 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:35.697299 17636 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:35.699470 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:35.702450 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:35.703903 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.704744 14556 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:35.707966 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:35.710218 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:35.711009 13945 raft_consensus.cc:2393] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:35.712122 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:35.715062 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:35.721176 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.680032 17344 raft_consensus.cc:697] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:35.754544 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:35.755353 14561 raft_consensus.cc:2376] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 12. Re-sending same reply.
I20260709 13:05:35.758292 17669 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:35.773291 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 66578.4us
I20260709 13:05:35.780097 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.742144 17636 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:35.759677 17675 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:35.781844 17676 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:35.792201 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:35.794427 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 86585.8us
I20260709 13:05:35.799930 14024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
W20260709 13:05:35.808374 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 152940us
I20260709 13:05:35.799991 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:05:35.810892 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 111576us
I20260709 13:05:35.816490 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:35.792044 17675 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:35.795102 17623 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.818444 17675 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:35.760893 17496 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:35.794217 17676 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:35.824376 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:35.808494 17666 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:05:35.829901 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:35.838326 17669 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:35.850243 17344 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:35.839916 17676 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:35.856986 17658 log.cc:927] Time spent T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.438s	user 0.000s	sys 0.002s
I20260709 13:05:35.867727 17661 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.807379 17664 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:35.886363 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:35.887543 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.887966 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:35.888504 14561 raft_consensus.cc:2410] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:35.888615 14556 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:35.889838 17629 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:35.888161 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.897246 17664 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:36.478873 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 670941us
I20260709 13:05:35.946025 13941 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:35.955708 17667 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:35.902845 17672 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:05:35.903451 17661 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:36.199216 13988 leader_election.cc:336] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.534s (SENT)
I20260709 13:05:36.487516 17626 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:36.486100 17679 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.487699 13988 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:36.488711 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 303754us
I20260709 13:05:35.905463 17620 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:36.490371 17620 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:36.488298 17679 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:36.493228 17679 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:35.895386 17623 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:36.229535 17676 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:36.041348 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.489323 17661 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:36.478108 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:35.909310 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:05:36.494745 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:36.495457 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:36.510483 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.907852 14561 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:35.892819 13946 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:35.891624 13950 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:36.480300 13838 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:36.506363 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:36.514084 14556 raft_consensus.cc:2410] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:05:35.904618 13989 leader_election.cc:336] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.990s (SENT)
W20260709 13:05:36.517876 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 613250us
W20260709 13:05:36.522950 13942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.005s	sys 0.014s
W20260709 13:05:36.523461 13942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.005s	sys 0.014s
I20260709 13:05:36.527237 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:36.520308 17675 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.002s
W20260709 13:05:36.523599 13989 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.490s (SENT) [suppressed 50 similar messages]
I20260709 13:05:36.507985 17496 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:36.534641 13838 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:36.535162 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 625947us
I20260709 13:05:36.535770 14024 raft_consensus.cc:606] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:05:36.523083 17669 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:36.532815 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:36.552562 13869 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:36.537846 17664 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:36.540514 17669 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.561654 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:36.562307 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:36.569847 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:36.546162 17659 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:36.549209 17665 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:36.575628 13872 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:36.558634 17664 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.578543 14553 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:05:36.559897 13840 connection.cc:256] RPC call timeout handler was delayed by -1.165141927000036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
I20260709 13:05:36.576252 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:36.536736 17675 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:36.579967 13840 connection.cc:256] RPC call timeout handler was delayed by -1.145341927000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 13:05:36.575852 14552 raft_consensus.cc:2368] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8.
I20260709 13:05:36.549405 17620 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:36.581533 17574 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:36.582005 17620 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:36.582185 17574 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:36.583544 17675 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:36.584798 17574 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:36.584853 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:36.576097 17674 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.301s	user 0.001s	sys 0.001s
I20260709 13:05:36.589885 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:36.570820 17669 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:36.587862 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:36.587904 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:36.589157 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:36.570294 17667 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:36.590606 13875 raft_consensus.cc:2376] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8. Re-sending same reply.
I20260709 13:05:36.642851 17692 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:36.643795 17692 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:36.646627 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:36.647516 13869 raft_consensus.cc:2376] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9. Re-sending same reply.
I20260709 13:05:36.579937 17664 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:36.577237 17665 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:36.574244 13771 catalog_manager.cc:5697] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 reported cstate change: term changed from 4 to 6, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 6 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:36.659531 17690 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:36.660195 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 182237us
I20260709 13:05:36.651685 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:36.651842 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:36.661223 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 80089.7us
I20260709 13:05:36.670553 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:36.660971 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:36.650619 13867 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:36.672000 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:36.675505 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:36.677151 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:36.660566 13870 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:36.665126 13987 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:05:36.665176 17686 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:05:36.666827 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:36.579035 17623 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
W20260709 13:05:36.665870 17683 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:05:36.698778 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:36.586894 17659 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:36.728404 13875 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:36.728484 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 204900us
I20260709 13:05:36.746284 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:36.729775 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:36.728863 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:36.754910 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:36.662084 17665 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:36.729202 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:36.745886 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:36.660573 17690 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.750813 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:36.733500 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1851) took 1080 ms. Trace:
I20260709 13:05:36.757105 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:36.755623 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 226114us
W20260709 13:05:36.761493 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 91030.7us
I20260709 13:05:36.758462 13872 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:36.765969 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:36.766870 13872 raft_consensus.cc:2368] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8.
I20260709 13:05:36.767266 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:36.769862 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:36.755470 13866 raft_consensus.cc:2376] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8. Re-sending same reply.
W20260709 13:05:36.780817 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 81081.9us
I20260709 13:05:36.755569 17667 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:36.761044 13942 rpcz_store.cc:276] 0709 13:05:35.652896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.653022 (+   126us) service_pool.cc:224] Handling call
0709 13:05:35.657051 (+  4029us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.662468 (+  5417us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:35.729904 (+ 67436us) log.cc:844] Serialized 14 byte log entry
0709 13:05:36.732951 (+1003047us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:36.733032 (+    81us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:36.733126 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.733233 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.733408 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":136230,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"thread_start_us":1065656,"threads_started":3,"wal-append.queue_time_us":819495}
I20260709 13:05:36.755146 17676 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:36.803397 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:36.809769 14555 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:36.763795 17692 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:36.783197 17686 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:05:36.814353 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:36.781088 17344 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:36.824046 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 549) took 1256 ms. Trace:
I20260709 13:05:36.824528 13873 rpcz_store.cc:276] 0709 13:05:35.558256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.558405 (+   149us) service_pool.cc:224] Handling call
0709 13:05:35.559508 (+  1103us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:35.559995 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.560022 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:35.560694 (+   672us) log.cc:844] Serialized 52 byte log entry
0709 13:05:35.739863 (+179169us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:35.739919 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.739999 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.813945 (+1073946us) raft_consensus.cc:1685] finished
0709 13:05:36.813975 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.814168 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":524,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"thread_start_us":169664,"threads_started":1,"wal-append.queue_time_us":461681}
I20260709 13:05:36.827526 13989 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:36.794236 17661 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:36.831521 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 103373us
I20260709 13:05:36.834432 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41088 (request call id 15) took 1034 ms. Trace:
I20260709 13:05:36.834992 14024 rpcz_store.cc:276] 0709 13:05:35.790694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.794345 (+  3651us) service_pool.cc:224] Handling call
0709 13:05:36.813510 (+1019165us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b54003bbfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c342d853 00007fb2c83fb60a 00007fb2c19817d4 00007fb2c1981762 00007fb2c1981691 00007fb2c198160c 00007fb2c19808c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:36.813634 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1689425792}
W20260709 13:05:36.838822 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 173797us
I20260709 13:05:36.847733 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:36.823242 17344 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:36.853719 17661 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:36.854337 17344 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:36.856388 17661 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:36.856788 17659 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:36.857342 17682 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
W20260709 13:05:36.853792 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.859637 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 630) took 1086 ms. Trace:
I20260709 13:05:36.780779 17690 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:36.860426 13945 rpcz_store.cc:276] 0709 13:05:35.773479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.778778 (+  5299us) service_pool.cc:224] Handling call
0709 13:05:35.786600 (+  7822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:35.786956 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.786996 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:35.787730 (+   734us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.023924 (+236194us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:36.023981 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.024088 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.859382 (+835294us) raft_consensus.cc:1685] finished
0709 13:05:36.859404 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.859564 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":235685,"threads_started":1,"wal-append.queue_time_us":757525}
W20260709 13:05:36.865134 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 68710.1us
I20260709 13:05:36.872846 13771 catalog_manager.cc:5697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 reported cstate change: term changed from 4 to 6, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 6 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.851670 17676 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:36.860654 17666 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.883009 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:36.873178 17681 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.299s	user 0.001s	sys 0.001s
W20260709 13:05:36.872789 17688 log.cc:927] Time spent T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 13:05:36.890552 13839 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:36.869627 17694 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.907994 14555 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:36.922603 13869 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:36.911733 17690 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:36.911309 13771 catalog_manager.cc:5697] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:36.843585 17699 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:05:36.912374 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1846) took 1266 ms. Trace:
I20260709 13:05:36.915042 13948 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:36.929476 17692 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:36.926837 17694 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:36.881398 17626 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:05:36.881225 17541 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:05:36.932210 13771 catalog_manager.cc:5697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.935153 14024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:05:36.935776 14024 raft_consensus.cc:606] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:05:36.935071 13771 catalog_manager.cc:5697] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 6 to 8, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 8 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.936142 14024 raft_consensus.cc:614] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:36.936851 13771 catalog_manager.cc:5697] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 reported cstate change: term changed from 5 to 6, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 6 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.908859 17666 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:36.945583 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:36.919831 13772 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57648 (request call id 279) took 12798 ms (client timeout 15000 ms). Trace:
I20260709 13:05:36.916894 17706 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:36.877154 17697 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.932214 17694 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:36.927673 13943 rpcz_store.cc:276] 0709 13:05:35.645615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.650434 (+  4819us) service_pool.cc:224] Handling call
0709 13:05:35.670535 (+ 20101us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:35.671066 (+   531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.671094 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:35.671657 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:05:35.983975 (+312318us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:35.984041 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.984165 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.912126 (+927961us) raft_consensus.cc:1685] finished
0709 13:05:36.912151 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.912304 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1895,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":4908,"thread_start_us":311758,"threads_started":1,"wal-append.queue_time_us":864718}
I20260709 13:05:36.945041 17623 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 13:05:36.942701 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 64354.5us
I20260709 13:05:36.943514 17667 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:36.876360 17687 log.cc:927] Time spent T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:05:36.920099 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:05:36.946605 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 56182.3us
I20260709 13:05:36.937798 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:36.883529 17661 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:36.881938 17701 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:36.960943 13988 leader_election.cc:336] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.975s (SENT)
I20260709 13:05:36.961424 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:36.991405 17344 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:36.994206 17666 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:36.995831 17669 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:36.959782 17541 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:36.964252 17574 raft_consensus.cc:3055] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:36.998492 17574 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:36.999476 17574 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:37.000001 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 79995.6us
I20260709 13:05:37.001034 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:37.001410 17692 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:37.001919 17692 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:37.002374 17692 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:36.946108 13772 rpcz_store.cc:269] 0709 13:05:24.121535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.122249 (+   714us) service_pool.cc:224] Handling call
0709 13:05:36.919534 (+12797285us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:29.047091 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:29.047672 (+   581us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:29.047709 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:05:29.047728 (+    19us) tablet.cc:662] Decoding operations
0709 13:05:29.048836 (+  1108us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:29.048902 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:29.048927 (+    25us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:05:29.054121 (+  5194us) tablet.cc:701] Row locks acquired
0709 13:05:29.054176 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:05:29.061397 (+  7221us) write_op.cc:270] Start()
0709 13:05:29.108359 (+ 46962us) write_op.cc:276] Timestamp: P: 1783602329054450 usec, L: 0
0709 13:05:29.108410 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 13:05:29.259064 (+150654us) log.cc:844] Serialized 4810 byte log entry
0709 13:05:29.970457 (+711393us) op_driver.cc:464] REPLICATION: finished
0709 13:05:30.139769 (+169312us) write_op.cc:301] APPLY: starting
0709 13:05:30.139922 (+   153us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:30.140195 (+   273us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:30.144093 (+  3898us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:30.151978 (+  7885us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:30.152403 (+   425us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:05:30.152482 (+    79us) write_op.cc:312] APPLY: finished
0709 13:05:30.152945 (+   463us) log.cc:844] Serialized 89 byte log entry
0709 13:05:30.153187 (+   242us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:30.153357 (+   170us) write_op.cc:454] Released schema lock
0709 13:05:30.153417 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:30.153865 (+   448us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:30.153908 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":611523,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":141269,"apply.run_cpu_time_us":4395,"apply.run_wall_time_us":24599,"num_ops":12,"prepare.queue_time_us":409047,"prepare.run_cpu_time_us":48624,"prepare.run_wall_time_us":338497,"raft.queue_time_us":440830,"raft.run_cpu_time_us":51878,"raft.run_wall_time_us":97504,"replication_time_us":856827,"spinlock_wait_cycles":4864,"thread_start_us":826943,"threads_started":4,"wal-append.queue_time_us":169188,"wal-append.run_cpu_time_us":97304,"wal-append.run_wall_time_us":1737187}]]}
W20260709 13:05:36.999411 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.006279 17667 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:37.008332 17703 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:37.009012 17703 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.002290 17694 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:37.014946 17665 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.015987 17675 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:37.016441 17675 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:37.018199 13771 catalog_manager.cc:5697] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 reported cstate change: term changed from 3 to 5, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 5 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:37.011684 17692 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.020301 13771 catalog_manager.cc:5697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 3 to 4, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 4 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:37.022637 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.026244 17669 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:37.023581 17703 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:37.030300 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 70982.6us
I20260709 13:05:37.031426 13839 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:37.033188 13839 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:37.022624 17692 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:37.034494 13771 catalog_manager.cc:5697] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 reported cstate change: term changed from 3 to 6, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 6 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:37.017477 17675 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.060559 17675 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:37.024884 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:37.035756 17694 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:37.063671 17694 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:37.064079 17694 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:37.064422 17496 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:37.065083 17694 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.069103 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:37.035684 17664 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.037541 13948 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:37.037703 13987 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:37.071964 17664 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:37.072063 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:37.072921 17664 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:37.073225 17676 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:37.038215 17703 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.073901 17676 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:37.074366 17703 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.075059 17676 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.076692 17703 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:37.077292 17623 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:37.038508 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:36.982223 17697 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.038913 17705 consensus_queue.cc:1059] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:37.039227 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:37.040453 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:37.041993 13945 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:37.082504 13941 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:37.042047 17574 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:37.043785 17692 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:37.044130 17707 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:37.017426 17665 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:37.047554 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 219182us
W20260709 13:05:37.053045 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 299620us
I20260709 13:05:37.036197 17701 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.053766 17679 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:37.054837 17626 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:05:37.058485 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 115718us
I20260709 13:05:37.069820 13944 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:37.087921 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.116322 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:37.117568 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:37.091192 17697 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:37.074229 17694 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:37.092325 17705 raft_consensus.cc:3055] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:05:37.093220 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52888us
I20260709 13:05:37.124922 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:37.093804 17699 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:37.094705 17344 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:37.098470 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:37.105381 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:37.128754 13941 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:37.106245 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:37.109634 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:37.110328 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:37.110975 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:37.131017 14022 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:37.111660 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:37.131843 13945 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:05:37.112995 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:37.138582 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.113565 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:37.103701 17707 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:37.150187 14015 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:37.119645 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:37.121927 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:37.122963 14017 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:37.151612 13838 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:37.127146 14016 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:37.105435 17496 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:05:37.127671 17705 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:37.155926 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:37.178539 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 53763us
I20260709 13:05:37.181259 14024 raft_consensus.cc:2410] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:05:37.194407 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:37.185702 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:37.195226 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:37.163575 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 297513us
W20260709 13:05:37.184406 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.196099 13946 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 13:05:37.178232 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 58729.8us
I20260709 13:05:37.196762 13942 raft_consensus.cc:2393] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:05:37.153128 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 120026us
I20260709 13:05:37.197746 14023 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:37.198369 13839 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:37.207626 14019 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:05:37.214434 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:37.158981 17690 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:37.206650 14522 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:37.190351 17626 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:37.221803 14023 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:37.221184 13945 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:05:37.235545 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.247700 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:37.248500 14019 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:37.251101 13915 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:37.250309 13944 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
W20260709 13:05:37.251253 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 47196.9us
I20260709 13:05:37.252278 13950 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:37.253608 13838 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:37.254339 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:37.191094 17697 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:05:37.223073 17690 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:37.277613 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 79312.8us
I20260709 13:05:37.239082 17659 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:37.286239 14023 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:05:37.287999 14023 pending_rounds.cc:85] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 6
I20260709 13:05:37.288261 14023 pending_rounds.cc:107] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:05:37.288673 14023 raft_consensus.cc:2889] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.266776 17712 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:37.294596 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 93518.9us
I20260709 13:05:37.295562 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:37.296278 14015 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:05:37.258090 17714 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:37.280831 17690 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:37.320689 13945 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:05:37.282384 17620 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:37.308533 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 101952us
W20260709 13:05:37.299088 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 79860.3us
I20260709 13:05:37.325831 13945 pending_rounds.cc:85] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 5
I20260709 13:05:37.294616 17712 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.261916 17623 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:05:37.261682 17667 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:05:37.286365 17662 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:37.334347 13945 pending_rounds.cc:107] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:05:37.204214 17707 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.274797 17692 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:37.298454 17714 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.338977 13945 raft_consensus.cc:2889] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.343592 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:37.346356 17714 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:37.334291 17620 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.380962 13839 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:37.330835 17626 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:37.338207 17712 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:37.395117 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 55478.4us
I20260709 13:05:37.394989 14016 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:37.334910 17724 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:37.422611 14016 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:37.424216 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:37.286948 17705 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:37.420356 17681 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:05:37.435882 14021 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:37.394912 17662 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:37.441221 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:37.394898 17710 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:37.443552 14023 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:37.445896 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:37.452241 14023 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:05:37.453183 13839 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:37.454912 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:37.399557 17693 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:05:37.456507 14018 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:37.460110 17693 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:37.460690 17693 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:37.461377 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:37.462150 14018 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:37.462697 14020 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:37.463176 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:37.463717 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:37.457384 17720 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:05:37.464365 14020 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:37.465242 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:37.465320 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:37.466508 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:37.466611 17718 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:05:37.467383 17620 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:37.467732 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:37.468947 14023 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:37.463757 14018 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:37.440601 17664 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:37.492707 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:37.493382 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 239096us
I20260709 13:05:37.494345 13989 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:37.494534 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:37.494817 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:37.495016 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 159046us
I20260709 13:05:37.434230 17728 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:37.476835 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:37.491362 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 158791us
I20260709 13:05:37.497792 14022 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:37.461589 17693 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.499694 17693 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:37.500850 13989 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:37.450704 17707 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:37.500649 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 47573.3us
I20260709 13:05:37.502017 13989 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:37.508141 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.504120 14023 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:37.522639 14020 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:37.523852 14022 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:05:37.476330 17620 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.494639 17692 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:37.500737 17496 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:05:37.533819 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 60204.5us
I20260709 13:05:37.525143 17669 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:37.504081 17623 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:37.539103 13839 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:37.528779 17699 raft_consensus.cc:2764] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:05:37.550719 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51523us
W20260709 13:05:37.545085 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52506.2us
I20260709 13:05:37.552021 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:37.528823 17705 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:37.470918 17626 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.537014 17669 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.536140 17728 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:37.536875 17620 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.571127 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:37.565800 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 244448us
I20260709 13:05:37.553889 13989 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:37.540606 17344 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:37.455088 17710 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.571199 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:37.575551 13869 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:37.575930 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.579698 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:37.538372 17659 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:05:37.580690 17496 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:37.582142 17716 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:05:37.589356 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.609321 17716 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:37.610702 17716 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.612766 17716 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:37.613286 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:37.592388 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.559573 17728 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.614555 17728 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:37.557063 17719 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:37.620234 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 68348.7us
I20260709 13:05:37.622371 17623 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:05:37.500634 17724 consensus_queue.cc:1175] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e1649107c781428692458f10597c5742" responder_term: 9 status { last_received { term: 8 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:05:37.630467 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 50852.4us
W20260709 13:05:37.651125 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.635022 17669 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:37.605806 17541 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:05:37.635318 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 81490.4us
I20260709 13:05:37.744272 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:37.745117 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:37.745255 17623 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:37.746520 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:37.746336 17623 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.639735 14024 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:37.642509 14019 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
W20260709 13:05:37.642699 17730 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 13:05:37.644050 17733 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:05:37.555996 17662 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.751037 17739 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:37.752076 17739 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:37.752658 17739 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:37.622784 17661 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:37.603659 17699 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:37.753682 17699 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:37.754184 17699 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:37.753820 17739 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.607405 17659 consensus_queue.cc:1243] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:37.749409 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:37.587750 17742 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:37.758260 13948 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:37.758395 17742 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.760483 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:37.761221 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:37.761648 17742 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:37.763613 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:37.607318 17697 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:37.775223 14018 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:37.621687 17666 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:05:37.826234 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62711.7us
W20260709 13:05:37.829435 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.762552 17667 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:37.831020 17541 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:37.771250 17742 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.847349 14021 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:37.851055 17746 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:37.851626 17746 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:37.852025 17746 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:37.852200 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:05:37.851133 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 89150.1us
W20260709 13:05:37.854568 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 296812us
I20260709 13:05:37.741678 17701 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:37.854826 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:37.778970 17697 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:37.861203 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:37.843894 17664 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:37.868113 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:37.869016 14021 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:37.862963 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 364752us
W20260709 13:05:37.870759 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 101772us
I20260709 13:05:37.879168 13988 leader_election.cc:400] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8.
I20260709 13:05:37.879734 13988 leader_election.cc:403] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:05:37.880813 17755 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:37.881016 17574 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.881549 17574 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.882025 17691 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.882651 17691 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.883930 17574 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:37.884740 17620 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:37.872462 17496 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.876593 17661 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:37.888384 17714 consensus_meta.cc:287] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.515s	user 0.008s	sys 0.011s
I20260709 13:05:37.856357 17679 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:37.897300 17661 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8.
I20260709 13:05:37.870365 17664 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.906463 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:37.906538 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 335403us
I20260709 13:05:37.906841 13941 pending_rounds.cc:85] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 3
I20260709 13:05:37.912058 13941 pending_rounds.cc:107] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:05:37.861495 17626 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.912361 13941 raft_consensus.cc:2889] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.912753 13941 pending_rounds.cc:107] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:05:37.913019 13941 raft_consensus.cc:2889] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.913028 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:37.858090 17742 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.924971 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:37.926496 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 48701us
I20260709 13:05:37.928841 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:37.930968 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:37.884771 17691 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:37.937834 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:37.945047 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:37.920125 17664 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:37.945117 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:37.917874 17675 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:37.950484 14016 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:37.954314 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:37.956560 13950 raft_consensus.cc:2368] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9.
I20260709 13:05:37.890671 17665 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
W20260709 13:05:37.965824 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.918298 17679 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:37.964043 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:37.947619 17661 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:05:38.004462 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 91555.9us
W20260709 13:05:38.031852 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.890992 17714 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:37.927426 17626 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:37.994503 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.955205 17747 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:38.055420 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 117638us
W20260709 13:05:38.060554 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 96747.5us
I20260709 13:05:37.979110 17666 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:38.070007 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:38.061975 17726 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.031644 17755 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.964872 17541 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.034368 17620 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:38.076668 17736 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:05:38.060487 17692 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:38.059767 17662 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:38.108155 17693 consensus_meta.cc:287] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.608s	user 0.012s	sys 0.003s
I20260709 13:05:38.110982 17692 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:38.111279 17620 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:38.119839 17725 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.120747 17725 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:38.123167 17756 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:05:38.130381 17752 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.157s	user 0.002s	sys 0.000s
I20260709 13:05:38.132122 17754 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.132906 17754 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:38.107621 17694 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.955s	user 0.018s	sys 0.006s
I20260709 13:05:38.158612 17697 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:38.160535 17710 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:38.179370 13943 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:38.161520 17762 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:38.167013 17742 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:38.167466 17679 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:38.079639 17726 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.171074 13989 leader_election.cc:400] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8.
W20260709 13:05:38.208735 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 64153us
W20260709 13:05:38.180100 17712 consensus_meta.cc:287] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.750s	user 0.008s	sys 0.005s
I20260709 13:05:38.209944 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:38.109127 17755 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:38.228420 17755 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:38.229607 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 4dd265d61fc141a48ca9d3d5c8f52921 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:05:38.230652 17753 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:05:38.204373 17690 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.880s	user 0.015s	sys 0.000s
W20260709 13:05:38.238880 14524 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.002s	sys 0.006s
W20260709 13:05:38.240177 14524 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.002s	sys 0.006s
W20260709 13:05:38.247982 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 676397us
I20260709 13:05:38.231940 17344 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.253134 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:38.259538 14555 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:38.260720 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:38.238593 17697 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:38.266536 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:38.267457 14554 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:38.269380 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:38.270311 14558 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:38.250713 17667 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:05:38.286680 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 434571us
I20260709 13:05:38.300006 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:38.301448 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:38.216892 17712 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:38.286036 17707 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.773s	user 0.012s	sys 0.000s
I20260709 13:05:38.243057 17690 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.317445 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 1867) took 1279 ms. Trace:
I20260709 13:05:38.323216 13943 rpcz_store.cc:276] 0709 13:05:37.038132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.038241 (+   109us) service_pool.cc:224] Handling call
0709 13:05:38.317246 (+1279005us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54001f4fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:38.317376 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2527061248}
W20260709 13:05:38.283034 17728 consensus_meta.cc:287] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.668s	user 0.008s	sys 0.004s
I20260709 13:05:38.322371 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:38.286005 17661 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.332058 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:38.268348 17344 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.341084 14558 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
W20260709 13:05:38.327600 17699 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.004s
I20260709 13:05:38.312424 17710 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
W20260709 13:05:38.346747 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 276329us
I20260709 13:05:38.316124 17667 raft_consensus.cc:697] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:38.347764 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.347374 13945 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:38.354983 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:38.328234 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:38.362145 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:38.365115 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:38.342653 17719 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:38.371009 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:05:38.387657 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.320010 17742 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:05:38.347324 17750 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.371518 13987 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:38.345664 17714 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:38.343930 17748 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:38.396200 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.344251 17703 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:05:38.405385 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:38.344610 17661 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.395368 17676 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:38.404273 17724 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:38.419170 17667 consensus_queue.cc:237] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.422966 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:38.339759 17728 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.357172 17541 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:05:38.427129 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:38.354259 17699 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.433859 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:38.378230 17762 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:38.401024 17719 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:38.406854 17675 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:38.445111 14559 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:05:38.406925 17748 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:38.429445 17714 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:38.448181 14554 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:05:38.507277 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:38.420946 17707 consensus_queue.cc:1059] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:05:38.457413 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:38.519490 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.446979 17676 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:38.465276 17719 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:38.457784 17726 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:38.538913 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 141559us
I20260709 13:05:38.552773 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:38.551049 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.501345 17748 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:38.557233 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:38.509509 17626 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:38.561228 13838 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:38.406541 17750 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:38.528923 17666 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:38.607681 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:38.613484 17703 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:38.614230 17774 consensus_queue.cc:1059] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.100s
I20260709 13:05:38.614274 17703 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.608822 13943 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
W20260709 13:05:38.618038 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 56927.8us
I20260709 13:05:38.550028 17659 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:38.665339 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:38.669411 13949 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:38.553521 17676 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:38.576170 17726 consensus_queue.cc:1059] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 13:05:38.598749 17725 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:38.662868 17690 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:38.712049 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 52558.3us
W20260709 13:05:38.732679 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 301691us
I20260709 13:05:38.697628 17659 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:38.736788 13987 leader_election.cc:400] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9.
I20260709 13:05:38.737252 13987 leader_election.cc:403] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:05:38.734989 17690 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:38.751231 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 194073us
W20260709 13:05:38.734375 17770 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.334s	user 0.002s	sys 0.002s
I20260709 13:05:38.698210 17666 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:05:38.733466 17786 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:38.737988 17773 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
W20260709 13:05:38.764410 17768 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
W20260709 13:05:38.770609 17767 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.342s	user 0.000s	sys 0.001s
W20260709 13:05:38.861207 17775 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:05:38.864293 13914 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:38.865242 17755 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:05:38.771072 17765 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.326s	user 0.002s	sys 0.000s
I20260709 13:05:38.865705 17755 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:38.867293 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 426) took 1406 ms. Trace:
I20260709 13:05:38.867733 14020 rpcz_store.cc:276] 0709 13:05:37.461111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.532408 (+ 71297us) service_pool.cc:224] Handling call
0709 13:05:37.543466 (+ 11058us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:37.769373 (+225907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:37.769412 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:37.953878 (+184466us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b2c000e8788. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:05:37.978859 (+ 24981us) log.cc:844] Serialized 26 byte log entry
0709 13:05:38.252116 (+273257us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:38.252153 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.252249 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:38.866931 (+614682us) raft_consensus.cc:1685] finished
0709 13:05:38.866956 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.867215 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4642,"raft.queue_time_us":33170,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3529,"spinlock_wait_cycles":2441088,"thread_start_us":276909,"threads_started":2,"wal-append.queue_time_us":293095}
I20260709 13:05:38.761185 17659 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:38.672636 17762 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.752053 17496 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:38.870576 17496 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:38.871502 17496 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.771960 17781 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:38.872867 17496 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:38.873653 17781 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:38.875139 17750 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:38.724931 17785 consensus_queue.cc:1059] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
W20260709 13:05:38.781112 13988 leader_election.cc:336] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.855s (SENT)
I20260709 13:05:38.875597 17712 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:38.876017 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:38.876296 17785 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:38.876742 17785 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:38.876992 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 95939.3us
W20260709 13:05:38.752714 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 93279.7us
I20260709 13:05:38.798465 13840 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:38.876894 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 429) took 1414 ms. Trace:
I20260709 13:05:38.824432 17790 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.828949 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:38.883356 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 85058.2us
I20260709 13:05:38.829174 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:38.880707 14022 rpcz_store.cc:276] 0709 13:05:37.462023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.605334 (+143311us) service_pool.cc:224] Handling call
0709 13:05:37.741387 (+136053us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:37.755901 (+ 14514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:37.755934 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:37.852447 (+ 96513us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b2c000e8788. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:05:37.852969 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 13:05:38.192450 (+339481us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:38.192518 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.192627 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:38.876519 (+683892us) raft_consensus.cc:1685] finished
0709 13:05:38.876550 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.876810 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13886,"raft.queue_time_us":14341,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":142,"spinlock_wait_cycles":33241344,"thread_start_us":325608,"threads_started":1,"wal-append.queue_time_us":387455}
I20260709 13:05:38.884159 13869 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
W20260709 13:05:38.884325 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.832701 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:38.885512 13948 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:38.835713 13871 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:38.837460 13872 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:38.837817 13875 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:05:38.839125 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:38.840487 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1877) took 1346 ms. Trace:
I20260709 13:05:38.888192 13949 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
W20260709 13:05:38.814630 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 404574us
I20260709 13:05:38.845075 13866 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:38.752473 17739 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:38.857242 17771 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.180s	user 0.001s	sys 0.001s
I20260709 13:05:38.760313 17690 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:38.858721 17783 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.195s	user 0.001s	sys 0.001s
I20260709 13:05:38.901022 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:38.866801 17755 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.874706 17781 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.780682 17705 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.734804 17661 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:38.876073 17694 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:38.888463 13945 rpcz_store.cc:276] 0709 13:05:37.494246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.499020 (+  4774us) service_pool.cc:224] Handling call
0709 13:05:37.499383 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.354459 (+855076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.840155 (+485696us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b5400087854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:38.840397 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1949564160}
W20260709 13:05:38.889458 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.843088 17754 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:38.885015 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:38.890139 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:38.893671 17669 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:38.848152 17786 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.859989 17692 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:38.851996 17728 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:38.837466 17344 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:38.920145 17746 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.068s	user 0.000s	sys 0.015s
W20260709 13:05:38.914908 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 174860us
I20260709 13:05:38.917992 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:38.919806 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:38.894953 13944 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:38.893450 17665 consensus_queue.cc:1059] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
W20260709 13:05:38.938228 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.924394 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.934659 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 49743.7us
I20260709 13:05:38.939397 13948 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:38.939944 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:38.940343 13945 raft_consensus.cc:2410] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:38.929234 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:38.931299 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.944118 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:38.878445 17699 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:38.944967 14019 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:38.945082 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:38.940743 14015 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:38.941885 13867 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:38.920557 17694 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.925066 17661 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:38.975567 13942 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:38.926631 17755 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:38.969472 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:38.976184 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:38.976856 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:38.982151 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:38.977458 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:38.979671 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:38.983083 14023 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:05:38.980715 13946 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:38.981191 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:38.877630 17785 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.989156 14022 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:05:38.995389 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 55879.1us
I20260709 13:05:38.995975 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:38.997056 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:38.934430 17574 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:38.989307 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:38.930817 17786 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:39.007036 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:38.980657 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:38.934063 17802 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:38.934274 17667 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.920380 17705 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.996945 13945 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:39.003230 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:39.005038 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:39.015137 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:39.015766 14022 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:38.851243 17676 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:39.016144 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:39.006870 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:38.964250 17344 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.896497 17790 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:38.989948 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:38.971503 17694 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:39.012305 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:39.014638 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:39.010820 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:39.062278 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:39.061949 17667 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:39.063589 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:39.089834 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 92875.8us
I20260709 13:05:38.983397 17703 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:39.003906 17574 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:39.079236 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:39.024263 14018 raft_consensus.cc:2393] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:39.083295 13872 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:39.105441 14015 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:05:39.081596 14023 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:39.084435 17344 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:39.092331 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:39.098658 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:39.099340 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.099442 13875 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:39.106727 14024 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:05:39.062366 17803 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.170s
I20260709 13:05:38.963564 17746 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:39.103546 13944 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:38.968907 17659 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:39.018988 17793 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.204s
I20260709 13:05:39.081573 17666 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.220088 13947 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:39.112113 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:39.340941 13871 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:39.210911 14016 raft_consensus.cc:2410] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:05:39.319842 13945 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:05:39.197871 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 91873.6us
I20260709 13:05:39.115118 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:39.102895 17776 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.275s	user 0.000s	sys 0.005s
I20260709 13:05:39.352983 14021 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:39.351922 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:39.308830 17792 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.429s	user 0.001s	sys 0.001s
I20260709 13:05:39.358235 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:39.356827 14017 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
W20260709 13:05:39.227298 17794 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 13:05:39.110828 17574 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:39.109318 17662 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:39.225525 17806 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
W20260709 13:05:39.235495 17763 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 13:05:39.226418 17797 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:05:39.358719 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:39.224447 17798 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.390568 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:39.405391 14021 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:39.405711 13945 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:05:39.220731 17796 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.381517 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:39.407775 14017 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:39.390614 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:39.391218 17799 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:05:39.411857 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:39.381222 17661 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:39.359663 17692 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:39.378178 17662 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:39.402688 17800 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.360126 17679 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.355350 17762 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.354928 17666 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:39.360538 17344 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:39.210105 17726 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.175s
I20260709 13:05:39.416461 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:39.419438 13867 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:39.107569 17741 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.535s
I20260709 13:05:39.423728 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:39.425199 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:39.431484 17719 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.865s	user 0.011s	sys 0.002s
I20260709 13:05:39.424577 17620 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:39.432824 17662 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:39.448910 13988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.031s	sys 0.031s
I20260709 13:05:39.437083 17690 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:39.460969 13988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.031s	sys 0.031s
I20260709 13:05:39.461269 13867 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:39.434157 17692 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:39.467432 17807 log.cc:927] Time spent T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:05:39.468389 17692 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:39.469102 17692 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:39.469573 13942 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:39.471725 14021 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:39.414948 17798 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:39.484421 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 125781us
I20260709 13:05:39.415158 17796 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:39.438263 17661 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:39.486616 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:39.500450 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.510900 17746 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:39.443866 17679 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:39.485154 17699 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:39.488289 17664 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.513s	user 0.008s	sys 0.008s
I20260709 13:05:39.559363 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:39.560699 13867 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:39.469587 17692 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:39.570840 17692 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:39.572791 17669 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.573320 17669 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:39.456573 17719 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:39.564134 17664 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:39.576135 17719 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:39.515028 17799 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:05:39.480079 17818 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:39.487602 17344 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:39.447890 17762 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:39.446990 17800 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:39.593364 14021 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:39.496074 17705 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:39.621847 17821 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:05:39.623940 17813 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:05:39.668593 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.635389 17344 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:39.561691 17699 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:39.682302 17754 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.683154 17790 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:39.563297 17817 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:39.748037 14021 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:39.779553 14020 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:05:39.921439 14020 pending_rounds.cc:85] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 4
I20260709 13:05:39.921806 14020 pending_rounds.cc:107] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:05:39.922181 14020 raft_consensus.cc:2889] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:39.746315 13988 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:39.923619 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 177455us
I20260709 13:05:39.776000 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:39.925920 14023 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:39.782245 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:39.764864 13872 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:39.775343 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:39.783003 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:39.927564 14022 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:39.763689 14017 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:05:39.919886 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 433420us
W20260709 13:05:39.919530 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:39.929169 13946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.003s	sys 0.015s
W20260709 13:05:39.929559 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:39.931416 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.934227 13947 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:05:39.929534 13946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.003s	sys 0.015s
W20260709 13:05:40.071498 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 142545us
W20260709 13:05:39.980630 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.102759 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:40.068331 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:40.084203 14018 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:05:39.942536 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:40.104629 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2489) took 1160 ms. Trace:
I20260709 13:05:40.105255 13867 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:39.836330 17623 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.014s	sys 0.021s
W20260709 13:05:40.071817 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 118386us
W20260709 13:05:39.912009 17816 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:05:39.879360 17694 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.771s	user 0.011s	sys 0.002s
W20260709 13:05:39.776628 17748 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.177s	user 0.008s	sys 0.003s
I20260709 13:05:39.916167 17785 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:40.172896 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:39.759503 17755 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.751s	user 0.005s	sys 0.006s
W20260709 13:05:39.914968 17739 consensus_meta.cc:287] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.019s	user 0.012s	sys 0.006s
I20260709 13:05:40.105121 14018 rpcz_store.cc:276] 0709 13:05:38.944301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.153082 (+208781us) service_pool.cc:224] Handling call
0709 13:05:39.782815 (+629733us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:40.104535 (+321720us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77819264}
W20260709 13:05:39.771152 17496 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.897s	user 0.012s	sys 0.002s
W20260709 13:05:39.749300 17697 consensus_meta.cc:287] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.474s	user 0.013s	sys 0.004s
I20260709 13:05:39.792984 17777 raft_consensus.cc:3055] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:39.791468 17719 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:40.282223 17719 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:40.282466 17574 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.893s	user 0.004s	sys 0.008s
W20260709 13:05:40.067081 17791 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.219s	user 0.004s	sys 0.000s
I20260709 13:05:40.282935 17574 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:40.060992 17786 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.041s	user 0.008s	sys 0.008s
I20260709 13:05:40.284262 17786 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.285321 17574 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:40.285390 13868 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 9. Ops: []
I20260709 13:05:39.697232 17669 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:39.854270 17818 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:40.286705 17732 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.286886 17818 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.287647 17669 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.275842 17820 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:40.288575 17820 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:40.288990 17820 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:39.914316 17666 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:39.684376 17798 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.082449 17620 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.290658 17620 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:39.877226 17821 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:40.292972 17821 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:39.788712 17667 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:40.294415 17667 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:40.294936 17667 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.297293 17667 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:40.297957 17786 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:39.755182 17541 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.240878 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.301867 13867 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:05:40.131641 17692 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.688230 17679 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:40.177414 17623 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.024s	sys 0.038s
I20260709 13:05:39.944527 17699 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:39.912418 17796 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:39.930308 17705 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:40.082379 17746 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:39.941848 17754 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.261144 17748 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.261981 17694 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.279911 14018 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:40.278228 17777 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:40.287505 17732 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.061506 17762 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:40.177809 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 74847.6us
I20260709 13:05:40.302951 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.351771 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:40.352000 13872 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:05:39.946626 17675 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:40.327018 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 401332us
I20260709 13:05:40.326030 17692 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:39.944741 17817 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.242381 17803 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:05:40.340162 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2499) took 1393 ms. Trace:
I20260709 13:05:40.393664 14018 rpcz_store.cc:276] 0709 13:05:38.947140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.279353 (+1332213us) service_pool.cc:224] Handling call
0709 13:05:40.279724 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.339812 (+ 60088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.340091 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:40.280282 17714 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:40.323726 17747 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.508s	user 0.022s	sys 0.067s
I20260709 13:05:40.298981 17626 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:05:40.350898 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:39.911371 17662 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.353710 13866 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:40.354243 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.356760 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.266633 17739 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9.
W20260709 13:05:40.391602 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 82669.4us
I20260709 13:05:40.339582 17699 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:40.398274 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.337571 17746 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.323292 17541 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.404595 13875 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:40.406816 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1787) took 1469 ms. Trace:
I20260709 13:05:40.340196 17705 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:40.408133 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 304407us
I20260709 13:05:40.631273 13867 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:40.631599 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.632294 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.467970 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 399671us
I20260709 13:05:40.551259 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:40.633813 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:40.634842 14558 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:40.510006 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:40.474186 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.636180 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 108414us
I20260709 13:05:40.427876 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:40.638001 13989 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.135s (SENT)
I20260709 13:05:40.638634 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:40.634034 14019 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:40.634693 13872 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
W20260709 13:05:40.644441 14019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2513) took 1692 ms (client timeout 2091 ms). Trace:
W20260709 13:05:40.517331 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.481909 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:40.482769 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:40.710616 14019 rpcz_store.cc:269] 0709 13:05:38.951970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.099125 (+147155us) service_pool.cc:224] Handling call
0709 13:05:40.551026 (+1451901us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:40.644347 (+ 93321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80318848}
W20260709 13:05:40.428198 17833 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:05:40.631657 17692 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:40.429756 17827 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:05:40.488076 14018 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: [6.4-7.5]
I20260709 13:05:40.728120 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2502) took 1780 ms. Trace:
I20260709 13:05:40.728732 14018 rpcz_store.cc:276] 0709 13:05:38.948083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.404424 (+1456341us) service_pool.cc:224] Handling call
0709 13:05:40.487803 (+ 83379us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:40.727595 (+239792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.728001 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:40.434682 17829 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:05:40.486955 17748 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:40.428977 17832 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:05:40.437523 17694 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:40.567287 17762 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:40.633564 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 281892us
I20260709 13:05:40.424739 17620 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.645555 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.488866 17732 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.645712 17820 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.645972 17754 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:40.486442 17728 raft_consensus.cc:3055] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:40.427867 17786 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:40.407514 17747 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.592s	user 0.023s	sys 0.068s
I20260709 13:05:40.420374 17749 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:40.649000 17662 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:40.652089 17705 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.653594 14015 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:40.654443 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:40.637074 13870 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
W20260709 13:05:40.751353 13870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2640) took 1421 ms (client timeout 1667 ms). Trace:
I20260709 13:05:40.656910 17344 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:40.660492 17785 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:40.661334 17665 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.661734 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:40.751878 13870 rpcz_store.cc:269] 0709 13:05:39.330103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.330381 (+   278us) service_pool.cc:224] Handling call
0709 13:05:40.356538 (+1026157us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b5800003698. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:40.751266 (+394728us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8104192}
I20260709 13:05:40.662834 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:40.663024 17541 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.754691 13948 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:40.633072 14020 rpcz_store.cc:276] 0709 13:05:38.937022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.971086 (+ 34064us) service_pool.cc:224] Handling call
0709 13:05:39.779138 (+808052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.082598 (+303460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.406577 (+323979us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b54000abc50. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:40.406746 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1792846720}
I20260709 13:05:40.636061 13947 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:05:40.755556 14015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2492) took 1810 ms (client timeout 2158 ms). Trace:
I20260709 13:05:40.756181 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:40.757047 13947 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:40.632640 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.757704 17665 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.663914 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:40.633658 13915 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:40.665376 13950 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:05:40.666371 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2632) took 1816 ms. Trace:
I20260709 13:05:40.666646 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:40.667860 17719 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.398623 17623 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:40.489368 14023 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:05:40.669679 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:40.670955 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:40.671873 17746 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.674189 17822 raft_consensus.cc:3055] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:40.674631 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:40.675274 17707 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.676299 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:40.676955 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:40.677001 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:40.678079 14524 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:40.403888 17781 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.275s
I20260709 13:05:40.679086 17739 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:40.740427 17762 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.757992 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:40.774089 13947 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:40.774425 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.775148 13870 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:40.776031 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.759253 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.759662 13941 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:40.450726 17714 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.471064 17718 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:40.484668 17817 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:40.680280 17789 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.681177 17831 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:40.683627 14017 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:40.490837 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2511) took 1539 ms. Trace:
I20260709 13:05:40.686509 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 560) took 2367 ms. Trace:
I20260709 13:05:40.688846 14558 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:05:40.688961 17800 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:40.690068 17661 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:40.690227 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 52257.9us
I20260709 13:05:40.690826 17664 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:40.699203 17814 consensus_queue.cc:1059] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:40.700852 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:40.703962 17724 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.704167 14522 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:40.637418 13873 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
W20260709 13:05:40.709015 13866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2634) took 1772 ms (client timeout 1793 ms). Trace:
I20260709 13:05:40.709969 13949 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:05:40.711525 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.511926 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:40.711982 13871 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:05:40.715674 17774 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:40.740671 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:40.728470 17741 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.742348 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.779134 17665 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.746311 17728 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:40.833148 17699 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.760759 13868 rpcz_store.cc:276] 0709 13:05:38.881759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.882419 (+   660us) service_pool.cc:224] Handling call
0709 13:05:38.893325 (+ 10906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.286045 (+1392720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.666131 (+380086us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54003acfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:40.666311 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3200773888}
I20260709 13:05:40.833380 17728 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.833884 17699 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.834630 17728 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:40.835278 13868 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:40.836743 17699 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.764277 14024 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
W20260709 13:05:40.837708 14024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2498) took 1890 ms (client timeout 2231 ms). Trace:
I20260709 13:05:40.661908 17679 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.838532 17679 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.839700 17574 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:05:40.839756 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 99207us
I20260709 13:05:40.840672 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:40.841192 17679 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.841394 17574 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:40.755952 14015 rpcz_store.cc:269] 0709 13:05:38.945118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.152409 (+207291us) service_pool.cc:224] Handling call
0709 13:05:39.782013 (+629604us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:40.755379 (+973366us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b54002986d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:40.755495 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1699278976}
I20260709 13:05:40.769805 13948 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:05:40.842734 13948 pending_rounds.cc:85] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 5
I20260709 13:05:40.838536 17774 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.842993 13948 pending_rounds.cc:107] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:05:40.843317 13948 raft_consensus.cc:2889] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:40.753293 17749 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.770556 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:40.846658 17774 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:40.740110 17825 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:40.766717 17822 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:40.847728 17749 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.847644 17825 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.848222 17822 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.848853 13872 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 5.
I20260709 13:05:40.849735 17822 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:40.846750 13944 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:40.732007 17828 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:40.763232 13945 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:40.435123 17777 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.772517 17762 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:40.850649 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 146561us
I20260709 13:05:40.851184 17749 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:40.765861 14555 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:05:40.776582 13870 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:40.766884 17746 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:40.814637 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:40.832260 17741 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.851872 13914 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:40.764998 17707 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.760618 13950 pending_rounds.cc:85] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 4
I20260709 13:05:40.880012 13950 pending_rounds.cc:107] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:05:40.880415 13950 raft_consensus.cc:2889] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:40.853658 17724 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.761917 17719 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:40.851257 17825 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.882532 17777 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:40.762904 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2512) took 1811 ms. Trace:
I20260709 13:05:40.762280 17623 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.890283 14023 rpcz_store.cc:276] 0709 13:05:38.951649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.488768 (+1537119us) service_pool.cc:224] Handling call
0709 13:05:40.489116 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:40.762505 (+273389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.762828 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:40.855360 17732 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:40.891811 17732 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:40.892138 17623 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:40.892421 17732 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:40.885717 17749 raft_consensus.cc:697] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:40.764434 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:40.855542 17814 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:40.855854 17831 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.761102 13942 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:40.856587 17718 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:05:40.760422 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 126874us
W20260709 13:05:40.857578 13989 leader_election.cc:336] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.423s (SENT)
I20260709 13:05:40.858037 17800 raft_consensus.cc:3055] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:40.858566 17714 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:40.770284 17781 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:05:40.854741 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 153941us
I20260709 13:05:40.854627 13946 raft_consensus.cc:2410] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 13:05:40.855505 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 177541us
I20260709 13:05:40.913586 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.905807 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:40.918315 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:40.921269 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:40.922118 14015 raft_consensus.cc:2376] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10. Re-sending same reply.
I20260709 13:05:40.859385 14022 rpcz_store.cc:276] 0709 13:05:38.951258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.475224 (+1523966us) service_pool.cc:224] Handling call
0709 13:05:40.475628 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.490353 (+ 14725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.490400 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:40.490448 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:40.490492 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.490580 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.490770 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:40.863163 14024 rpcz_store.cc:269] 0709 13:05:38.946780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.152895 (+206115us) service_pool.cc:224] Handling call
0709 13:05:40.674228 (+1521333us) spinlock_profiling.cc:243] Waited 31.6 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:05:40.837625 (+163397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72573312}
I20260709 13:05:40.927017 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:40.927768 14015 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:40.931618 13872 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:40.932848 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.864812 13869 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:40.858770 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2515) took 1852 ms. Trace:
W20260709 13:05:40.864081 13866 rpcz_store.cc:269] 0709 13:05:38.937055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.937316 (+   261us) service_pool.cc:224] Handling call
0709 13:05:40.708742 (+1771426us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54000e9554. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:40.708898 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3124902656}
I20260709 13:05:40.875315 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:40.877468 13838 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:40.934628 14017 rpcz_store.cc:276] 0709 13:05:39.006395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.683128 (+1676733us) service_pool.cc:224] Handling call
0709 13:05:40.683404 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.858478 (+175074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.858707 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:40.765295 14554 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:40.892297 17707 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:40.882684 17620 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:40.804441 17820 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:40.860483 13874 rpcz_store.cc:276] 0709 13:05:38.319425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.321085 (+  1660us) service_pool.cc:224] Handling call
0709 13:05:38.321512 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.321802 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:39.235740 (+913938us) spinlock_profiling.cc:243] Waited 64.2 ms on lock 0x7b1800455e60. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c6e2b3c7 00007fb2c343d9a7 00007fb2c344d0bf
0709 13:05:39.235903 (+   163us) log.cc:844] Serialized 14 byte log entry
0709 13:05:39.450064 (+214161us) log.cc:844] Serialized 14 byte log entry
0709 13:05:39.786487 (+336423us) spinlock_profiling.cc:243] Waited 7.05 ms on lock 0x7b1800455e60. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:40.684761 (+898274us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b1800455e60. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b
0709 13:05:40.685975 (+  1214us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b1800455e60. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287
0709 13:05:40.686101 (+   126us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:40.686155 (+    54us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:40.686213 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.686286 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.686408 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1850,"prepare.queue_time_us":176743,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":96,"raft.queue_time_us":189246,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":5030,"spinlock_wait_cycles":242643200,"thread_start_us":532274,"threads_started":3,"wal-append.queue_time_us":341329,"wal-append.run_cpu_time_us":1852,"wal-append.run_wall_time_us":1067821}
I20260709 13:05:40.937362 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:40.896538 17699 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:40.885087 17705 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:40.893875 13943 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
W20260709 13:05:40.945034 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 87484.3us
W20260709 13:05:40.956524 13989 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.490s (SENT)
W20260709 13:05:40.902590 17818 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.020s	sys 0.017s
I20260709 13:05:40.957329 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.901510 17825 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:40.948765 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:40.949230 13874 raft_consensus.cc:2368] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9.
I20260709 13:05:40.952129 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.853766 17789 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:40.904749 17814 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:40.951977 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:40.979878 14023 raft_consensus.cc:2376] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10. Re-sending same reply.
I20260709 13:05:40.982494 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:40.947335 13988 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:40.963177 13868 raft_consensus.cc:2410] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:05:40.952508 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:40.963419 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 86018.4us
I20260709 13:05:40.950317 17664 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:40.952992 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 77751.6us
W20260709 13:05:40.960516 17818 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.020s	sys 0.018s
I20260709 13:05:40.975397 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.967973 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.978017 14017 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:40.903761 17800 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:40.956950 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:40.943576 17623 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:05:40.963303 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.960445 17707 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:40.987262 14020 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:40.988173 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.988440 17724 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:40.984634 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:41.003109 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:40.996110 13874 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:40.909184 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:40.906342 13945 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
W20260709 13:05:41.007879 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51401.5us
I20260709 13:05:40.975049 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:40.876658 17828 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:41.017807 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:40.963618 17705 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:41.001621 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:40.954351 17714 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:05:40.986196 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:40.965322 17679 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:41.011677 13872 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:41.012971 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:41.018270 13989 leader_election.cc:336] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.615s (SENT)
I20260709 13:05:41.028340 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:41.019263 14016 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:40.993233 17664 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:41.020421 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 111345us
I20260709 13:05:41.022835 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:40.995260 17762 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:41.020903 13949 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:41.039333 13950 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:41.032207 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:41.046185 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:41.047060 13871 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:41.022867 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:40.990085 17801 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.009s	sys 0.022s
I20260709 13:05:40.957894 17699 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:40.965529 17825 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:41.057535 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:41.017956 17779 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 2.328s	user 0.027s	sys 0.015s
I20260709 13:05:41.044055 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:41.063220 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:40.908600 17781 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:41.054016 17786 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:05:40.959322 17774 consensus_queue.cc:1059] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
W20260709 13:05:41.051564 17801 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.009s	sys 0.022s
I20260709 13:05:41.018971 17661 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:41.024698 17746 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:41.078727 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:41.083113 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 64881.8us
I20260709 13:05:41.044909 17817 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:41.053849 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:41.089964 13866 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:41.053954 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:41.091394 13869 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:41.043037 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:41.092141 13873 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:40.960613 17620 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:41.059501 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:41.053306 17694 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:41.054345 17623 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:41.021530 17785 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:41.046530 13838 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:41.063486 17739 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:41.109110 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 62664.9us
I20260709 13:05:41.110112 13838 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:41.110347 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:41.112190 13944 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:41.018589 17707 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.063053 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:41.113673 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:41.064677 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:41.070825 13942 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:41.070329 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:41.069479 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:41.067175 17781 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:41.067478 17774 consensus_queue.cc:1243] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer 4dd265d61fc141a48ca9d3d5c8f52921 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:41.123495 13945 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:41.071839 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:41.029517 17679 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.051247 17664 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:41.052117 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:41.129833 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 77816.2us
I20260709 13:05:41.130569 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:41.098349 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:41.099548 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:41.069906 17843 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.094s	user 0.001s	sys 0.001s
I20260709 13:05:41.061213 17754 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:41.105821 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.132345 14023 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:41.107856 13950 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:41.133802 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:40.990293 17749 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:41.115713 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:41.116691 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.137043 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:41.116551 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:41.138170 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:41.138419 13873 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:41.119400 13872 raft_consensus.cc:2368] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9.
I20260709 13:05:41.057740 17821 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:05:41.127068 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:41.119737 17662 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:41.146835 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:41.089432 17817 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:41.128432 13871 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:05:41.148229 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:41.148845 13872 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
W20260709 13:05:41.063403 17779 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 2.376s	user 0.027s	sys 0.016s
W20260709 13:05:41.132731 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.108489 17785 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.138744 13869 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:41.129137 17679 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:41.116348 17707 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:41.127138 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:41.144119 14024 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:41.119128 17820 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:41.146857 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:41.140062 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.152858 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:41.024375 17718 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:41.115293 17623 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:41.147639 17781 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:41.144448 17694 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:41.144883 17839 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:41.155149 17785 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:41.161164 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.153728 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:41.167299 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:41.154230 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:41.186712 13873 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:41.147569 17662 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.154623 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:41.104157 17814 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:41.177697 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:41.158649 17831 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:41.182377 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:41.183931 13869 raft_consensus.cc:2410] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:41.186646 13950 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
W20260709 13:05:41.193986 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 200448us
I20260709 13:05:41.143666 17754 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:41.190845 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:41.164790 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:41.228439 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.135996 17705 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:41.239149 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.277060 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:41.572321 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.248521 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:41.252679 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 61902.7us
I20260709 13:05:41.574009 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:41.336257 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:41.443063 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.238607 13944 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:41.570832 13868 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:05:41.572060 14561 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:41.337204 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:41.573094 13838 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:41.580837 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:05:41.655021 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:41.678378 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 104492us
I20260709 13:05:41.678400 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:41.169000 17820 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:41.684664 13944 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:41.685619 13868 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:41.693710 13868 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:41.696132 13868 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:41.697206 13866 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:41.701378 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.691632 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:41.353677 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:41.403798 17344 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.199913 17755 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:41.706877 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:41.706966 17755 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.707165 17820 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:41.707794 17755 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:41.711262 17754 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:41.720487 13873 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:05:41.723202 17719 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.837s	user 0.012s	sys 0.000s
I20260709 13:05:41.727195 17800 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:41.728786 17800 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:41.732419 13875 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
W20260709 13:05:41.735445 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:41.736037 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:41.397701 17728 consensus_meta.cc:287] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.003s
I20260709 13:05:41.200133 17789 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:41.743979 17746 consensus_meta.cc:287] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.655s	user 0.011s	sys 0.000s
I20260709 13:05:41.186406 17839 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:41.744604 17623 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:41.744764 17746 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:41.744606 17839 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:41.746093 13946 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:05:41.398613 17844 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.287s	user 0.000s	sys 0.000s
I20260709 13:05:41.745163 17828 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:41.749071 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 176109us
W20260709 13:05:41.659735 17849 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.169s	user 0.003s	sys 0.001s
I20260709 13:05:41.538900 17620 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:41.195266 17662 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:41.804724 13840 leader_election.cc:336] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.768s (SENT)
I20260709 13:05:41.430658 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:41.249176 17699 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:41.310175 17741 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:41.766969 13867 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:05:41.768388 13943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.004s	sys 0.043s
I20260709 13:05:41.775867 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:41.632319 17694 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:41.805238 14019 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.019s	sys 0.033s
I20260709 13:05:41.779042 13946 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:41.814031 13946 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:41.305354 17838 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:41.387153 17825 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:41.793390 13941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.022s	sys 0.029s
W20260709 13:05:41.776219 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.768327 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2655) took 1257 ms. Trace:
W20260709 13:05:41.810976 13943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.004s	sys 0.043s
I20260709 13:05:41.811620 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:41.792511 17822 consensus_meta.cc:287] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.918s	user 0.011s	sys 0.002s
I20260709 13:05:41.814911 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:41.815148 14019 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.020s	sys 0.033s
I20260709 13:05:41.773782 17719 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:41.830714 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:41.831009 13941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.022s	sys 0.032s
I20260709 13:05:41.830109 13875 rpcz_store.cc:276] 0709 13:05:40.510953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.654089 (+143136us) service_pool.cc:224] Handling call
0709 13:05:41.768109 (+1114020us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b54003abbd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:41.768250 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2237875456}
W20260709 13:05:41.773633 17777 consensus_meta.cc:287] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.887s	user 0.012s	sys 0.002s
W20260709 13:05:41.840941 14017 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.003s	sys 0.047s
W20260709 13:05:41.796100 17732 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.897s	user 0.012s	sys 0.000s
W20260709 13:05:41.841800 14017 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.003s	sys 0.048s
W20260709 13:05:41.843113 14524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.018s	sys 0.028s
I20260709 13:05:41.843586 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:41.844096 14524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.018s	sys 0.028s
W20260709 13:05:41.844728 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 537138us
W20260709 13:05:41.844856 14021 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.023s	sys 0.023s
W20260709 13:05:41.845314 14021 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.023s	sys 0.023s
I20260709 13:05:41.845712 14015 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:41.845494 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:41.823493 13946 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:41.822860 14552 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:41.852101 13875 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:41.854405 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:41.857144 13946 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:41.307019 17817 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:41.863427 13989 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.008s	sys 0.013s
I20260709 13:05:41.852329 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:41.822687 17838 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:41.706334 17344 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:41.878665 13989 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.008s	sys 0.013s
W20260709 13:05:41.918468 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 712103us
W20260709 13:05:41.920224 13989 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.952s (SENT) [suppressed 23 similar messages]
I20260709 13:05:41.843003 17620 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:41.839450 17719 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.743909 17755 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:41.914183 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 61947.8us
W20260709 13:05:41.915946 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44734.7us
I20260709 13:05:41.945267 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:41.945928 14559 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
W20260709 13:05:41.923184 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 231669us
I20260709 13:05:41.848281 17747 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.929458 13989 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:41.931886 17846 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.020s	sys 0.042s
I20260709 13:05:41.953994 14024 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
W20260709 13:05:41.954931 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.917429 17838 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:41.933216 17719 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:41.958962 13942 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:05:41.974645 17853 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:41.959792 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:41.962085 17746 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:41.963410 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:41.986608 14015 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:41.987798 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:41.988726 14015 raft_consensus.cc:2393] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 5: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:41.964694 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:41.989347 14018 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:41.967023 17789 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:41.967696 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:41.969079 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2488) took 1064 ms. Trace:
W20260709 13:05:41.970669 17852 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
W20260709 13:05:41.959988 14020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.010s	sys 0.031s
I20260709 13:05:41.847086 17732 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:41.995496 14020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.011s	sys 0.032s
I20260709 13:05:41.997273 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2514) took 3045 ms. Trace:
I20260709 13:05:41.974385 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2498) took 1062 ms. Trace:
W20260709 13:05:41.912865 17664 consensus_meta.cc:287] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.762s	user 0.011s	sys 0.003s
I20260709 13:05:41.980218 17853 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:41.971586 17846 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.020s	sys 0.043s
I20260709 13:05:41.994474 13867 rpcz_store.cc:276] 0709 13:05:40.905005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.905242 (+   237us) service_pool.cc:224] Handling call
0709 13:05:41.968876 (+1063634us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54002d5354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:41.969003 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1793665408}
I20260709 13:05:41.995031 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:42.020220 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.999828 13869 rpcz_store.cc:276] 0709 13:05:40.911935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.684088 (+772153us) service_pool.cc:224] Handling call
0709 13:05:41.684375 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:41.701815 (+ 17440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.701862 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:41.734581 (+ 32719us) log.cc:844] Serialized 26 byte log entry
0709 13:05:41.923912 (+189331us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:41.923988 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.924107 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:41.973443 (+ 49336us) raft_consensus.cc:1685] finished
0709 13:05:41.973571 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:41.974113 (+   542us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55919,"raft.queue_time_us":17724,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":99,"thread_start_us":188838,"threads_started":1,"wal-append.queue_time_us":231838}
W20260709 13:05:41.992269 17707 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.822s	user 0.012s	sys 0.001s
I20260709 13:05:42.000049 13946 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:05:42.022372 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:42.034729 13866 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:41.998145 14021 rpcz_store.cc:276] 0709 13:05:38.952324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.681343 (+1729019us) service_pool.cc:224] Handling call
0709 13:05:40.681738 (+   395us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:05:40.682600 (+   862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.682632 (+    32us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:05:40.683443 (+   811us) log.cc:844] Serialized 104 byte log entry
0709 13:05:41.148693 (+465250us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:41.845647 (+696954us) log.cc:844] Serialized 14 byte log entry
0709 13:05:41.928574 (+ 82927us) log.cc:844] Serialized 14 byte log entry
0709 13:05:41.970880 (+ 42306us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b18002eabc0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f
0709 13:05:41.979523 (+  8643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.979609 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:41.996802 (+ 17193us) raft_consensus.cc:1685] finished
0709 13:05:41.996855 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:41.997186 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99880,"prepare.queue_time_us":822672,"raft.queue_time_us":105225,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":17869,"spinlock_wait_cycles":18101376,"thread_start_us":1134981,"threads_started":2,"wal-append.queue_time_us":714744}
I20260709 13:05:41.974684 17661 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:05:41.957975 17831 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:05:42.036425 14018 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:41.916654 17726 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:05:42.037581 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:41.957602 17747 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:41.991992 17699 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:42.054904 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:42.055713 14018 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:42.056728 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:42.041863 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:41.994165 17789 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.030925 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:41.971784 17620 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:05:42.020267 17853 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:41.971355 17777 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:05:42.064882 13875 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:42.045164 14554 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:42.045890 14024 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:41.953289 17754 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:05:42.029103 17705 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:05:42.057487 14018 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:41.995527 17694 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:05:42.021333 17664 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:42.029452 17541 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:42.062788 17789 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:42.042232 17741 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:05:42.098526 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 448) took 1116 ms. Trace:
I20260709 13:05:42.102365 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:42.062747 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:42.105495 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2516) took 1076 ms. Trace:
I20260709 13:05:42.087306 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2508) took 1126 ms. Trace:
I20260709 13:05:42.055374 17699 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:42.045001 17781 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.866s	user 0.006s	sys 0.007s
I20260709 13:05:42.038748 17707 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:42.073501 17679 consensus_meta.cc:287] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.906s	user 0.015s	sys 0.000s
I20260709 13:05:42.066970 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:42.103896 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:42.106043 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:42.148756 13869 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:42.150545 13872 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
W20260709 13:05:42.151299 13871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.006s	sys 0.019s
W20260709 13:05:42.151695 13871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.006s	sys 0.019s
I20260709 13:05:42.106827 14021 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:42.099049 14019 rpcz_store.cc:276] 0709 13:05:40.981826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.981936 (+   110us) service_pool.cc:224] Handling call
0709 13:05:40.987510 (+  5574us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:40.987752 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.987778 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:40.988318 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 13:05:41.151359 (+163041us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:41.836971 (+685612us) log.cc:844] Serialized 14 byte log entry
0709 13:05:41.837254 (+   283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.837352 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.098244 (+260892us) raft_consensus.cc:1685] finished
0709 13:05:42.098269 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.098439 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31460,"prepare.queue_time_us":888244,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":2643,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":4738,"thread_start_us":799421,"threads_started":2,"wal-append.queue_time_us":471352}
W20260709 13:05:42.108856 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 476552us
I20260709 13:05:42.115530 14023 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:42.082211 13916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.000s	sys 0.016s
W20260709 13:05:42.153896 13916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.000s	sys 0.016s
I20260709 13:05:42.154561 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1806) took 1032 ms. Trace:
W20260709 13:05:42.155833 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.155893 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:42.156792 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:42.157626 13950 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:42.156790 13945 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:42.155166 14019 rpcz_store.cc:276] 0709 13:05:41.122236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.153348 (+1031112us) service_pool.cc:224] Handling call
0709 13:05:42.153852 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.154124 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:42.154162 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:42.154225 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:42.154271 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.154347 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.154483 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:42.085999 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:42.084370 17874 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:42.162395 14018 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:42.162508 17874 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:42.163563 17874 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:42.164801 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.126432 13875 rpcz_store.cc:276] 0709 13:05:41.029150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.851582 (+822432us) service_pool.cc:224] Handling call
0709 13:05:41.851942 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.065355 (+213413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.105420 (+ 40065us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:42.126120 13866 rpcz_store.cc:276] 0709 13:05:40.961239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.699892 (+738653us) service_pool.cc:224] Handling call
0709 13:05:41.700267 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.038288 (+338021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.086977 (+ 48689us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54003acfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:42.087227 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768588032}
W20260709 13:05:42.170605 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.171980 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 223866us
W20260709 13:05:42.173527 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 218861us
I20260709 13:05:42.163573 14018 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:42.186808 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1809) took 1011 ms. Trace:
I20260709 13:05:42.187426 14018 rpcz_store.cc:276] 0709 13:05:41.175681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.163098 (+987417us) service_pool.cc:224] Handling call
0709 13:05:42.163388 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.186347 (+ 22959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.186723 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:42.096320 17812 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:42.168624 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:42.208870 14018 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:42.132292 13915 connection.cc:256] RPC call timeout handler was delayed by -2.151099457999976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
W20260709 13:05:42.212895 13915 connection.cc:256] RPC call timeout handler was delayed by -2.1456994579999673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
I20260709 13:05:42.173156 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:42.199476 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:42.201597 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:42.098376 17541 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:42.203517 14555 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:42.105047 17746 consensus_queue.cc:1059] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:05:42.209275 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:42.214867 13875 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:42.210669 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:42.146070 17699 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:42.218580 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 45428.2us
I20260709 13:05:42.220705 13869 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
W20260709 13:05:42.213681 13915 connection.cc:256] RPC call timeout handler was delayed by -2.1096994579999091s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18330000000000002s for a response
I20260709 13:05:42.222293 14018 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:05:42.225463 14021 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:05:42.230829 17718 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:42.240362 14554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 745) took 1141 ms. Trace:
W20260709 13:05:42.244753 13915 connection.cc:256] RPC call timeout handler was delayed by -2.106099458000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
W20260709 13:05:42.288139 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.218917 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1807) took 1079 ms. Trace:
W20260709 13:05:42.241596 17852 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:05:42.214546 17812 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.166612 17679 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:42.294363 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 364939us
I20260709 13:05:42.210807 17755 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:42.286669 17863 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:05:42.301391 14024 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:05:42.286818 17879 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:05:42.258080 17814 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:42.302042 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1801) took 1240 ms. Trace:
W20260709 13:05:42.305114 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.310973 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.310670 13915 connection.cc:256] RPC call timeout handler was delayed by -2.0601994579999428s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
I20260709 13:05:42.285889 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:42.310729 14023 rpcz_store.cc:276] 0709 13:05:41.139135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.142513 (+  3378us) service_pool.cc:224] Handling call
0709 13:05:42.218659 (+1076146us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b540020d0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:42.218840 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2136331520}
I20260709 13:05:42.307832 14554 rpcz_store.cc:276] 0709 13:05:41.099183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.099323 (+   140us) service_pool.cc:224] Handling call
0709 13:05:41.099594 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.067948 (+968354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.240012 (+172064us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b540040a6d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:42.240243 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2164238720}
W20260709 13:05:42.377643 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.244457 17875 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:42.415062 13868 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:42.320856 14522 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:42.311589 17344 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:42.327323 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:42.426630 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 99392.1us
W20260709 13:05:42.328209 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 153839us
I20260709 13:05:42.427886 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:42.331887 13870 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:05:42.334995 17860 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:05:42.334559 17870 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:42.282871 17626 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:05:42.335827 17707 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:42.337018 17859 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:05:42.352589 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:42.452119 13875 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
W20260709 13:05:42.353286 13915 connection.cc:256] RPC call timeout handler was delayed by -2.0088994579999735s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
W20260709 13:05:42.453079 13915 connection.cc:256] RPC call timeout handler was delayed by -1.9917994579999458s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
W20260709 13:05:42.378000 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 92193.9us
I20260709 13:05:42.308224 17620 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:42.318550 17732 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:42.377215 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1811) took 1111 ms. Trace:
I20260709 13:05:42.350992 14022 rpcz_store.cc:276] 0709 13:05:41.061906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.075670 (+ 13764us) service_pool.cc:224] Handling call
0709 13:05:41.076067 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:41.076311 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:41.571290 (+494979us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.301663 (+730373us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:42.301721 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:42.301773 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.301845 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.301960 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12168,"prepare.queue_time_us":872921,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":188988,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"thread_start_us":1153754,"threads_started":3,"wal-append.queue_time_us":604063}
I20260709 13:05:42.382135 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:42.384804 17888 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:42.385792 14552 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
W20260709 13:05:42.388010 17800 consensus_meta.cc:287] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.659s	user 0.012s	sys 0.002s
I20260709 13:05:42.324383 17812 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:42.390661 17872 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:05:42.402004 17541 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:42.395972 17880 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.396217 17746 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:42.397387 13840 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.544s (SENT)
W20260709 13:05:42.498603 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 101246us
I20260709 13:05:42.398823 17664 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:42.422431 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 101698us
W20260709 13:05:42.422809 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.429792 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:42.430245 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:42.521237 17662 consensus_meta.cc:287] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.697s	user 0.014s	sys 0.000s
I20260709 13:05:42.521759 17662 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:42.522872 13873 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:42.524920 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2512) took 1496 ms. Trace:
I20260709 13:05:42.525310 13873 rpcz_store.cc:276] 0709 13:05:41.028239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.721581 (+693342us) service_pool.cc:224] Handling call
0709 13:05:41.721913 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.523282 (+801369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.524661 (+  1379us) spinlock_profiling.cc:243] Waited 801 ms on lock 0x7b5400450954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:42.524856 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1841569024}
I20260709 13:05:42.526150 14551 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:42.526818 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:42.372929 17814 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:42.413573 17861 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
W20260709 13:05:42.420897 17868 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:05:42.454250 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:42.430449 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:42.429515 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:42.457859 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:42.474123 14019 rpcz_store.cc:276] 0709 13:05:41.264003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.206511 (+942508us) service_pool.cc:224] Handling call
0709 13:05:42.281803 (+ 75292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.282434 (+   631us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:42.298332 (+ 15898us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.375068 (+ 76736us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:42.375137 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:42.375205 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.375292 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.375476 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38005,"prepare.queue_time_us":16718,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":32009,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"thread_start_us":52640,"threads_started":1}
I20260709 13:05:42.404008 17679 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:42.479496 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2497) took 1567 ms. Trace:
I20260709 13:05:42.484416 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:42.485754 14559 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:42.427207 17785 consensus_meta.cc:287] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.223s	user 0.002s	sys 0.012s
I20260709 13:05:42.621740 13871 rpcz_store.cc:276] 0709 13:05:40.911674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.153613 (+241939us) service_pool.cc:224] Handling call
0709 13:05:41.169882 (+ 16269us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:05:41.192441 (+ 22559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.192485 (+    44us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:05:41.193579 (+  1094us) log.cc:844] Serialized 104 byte log entry
0709 13:05:42.152191 (+958612us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:42.455718 (+303527us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.456139 (+   421us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.479072 (+ 22933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.479163 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.479210 (+    47us) raft_consensus.cc:1685] finished
0709 13:05:42.479222 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.479417 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35683,"prepare.queue_time_us":488806,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":23008,"raft.run_cpu_time_us":513,"raft.run_wall_time_us":80029,"thread_start_us":852777,"threads_started":2,"wal-append.queue_time_us":1139089}
I20260709 13:05:42.623682 17785 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:42.624912 13874 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:42.487469 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:42.326030 17871 raft_consensus.cc:3055] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:42.414942 17875 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:42.491711 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:42.485550 17888 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:42.494102 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59630 (request call id 527) took 1214 ms. Trace:
W20260709 13:05:42.491215 17839 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.742s	user 0.012s	sys 0.003s
W20260709 13:05:42.496093 17838 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.507s	user 0.004s	sys 0.008s
I20260709 13:05:42.451658 17886 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:42.503667 14558 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:42.508203 14522 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:42.509884 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:42.459513 17893 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:42.528288 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1930) took 1391 ms. Trace:
I20260709 13:05:42.519343 17854 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:42.484359 17620 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:42.665946 17854 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.666452 17854 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:42.668330 17620 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:42.537052 17754 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:42.669204 17754 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:42.670398 17754 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:42.672209 17754 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:42.548979 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2507) took 1621 ms. Trace:
W20260709 13:05:42.551461 17719 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.005s
I20260709 13:05:42.487074 17874 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:05:42.674258 13868 rpcz_store.cc:276] 0709 13:05:40.927363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.698303 (+770940us) service_pool.cc:224] Handling call
0709 13:05:41.698560 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.427168 (+728608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.548685 (+121517us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54000e9a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:42.548908 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1563233024}
I20260709 13:05:42.674482 17719 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:42.555768 17747 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:42.558454 13946 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:42.675796 13946 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:42.676669 13946 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:42.677204 17719 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:42.554337 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1797) took 1493 ms. Trace:
I20260709 13:05:42.557435 17853 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:42.540537 17707 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.678344 17707 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:42.677874 14017 rpcz_store.cc:276] 0709 13:05:41.060748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.071306 (+ 10558us) service_pool.cc:224] Handling call
0709 13:05:41.071734 (+   428us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:41.101711 (+ 29977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.101760 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:41.136571 (+ 34811us) log.cc:844] Serialized 78 byte log entry
0709 13:05:41.843727 (+707156us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:42.111668 (+267941us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.325155 (+213487us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.325767 (+   612us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.326027 (+   260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.326159 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.554075 (+227916us) raft_consensus.cc:1685] finished
0709 13:05:42.554102 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.554270 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38573,"prepare.queue_time_us":264915,"prepare.run_cpu_time_us":410,"prepare.run_wall_time_us":9855,"raft.queue_time_us":243213,"raft.run_cpu_time_us":388,"raft.run_wall_time_us":12375,"spinlock_wait_cycles":2816,"thread_start_us":1041740,"threads_started":3,"wal-append.queue_time_us":961148}
I20260709 13:05:42.680208 17853 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:42.561337 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1925) took 1448 ms. Trace:
I20260709 13:05:42.545178 17662 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:42.680940 13947 rpcz_store.cc:276] 0709 13:05:41.112593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.488448 (+375855us) service_pool.cc:224] Handling call
0709 13:05:41.679431 (+190983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:41.980756 (+301325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.980805 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:42.002177 (+ 21372us) log.cc:844] Serialized 26 byte log entry
0709 13:05:42.281243 (+279066us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:42.281285 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.281371 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.561070 (+279699us) raft_consensus.cc:1685] finished
0709 13:05:42.561091 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.561267 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":867,"raft.queue_time_us":113234,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":16557,"thread_start_us":536787,"threads_started":2,"wal-append.queue_time_us":533924}
W20260709 13:05:42.682255 17861 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.051s	user 0.001s	sys 0.003s
I20260709 13:05:42.508407 17746 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:42.683233 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:42.566452 17789 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:42.683902 13947 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:42.684486 17812 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:42.527026 17718 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:42.685323 17718 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:42.503917 17880 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:42.686365 17718 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:42.636381 17699 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:42.687708 17718 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:42.688201 17880 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:42.689072 17699 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:42.578281 17814 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:42.570381 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1804) took 1504 ms. Trace:
I20260709 13:05:42.690204 14015 rpcz_store.cc:276] 0709 13:05:41.065529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.167110 (+1101581us) service_pool.cc:224] Handling call
0709 13:05:42.167464 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:42.167830 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:42.167867 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:42.199582 (+ 31715us) log.cc:844] Serialized 26 byte log entry
0709 13:05:42.456536 (+256954us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:42.494382 (+ 37846us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.495012 (+   630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.495107 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.570132 (+ 75025us) raft_consensus.cc:1685] finished
0709 13:05:42.570154 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.570306 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45944,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":69286,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":175,"thread_start_us":131164,"threads_started":1,"wal-append.queue_time_us":285057}
I20260709 13:05:42.660465 17871 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:42.672222 17817 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:42.699213 17871 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:42.700600 17871 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:42.455677 14524 connection.cc:256] RPC call timeout handler was delayed by -1.5186273250000113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
I20260709 13:05:42.623010 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:42.701821 14524 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.630s (SENT)
I20260709 13:05:42.677776 13946 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:42.702740 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1921) took 1997 ms. Trace:
I20260709 13:05:42.704142 17888 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:42.703214 13941 rpcz_store.cc:276] 0709 13:05:40.705487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.874972 (+169485us) service_pool.cc:224] Handling call
0709 13:05:40.875374 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:40.878116 (+  2742us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:41.155860 (+277744us) log.cc:844] Serialized 14 byte log entry
0709 13:05:41.842569 (+686709us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:41.847672 (+  5103us) log.cc:844] Serialized 26 byte log entry
0709 13:05:41.847886 (+   214us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:41.847931 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.848012 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.702494 (+854482us) raft_consensus.cc:1685] finished
0709 13:05:42.702517 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.702674 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1556,"prepare.queue_time_us":548572,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":55,"raft.queue_time_us":10591,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":17050,"thread_start_us":916043,"threads_started":2,"wal-append.queue_time_us":931513}
I20260709 13:05:42.705389 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:42.656141 13870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2489) took 1750 ms (client timeout 2051 ms). Trace:
W20260709 13:05:42.590320 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.570412 17541 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.661405 13875 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:42.661237 13942 rpcz_store.cc:276] 0709 13:05:41.137321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.580007 (+442686us) service_pool.cc:224] Handling call
0709 13:05:41.674498 (+ 94491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:41.978513 (+304015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.527995 (+549482us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b54003309d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:42.528220 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":653398784}
W20260709 13:05:42.627238 13874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 571) took 1618 ms (client timeout 2004 ms). Trace:
W20260709 13:05:42.758203 13870 rpcz_store.cc:269] 0709 13:05:40.905340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.905468 (+   128us) service_pool.cc:224] Handling call
0709 13:05:42.655894 (+1750426us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b54002e5ed4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:42.656054 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2128867200}
W20260709 13:05:42.762941 13874 rpcz_store.cc:269] 0709 13:05:41.008429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.022526 (+ 14097us) service_pool.cc:224] Handling call
0709 13:05:42.627031 (+1604505us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54003ac0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:42.627175 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3626839808}
I20260709 13:05:42.629458 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:42.578431 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:42.660734 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 206579us
I20260709 13:05:42.627381 14555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 526) took 1384 ms. Trace:
W20260709 13:05:42.765265 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 311649us
I20260709 13:05:42.664535 17841 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.765457 13987 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:42.766036 17841 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:42.767540 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:42.768502 17841 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:42.665186 17661 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:05:42.683475 17903 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.053s	user 0.004s	sys 0.000s
I20260709 13:05:42.769232 17661 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.769703 17661 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:42.772747 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.684948 17746 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:42.774823 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1934) took 1098 ms. Trace:
W20260709 13:05:42.776101 13915 connection.cc:256] RPC call timeout handler was delayed by -1.9548994579999999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2005s for a response
I20260709 13:05:42.775353 13944 rpcz_store.cc:276] 0709 13:05:41.676198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.694406 (+ 18208us) service_pool.cc:224] Handling call
0709 13:05:41.694794 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:42.121390 (+426596us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:42.121436 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:42.122139 (+   703us) log.cc:844] Serialized 52 byte log entry
0709 13:05:42.600373 (+478234us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:42.600428 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.600540 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.774491 (+173951us) raft_consensus.cc:1685] finished
0709 13:05:42.774522 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.774738 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127760,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":7370,"thread_start_us":817641,"threads_started":2,"wal-append.queue_time_us":454691}
W20260709 13:05:42.776661 13915 connection.cc:256] RPC call timeout handler was delayed by -1.9539994579999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2006s for a response
W20260709 13:05:42.777107 13915 connection.cc:256] RPC call timeout handler was delayed by -1.9089994579999257s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
W20260709 13:05:42.777472 13915 connection.cc:256] RPC call timeout handler was delayed by -1.8612994579999622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2109s for a response
I20260709 13:05:42.685906 17789 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:42.567863 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:42.573909 13873 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
W20260709 13:05:42.711561 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.706007 13941 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:42.779846 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:42.780448 13945 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:42.780580 13941 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:42.674861 17874 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:42.782634 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:42.782699 17874 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:42.783413 13941 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:42.784127 17874 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:42.777969 13915 connection.cc:256] RPC call timeout handler was delayed by -1.8171994580000046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
W20260709 13:05:42.786201 13915 connection.cc:256] RPC call timeout handler was delayed by -1.8072852709999552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 13:05:42.786547 13915 connection.cc:256] RPC call timeout handler was delayed by -1.7766994579999391s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
W20260709 13:05:42.787058 13915 connection.cc:256] RPC call timeout handler was delayed by -1.7514994580000121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2231s for a response
I20260709 13:05:42.791906 17626 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:05:42.624460 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 166677us
I20260709 13:05:42.759033 13948 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:42.759588 14561 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:42.795147 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:42.765609 14555 rpcz_store.cc:276] 0709 13:05:41.243068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.309416 (+ 66348us) service_pool.cc:224] Handling call
0709 13:05:41.309830 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.241294 (+931464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.627152 (+385858us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b54003d72d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:42.627320 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2054742912}
I20260709 13:05:42.766460 17893 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:42.796634 17789 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:42.796069 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:42.707010 17875 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:42.800156 17741 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:05:42.743557 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 235438us
I20260709 13:05:42.761243 13946 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:05:42.801247 13946 pending_rounds.cc:85] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 4
I20260709 13:05:42.801532 13946 pending_rounds.cc:107] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:05:42.801776 13946 raft_consensus.cc:2889] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:42.803906 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.805001 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.797139 13989 leader_election.cc:400] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9.
I20260709 13:05:42.777711 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:42.810572 14022 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:42.777526 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:42.814463 13944 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:42.815858 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:42.817101 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.785946 17874 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:42.822557 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.823767 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.824904 13914 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:42.819660 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:42.826514 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:42.827015 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.827380 13942 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:42.835949 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:42.787577 13915 connection.cc:256] RPC call timeout handler was delayed by -1.6139278240000294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20910000000000004s for a response
W20260709 13:05:42.844453 13915 connection.cc:256] RPC call timeout handler was delayed by -1.5822994579999659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2419s for a response
I20260709 13:05:42.625946 17785 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:42.845782 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 100412us
W20260709 13:05:42.855031 14524 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.876s (SENT)
W20260709 13:05:42.849381 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.850378 13915 connection.cc:256] RPC call timeout handler was delayed by -1.414126630999931s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
I20260709 13:05:42.708312 17707 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:42.660912 14552 rpcz_store.cc:276] 0709 13:05:41.276051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.305039 (+ 28988us) service_pool.cc:224] Handling call
0709 13:05:42.490327 (+1185288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3328}
I20260709 13:05:42.627713 17890 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:05:42.847855 17694 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
W20260709 13:05:42.870905 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.877802 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.881594 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:42.885632 17789 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:42.886865 17812 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:42.890431 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1803) took 1825 ms. Trace:
I20260709 13:05:42.845682 17626 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:05:42.823541 14554 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:05:42.924316 14022 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
W20260709 13:05:42.893424 13839 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.024s	sys 0.037s
W20260709 13:05:42.926327 13839 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.024s	sys 0.037s
W20260709 13:05:42.926733 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 553308us
I20260709 13:05:42.895989 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2476) took 1885 ms. Trace:
I20260709 13:05:42.853156 17746 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.928853 13948 rpcz_store.cc:276] 0709 13:05:41.010883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.061729 (+ 50846us) service_pool.cc:224] Handling call
0709 13:05:42.713223 (+1651494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.795941 (+ 82718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.895643 (+ 99702us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54003b01d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:42.895889 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3773100288}
W20260709 13:05:42.856302 13915 connection.cc:256] RPC call timeout handler was delayed by -1.3671994579999591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2658s for a response
I20260709 13:05:42.914953 17841 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:42.920082 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 100439us
W20260709 13:05:42.936697 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.822566 17853 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:42.854525 17705 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:42.706854 17839 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:42.950798 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 154797us
I20260709 13:05:42.742687 17838 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:42.954691 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.826184 17541 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:42.933619 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.936887 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:42.921979 14020 rpcz_store.cc:276] 0709 13:05:41.065243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.150459 (+ 85216us) service_pool.cc:224] Handling call
0709 13:05:41.165440 (+ 14981us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:05:41.174751 (+  9311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.174784 (+    33us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:05:41.196636 (+ 21852us) log.cc:844] Serialized 104 byte log entry
0709 13:05:41.996377 (+799741us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:42.031662 (+ 35285us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.032077 (+   415us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.032359 (+   282us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.236009 (+203650us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b180034a740. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:42.296833 (+ 60824us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.323426 (+ 26593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.323584 (+   158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.538319 (+214735us) spinlock_profiling.cc:243] Waited 4.04 ms on lock 0x7b180034a740. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:42.565449 (+ 27130us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b180034a740. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1
0709 13:05:42.566180 (+   731us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b180034a740. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:42.890140 (+323960us) raft_consensus.cc:1685] finished
0709 13:05:42.890167 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.890345 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88851,"prepare.queue_time_us":581863,"prepare.run_cpu_time_us":163,"prepare.run_wall_time_us":4469,"raft.queue_time_us":516394,"raft.run_cpu_time_us":812,"raft.run_wall_time_us":56355,"spinlock_wait_cycles":147921792,"thread_start_us":890424,"threads_started":2,"wal-append.queue_time_us":939791}
I20260709 13:05:42.938254 13947 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:42.941118 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:42.942754 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 87775.2us
I20260709 13:05:42.943745 17841 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:42.968804 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:42.952205 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:42.908661 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:42.825393 17820 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:05:42.940402 17705 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:42.977432 13949 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:42.963857 13948 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:42.963848 14524 leader_election.cc:336] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.098s (SENT)
I20260709 13:05:42.940478 17853 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.970331 13943 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:42.972589 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:42.962600 13838 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:42.958794 13945 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:42.975217 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:42.933326 13915 connection.cc:256] RPC call timeout handler was delayed by -1.3347994579999067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26940000000000003s for a response
W20260709 13:05:42.983804 13915 connection.cc:256] RPC call timeout handler was delayed by -1.1920512940000663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
W20260709 13:05:42.984386 13915 connection.cc:256] RPC call timeout handler was delayed by -1.139690244999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
W20260709 13:05:42.984844 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0454523130000553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
W20260709 13:05:42.985307 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0454523130000553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
I20260709 13:05:42.986325 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:42.934948 17746 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.027933 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.034215 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.035255 13945 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:42.973209 17890 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:42.971673 17831 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:42.977656 17679 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:42.987118 17853 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:43.065636 13944 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
W20260709 13:05:43.062523 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 87398.3us
I20260709 13:05:43.074707 13948 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:43.075176 17719 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:43.076014 17626 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:43.078720 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.080363 13942 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:43.061909 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:43.026624 17741 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:42.986637 17705 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:43.034940 17820 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:43.098839 17719 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:43.085256 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.108474 13942 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:05:43.105480 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 119415us
W20260709 13:05:43.086467 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 113949us
W20260709 13:05:43.087098 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.099455 17820 raft_consensus.cc:697] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:42.971866 17699 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.092s
I20260709 13:05:43.080248 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:43.105752 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:43.083137 17785 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:43.138770 13947 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:05:43.142163 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:43.132337 13838 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:43.089685 17741 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:42.987906 17814 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.101840 17831 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:43.111820 17893 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:43.140075 17719 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:43.149416 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:43.140070 17620 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:05:43.162745 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 82643.9us
I20260709 13:05:43.179000 13840 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:43.189081 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:43.161903 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:43.196908 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:43.197422 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.197695 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:43.199424 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:43.205286 14017 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:43.176100 17825 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:43.167654 17907 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:05:43.210357 14019 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:43.134509 17896 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:05:43.166014 13838 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:43.168458 17679 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:43.210011 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:43.180574 17873 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:43.190299 13947 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
W20260709 13:05:43.167865 17908 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 13:05:43.211138 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52745.7us
I20260709 13:05:43.177601 17620 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:43.214859 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.170648 17741 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.154109 17890 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:05:43.219825 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:43.221356 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:43.221675 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:43.247913 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:43.248669 14019 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:43.204437 17707 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:43.248981 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:43.252101 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 96134.8us
I20260709 13:05:43.258459 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:43.260115 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:43.260241 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:43.169405 17820 consensus_queue.cc:237] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.223496 17732 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:43.266357 17732 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.266763 17732 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:43.266803 17874 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:43.270252 14020 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:05:43.272094 17854 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.605s	user 0.009s	sys 0.001s
I20260709 13:05:43.272614 17854 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.274232 17841 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:43.218292 17728 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:43.275952 17728 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:43.276484 17728 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.245513 17620 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:43.262777 17746 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:43.222970 17896 consensus_queue.cc:1243] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:05:43.289085 17746 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:43.303791 17746 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:43.288496 17732 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.300788 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:43.304308 17918 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:05:43.305716 17814 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:43.304312 17746 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:43.308275 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 61362.1us
I20260709 13:05:43.310977 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:43.314289 17662 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:43.314848 17662 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:43.315734 17662 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.316919 17781 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:43.317764 17626 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:05:43.319036 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.320022 17679 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:43.320609 17679 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.322393 17719 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.322865 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1935) took 1646 ms. Trace:
I20260709 13:05:43.323511 13949 rpcz_store.cc:276] 0709 13:05:41.676575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.715252 (+ 38677us) service_pool.cc:224] Handling call
0709 13:05:41.715774 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.990427 (+1274653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.322550 (+332123us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400090954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:43.322786 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2892281600}
W20260709 13:05:43.327157 17875 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.004s
I20260709 13:05:43.327821 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1937) took 1372 ms. Trace:
I20260709 13:05:43.327653 17875 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.328202 13943 rpcz_store.cc:276] 0709 13:05:41.955376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.955587 (+   211us) service_pool.cc:224] Handling call
0709 13:05:41.963005 (+  7418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.986839 (+1023834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.327524 (+340685us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400053a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:43.327750 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2316088192}
I20260709 13:05:43.330394 17915 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:43.330646 17913 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 13:05:43.331173 13941 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:43.331038 17915 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.333308 17915 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:43.322813 17679 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:43.247393 17825 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:43.352550 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 198601us
W20260709 13:05:43.285962 17661 consensus_meta.cc:287] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 13:05:43.329818 17875 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:43.341866 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:43.338303 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 76601.3us
I20260709 13:05:43.247301 17726 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:43.354784 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.258525 17785 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:05:43.298686 17871 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.598s	user 0.012s	sys 0.017s
W20260709 13:05:43.363221 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 231152us
W20260709 13:05:43.353407 17817 consensus_meta.cc:287] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.654s	user 0.008s	sys 0.016s
I20260709 13:05:43.371796 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:43.378544 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.308406 17839 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:43.381592 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:43.361557 17718 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.673s	user 0.015s	sys 0.009s
I20260709 13:05:43.383474 13941 raft_consensus.cc:2410] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:43.385694 17785 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:43.387423 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:43.392750 13989 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:43.394248 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:43.384783 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:43.395794 13947 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:43.396179 14016 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:05:43.396271 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:43.397432 13941 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:43.397018 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.399557 13947 raft_consensus.cc:2393] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:43.396579 14016 pending_rounds.cc:85] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 5
I20260709 13:05:43.400319 14016 pending_rounds.cc:107] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:05:43.400705 14016 raft_consensus.cc:2889] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:43.401212 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.404790 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.380402 17712 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.380277 17831 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:43.405447 13941 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:43.376461 17726 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:43.392118 17725 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:43.419526 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:43.421506 13941 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:43.367440 17853 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.393877 17789 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:43.402595 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:43.426558 13941 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:05:43.374104 17661 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.404016 17886 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.404734 17871 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:05:43.413400 17912 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.624s	user 0.000s	sys 0.005s
I20260709 13:05:43.387629 17694 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:05:43.395289 17814 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:05:43.413805 17831 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:43.450616 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 261620us
I20260709 13:05:43.416193 17839 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:43.414654 17712 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.410813 17728 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:43.399802 17817 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:05:43.425792 17726 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:43.447240 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:43.451769 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 139233us
W20260709 13:05:43.496070 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 49535.8us
I20260709 13:05:43.496308 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:43.497596 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:43.458508 13947 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:05:43.455406 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:43.464115 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:43.488283 14017 raft_consensus.cc:2368] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10.
I20260709 13:05:43.490530 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:43.438442 17725 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:43.500538 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.450951 17789 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.512166 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.512367 14524 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:43.503298 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:43.459139 17831 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:43.418202 17785 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.442004 17705 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.513908 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:43.525631 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.460232 17919 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:43.459590 17839 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:43.525827 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 123367us
I20260709 13:05:43.551115 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:43.494624 17915 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.555222 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:43.557693 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.554150 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:43.564390 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:43.565300 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:43.568285 14558 raft_consensus.cc:2376] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 10. Re-sending same reply.
I20260709 13:05:43.569631 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:43.570387 14558 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:43.511462 17871 raft_consensus.cc:2764] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:05:43.425396 17825 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.563678 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:43.508610 17920 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:43.546988 17754 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.874s	user 0.011s	sys 0.015s
I20260709 13:05:43.509213 17781 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:43.590981 14560 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:43.529536 17874 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:43.592479 13946 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:43.549810 17896 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:43.589811 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:43.598775 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:43.615674 13874 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:43.591563 14561 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:43.572069 17699 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:43.602898 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:43.625186 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:43.627751 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 172429us
I20260709 13:05:43.634753 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:43.635756 14021 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:43.636714 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:43.637346 14021 raft_consensus.cc:2376] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 8. Re-sending same reply.
I20260709 13:05:43.606366 17925 consensus_queue.cc:1059] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:43.607137 17924 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:43.549360 17828 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:43.608383 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:43.543895 17934 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:43.594269 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:43.611447 14023 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:05:43.566720 17839 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:43.449149 17886 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.600598 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:43.562136 17875 raft_consensus.cc:2764] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 13:05:43.551694 17919 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:43.547322 17921 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:43.540197 17923 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:43.539512 17789 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.640349 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:43.641016 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.641824 14022 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:43.642424 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:43.596386 17920 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:43.590915 17915 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.653497 14551 raft_consensus.cc:2368] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8.
I20260709 13:05:43.668905 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:43.676846 13839 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:43.610070 17694 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:43.654896 14023 pending_rounds.cc:85] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 4
I20260709 13:05:43.685325 14023 pending_rounds.cc:107] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:05:43.685678 14023 raft_consensus.cc:2889] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:43.637702 17541 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:43.619050 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:43.603966 17883 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:05:43.653213 17934 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.686705 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:43.695792 13947 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:43.687964 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:43.706969 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:43.652037 17699 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:43.619710 17814 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:05:43.656080 17817 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:43.650292 17626 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:43.715031 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:43.661741 17919 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:43.640259 17620 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.660089 17662 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.683960 17875 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:43.729552 14556 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:43.682533 17921 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:43.685251 17694 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:43.666076 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 111992us
I20260709 13:05:43.666992 17679 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:05:43.657353 17935 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:05:43.682807 17920 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:43.743309 14552 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:43.744037 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:43.710256 17934 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:43.693967 17728 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:05:43.751541 13941 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:43.729898 17817 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:43.765339 14021 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:43.768359 17948 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:43.768977 17950 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:43.770996 17820 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:43.746896 17747 raft_consensus.cc:3055] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:05:43.759650 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 82914.4us
I20260709 13:05:43.779294 14022 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:43.659643 17725 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.744163 17694 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:43.741353 17853 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:43.740348 17732 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:43.782567 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:43.726308 17541 consensus_queue.cc:1059] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:05:43.747130 17661 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:43.750211 17925 raft_consensus.cc:3055] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:43.751017 17712 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:43.805284 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:43.806604 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:43.801028 17747 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:43.812224 13947 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:05:43.751446 17828 consensus_queue.cc:1059] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:05:43.788538 17817 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:43.796000 17820 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:43.831808 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 672) took 1022 ms. Trace:
I20260709 13:05:43.887404 13946 rpcz_store.cc:276] 0709 13:05:42.809882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.908268 (+ 98386us) service_pool.cc:224] Handling call
0709 13:05:43.831621 (+923353us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b540004c254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:43.831743 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426287872}
I20260709 13:05:43.824461 17936 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:43.826854 13873 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:43.842100 13874 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:43.741196 17662 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.855577 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:43.856232 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:43.857736 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:43.918792 13867 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:43.855779 17925 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:43.862161 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:43.868448 17705 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:43.864456 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:43.865950 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:43.832082 17814 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
W20260709 13:05:43.866918 17928 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:05:43.807332 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:43.918290 17922 log.cc:927] Time spent T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.371s	user 0.001s	sys 0.001s
I20260709 13:05:43.740733 17626 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.956470 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:43.871942 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:43.958272 13947 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:43.959239 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:43.959438 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1955) took 1165 ms. Trace:
I20260709 13:05:43.815654 14021 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:43.808527 17948 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:43.889269 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:43.963491 13946 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:43.960635 13948 rpcz_store.cc:276] 0709 13:05:42.793622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.102266 (+308644us) service_pool.cc:224] Handling call
0709 13:05:43.139500 (+ 37234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.139793 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:43.155495 (+ 15702us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.371603 (+216108us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b18004355e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287 00007fb2c344e208
0709 13:05:43.659892 (+288289us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.660388 (+   496us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.939507 (+279119us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.940312 (+   805us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:43.940365 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:43.940425 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.958989 (+ 18564us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.959300 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12124,"prepare.queue_time_us":493679,"prepare.run_cpu_time_us":189,"prepare.run_wall_time_us":4771,"raft.queue_time_us":105624,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":53456,"spinlock_wait_cycles":22196352,"thread_start_us":546542,"threads_started":3}
I20260709 13:05:43.889523 14522 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:43.853777 17950 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:05:43.888797 17929 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 13:05:43.863823 17541 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:43.834982 17699 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:44.039182 13761 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.307s	user 0.007s	sys 0.003s
W20260709 13:05:44.019071 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 475027us
I20260709 13:05:43.744910 17875 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:43.852820 17754 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:05:43.952987 17942 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:43.917863 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:43.839107 13866 raft_consensus.cc:2410] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:05:43.955763 13945 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.013s	sys 0.021s
W20260709 13:05:43.966933 14015 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.009s	sys 0.008s
W20260709 13:05:44.083384 13945 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.013s	sys 0.021s
W20260709 13:05:44.083509 14015 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.010s	sys 0.008s
W20260709 13:05:43.967880 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 78458.1us
I20260709 13:05:43.980723 13946 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:44.085225 13869 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 11 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:43.986919 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:43.989706 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.995764 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:43.931092 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.951696 17873 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:05:44.062476 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 196612us
I20260709 13:05:44.090025 14022 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:44.091543 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:44.096287 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:43.952540 17935 consensus_queue.cc:1059] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.105s
W20260709 13:05:44.074370 17903 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:05:44.100716 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:44.109776 13989 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:44.110185 14021 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:44.103581 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:43.969123 17950 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:44.110826 13869 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:44.099814 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2483) took 1319 ms. Trace:
I20260709 13:05:43.963857 17948 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:44.120180 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1959) took 1015 ms. Trace:
I20260709 13:05:43.925331 17936 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:43.997097 17541 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:44.045053 17915 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:43.998165 17712 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:44.126309 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:44.075332 17957 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:05:44.077883 17789 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:44.131498 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:44.170104 14022 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:44.129871 13944 rpcz_store.cc:276] 0709 13:05:42.780656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.100176 (+319520us) service_pool.cc:224] Handling call
0709 13:05:43.100590 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.108375 (+  7785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.108456 (+    81us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:43.109455 (+   999us) log.cc:844] Serialized 26 byte log entry
0709 13:05:43.360186 (+250731us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:43.360245 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.360350 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.099574 (+739224us) raft_consensus.cc:1685] finished
0709 13:05:44.099596 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.099749 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5285,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":25802,"thread_start_us":200570,"threads_started":1,"wal-append.queue_time_us":317857}
I20260709 13:05:44.046492 17937 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:44.108880 17933 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
I20260709 13:05:44.104519 17924 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:05:44.164397 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.106433 17620 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:44.172777 13945 rpcz_store.cc:276] 0709 13:05:43.104467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.326627 (+222160us) service_pool.cc:224] Handling call
0709 13:05:43.326971 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.327195 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:44.083888 (+756693us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.119767 (+ 35879us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.119843 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:44.119907 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.119992 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.120112 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37392,"prepare.queue_time_us":413671,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":13683,"raft.queue_time_us":115,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"thread_start_us":753601,"threads_started":2}
I20260709 13:05:44.175988 17950 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.125731 17945 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.183826 13944 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:44.187856 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:44.321398 13866 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:43.938068 17921 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:44.190390 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:44.073369 17925 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.164348 17712 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:44.197975 17944 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 13:05:44.186493 17789 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:44.164495 17926 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.150s
I20260709 13:05:43.996315 17820 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.159132 17951 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:44.313161 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:44.191816 17785 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:44.359818 17875 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:44.342810 17712 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:44.363240 17741 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:44.367192 17626 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:44.368026 17961 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:05:44.368132 17951 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.372382 17946 raft_consensus.cc:3055] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:44.372869 17946 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:44.090219 17942 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.377339 17886 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:44.379454 17942 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:44.381573 13946 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:44.259631 17831 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.715s	user 0.011s	sys 0.014s
I20260709 13:05:44.382643 17831 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.385183 17831 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:44.385356 17838 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:44.370785 17951 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:44.198731 17947 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:05:44.344017 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1956) took 1412 ms. Trace:
I20260709 13:05:44.397702 13942 rpcz_store.cc:276] 0709 13:05:42.931853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.138919 (+207066us) service_pool.cc:224] Handling call
0709 13:05:43.139298 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.360421 (+221123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.360452 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:43.362336 (+  1884us) log.cc:844] Serialized 26 byte log entry
0709 13:05:43.514699 (+152363us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:43.514744 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.514821 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.343727 (+828906us) raft_consensus.cc:1685] finished
0709 13:05:44.343755 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.343934 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5140,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":5997,"thread_start_us":320321,"threads_started":2,"wal-append.queue_time_us":294424}
I20260709 13:05:44.311019 17541 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:44.401540 17541 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:44.406230 17973 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
W20260709 13:05:44.333333 17781 consensus_meta.cc:287] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.721s	user 0.010s	sys 0.004s
I20260709 13:05:44.315670 17915 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:44.389462 17746 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.083s	user 0.013s	sys 0.000s
I20260709 13:05:44.390864 17893 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:44.391955 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:44.395164 17888 consensus_meta.cc:287] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.690s	user 0.020s	sys 0.008s
I20260709 13:05:44.335834 17945 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.321219 17828 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.326469 17662 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:44.341094 17927 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.208s	user 0.008s	sys 0.001s
I20260709 13:05:44.429553 17926 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:44.430265 17781 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:44.407404 17785 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:44.192631 17937 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:44.450457 17893 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:44.457695 17888 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:44.482568 17888 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:44.485028 17839 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.820s	user 0.011s	sys 0.002s
I20260709 13:05:44.445045 17746 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.489141 17746 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:44.493278 17854 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:44.465711 17828 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.496835 17828 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:44.484874 17924 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.498098 17924 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:44.499678 17896 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:44.500396 17896 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:44.485882 17839 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:44.504037 17950 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:44.497498 17931 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.750s	user 0.001s	sys 0.000s
I20260709 13:05:44.373786 17946 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.444247 17915 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.469564 17926 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:44.529834 17936 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:44.538312 17926 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:44.541066 17962 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:05:44.544600 17945 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:44.545838 17934 consensus_meta.cc:287] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.749s	user 0.012s	sys 0.000s
I20260709 13:05:44.546567 17946 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:44.548508 17919 consensus_meta.cc:287] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.807s	user 0.001s	sys 0.016s
I20260709 13:05:44.547031 17937 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:44.235685 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 137232us
I20260709 13:05:44.622419 13838 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:44.630971 13873 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.026s	sys 0.034s
W20260709 13:05:44.631731 13873 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.026s	sys 0.034s
I20260709 13:05:44.635223 13872 raft_consensus.cc:2410] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:05:44.640121 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.325843 13871 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:44.646412 13875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.024s	sys 0.016s
W20260709 13:05:44.647148 13875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.024s	sys 0.016s
W20260709 13:05:44.647440 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.650286 13948 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:44.650535 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1960) took 1516 ms. Trace:
W20260709 13:05:44.653316 13987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.002s	sys 0.002s
W20260709 13:05:44.653820 13987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.002s	sys 0.002s
I20260709 13:05:44.651487 13943 rpcz_store.cc:276] 0709 13:05:43.134104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.329158 (+195054us) service_pool.cc:224] Handling call
0709 13:05:43.329565 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.329943 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.329981 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:43.353712 (+ 23731us) log.cc:844] Serialized 26 byte log entry
0709 13:05:43.498588 (+144876us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:43.498636 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.498720 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.496693 (+997973us) spinlock_profiling.cc:243] Waited 35.9 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:44.649410 (+152717us) raft_consensus.cc:1685] finished
0709 13:05:44.649485 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.650113 (+   628us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":962,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":82592512,"thread_start_us":104563,"threads_started":1,"wal-append.queue_time_us":308126}
W20260709 13:05:44.635358 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.426260 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2531) took 1164 ms. Trace:
I20260709 13:05:44.667414 14019 raft_consensus.cc:2393] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:44.668176 14024 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.001s	sys 0.009s
W20260709 13:05:44.668191 14020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.000s	sys 0.004s
I20260709 13:05:44.671934 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:44.675103 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1834) took 1477 ms. Trace:
I20260709 13:05:44.677510 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:44.678961 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:44.703491 13944 raft_consensus.cc:2410] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:05:44.703238 13875 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:44.709268 17915 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:44.710323 17926 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:44.710722 17817 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.795s	user 0.010s	sys 0.003s
W20260709 13:05:44.721709 14020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.000s	sys 0.004s
I20260709 13:05:44.723163 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:44.725042 14024 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.002s	sys 0.009s
I20260709 13:05:44.729076 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1836) took 1099 ms. Trace:
W20260709 13:05:44.759382 17825 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.075s	sys 0.120s
I20260709 13:05:44.759682 17984 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:44.760886 17984 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.762336 17662 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:44.702250 14018 rpcz_store.cc:276] 0709 13:05:43.197649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.198143 (+   494us) service_pool.cc:224] Handling call
0709 13:05:43.198499 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.198882 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:43.206201 (+  7319us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.601390 (+395189us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:43.601982 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:05:43.602156 (+   174us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:43.602977 (+   821us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.842149 (+239172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.842242 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.483720 (+641478us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b18002d1660. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:44.674388 (+190668us) raft_consensus.cc:1685] finished
0709 13:05:44.674456 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.674795 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50241,"prepare.queue_time_us":914571,"prepare.run_cpu_time_us":281,"prepare.run_wall_time_us":195751,"raft.queue_time_us":139000,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":24208,"spinlock_wait_cycles":28243968,"thread_start_us":526234,"threads_started":3,"wal-append.queue_time_us":772520}
I20260709 13:05:44.766417 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:44.768097 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:44.769488 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:44.547323 17934 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:44.770819 13989 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.011s	sys 0.008s
W20260709 13:05:44.771528 13989 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.011s	sys 0.008s
I20260709 13:05:44.773272 17934 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:44.760143 17825 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.076s	sys 0.120s
I20260709 13:05:44.776194 13945 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:44.777946 17839 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:44.781087 17920 consensus_meta.cc:287] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.993s	user 0.015s	sys 0.009s
I20260709 13:05:44.549177 17919 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:44.789345 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 677) took 1428 ms. Trace:
I20260709 13:05:44.790908 14021 raft_consensus.cc:2368] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 7. Current term is 8.
I20260709 13:05:44.792017 17984 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:44.794776 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 116770us
W20260709 13:05:44.795729 14017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2530) took 1535 ms (client timeout 1686 ms). Trace:
I20260709 13:05:44.795827 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1793) took 3736 ms. Trace:
W20260709 13:05:44.796217 14017 rpcz_store.cc:269] 0709 13:05:43.259753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.259891 (+   138us) service_pool.cc:224] Handling call
0709 13:05:44.795162 (+1535271us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400020dd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:44.795463 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468128000}
I20260709 13:05:44.796368 14016 rpcz_store.cc:276] 0709 13:05:41.059513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.069811 (+ 10298us) service_pool.cc:224] Handling call
0709 13:05:41.071734 (+  1923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.401324 (+2329590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.795139 (+1393815us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b5400412154. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:44.795572 (+   433us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5337251968}
I20260709 13:05:44.799078 14017 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:44.799432 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1832) took 1602 ms. Trace:
I20260709 13:05:44.800308 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 455) took 1134 ms. Trace:
I20260709 13:05:44.800938 14017 rpcz_store.cc:276] 0709 13:05:43.665764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.797922 (+1132158us) service_pool.cc:224] Handling call
0709 13:05:44.798615 (+   693us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.799992 (+  1377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.800226 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:44.803539 14017 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:44.806437 14554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 752) took 1403 ms. Trace:
I20260709 13:05:44.807156 17924 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.807973 17785 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.810127 13947 rpcz_store.cc:276] 0709 13:05:43.360794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.961313 (+600519us) service_pool.cc:224] Handling call
0709 13:05:43.961584 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.105104 (+143520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.105153 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.105221 (+    68us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:44.105271 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.788679 (+683408us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540004bfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:44.788777 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.789049 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329384832}
I20260709 13:05:44.789489 17919 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:44.800067 14023 rpcz_store.cc:276] 0709 13:05:43.197043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.199143 (+  2100us) service_pool.cc:224] Handling call
0709 13:05:43.210887 (+ 11744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.686426 (+475539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.798785 (+1112359us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54003bb0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:44.799158 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":920063744}
I20260709 13:05:44.712018 17817 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.817268 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 461) took 1067 ms. Trace:
I20260709 13:05:44.796105 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:44.809856 17924 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:44.818420 14017 rpcz_store.cc:276] 0709 13:05:43.744658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.802550 (+1057892us) service_pool.cc:224] Handling call
0709 13:05:44.803176 (+   626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.811858 (+  8682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.812289 (+   431us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:44.820644 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 460) took 1092 ms. Trace:
I20260709 13:05:44.806947 14554 rpcz_store.cc:276] 0709 13:05:43.402492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.402720 (+   228us) service_pool.cc:224] Handling call
0709 13:05:43.403186 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.523465 (+120279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.523516 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:43.524164 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 13:05:43.682531 (+158367us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:43.682588 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.805441 (+1122853us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54005622d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:44.805538 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.805663 (+   125us) raft_consensus.cc:1685] finished
0709 13:05:44.805684 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.806166 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2333,"raft.queue_time_us":1637,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":9132,"spinlock_wait_cycles":248573824,"thread_start_us":82400,"threads_started":1,"wal-append.queue_time_us":279873}
I20260709 13:05:44.781750 17920 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.827284 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1849) took 1188 ms. Trace:
I20260709 13:05:44.827471 17841 raft_consensus.cc:3055] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:44.827883 14016 rpcz_store.cc:276] 0709 13:05:43.639023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.797859 (+1158836us) service_pool.cc:224] Handling call
0709 13:05:44.816644 (+ 18785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.826730 (+ 10086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.826775 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.826870 (+    95us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:44.826917 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.827007 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.827177 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:44.827984 17841 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:44.824183 14021 rpcz_store.cc:276] 0709 13:05:43.724864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.126009 (+401145us) service_pool.cc:224] Handling call
0709 13:05:44.793958 (+667949us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b54003bbfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:44.816881 (+ 22923us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1403614080}
I20260709 13:05:44.834967 17814 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:44.842510 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:44.738984 14024 rpcz_store.cc:276] 0709 13:05:43.629822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.630025 (+   203us) service_pool.cc:224] Handling call
0709 13:05:43.630623 (+   598us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:43.736786 (+106163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.736841 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:43.754458 (+ 17617us) log.cc:844] Serialized 52 byte log entry
0709 13:05:43.969274 (+214816us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:44.125213 (+155939us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.728434 (+603221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.728617 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.728686 (+    69us) raft_consensus.cc:1685] finished
0709 13:05:44.728706 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.729001 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":229124,"prepare.run_cpu_time_us":8518,"prepare.run_wall_time_us":115552,"raft.queue_time_us":169769,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":34244,"thread_start_us":935699,"threads_started":4,"wal-append.queue_time_us":364229}
I20260709 13:05:44.827920 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:44.853546 13870 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:44.844863 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1837) took 1214 ms. Trace:
I20260709 13:05:44.847985 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:44.860677 13869 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.729s	user 0.013s	sys 0.008s
I20260709 13:05:44.861456 13869 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
W20260709 13:05:44.862201 13866 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.510s	user 0.014s	sys 0.002s
I20260709 13:05:44.862751 13866 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:44.867640 17920 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:44.870313 13871 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
W20260709 13:05:44.872224 13867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.011s	sys 0.020s
W20260709 13:05:44.872798 14019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2529) took 1613 ms (client timeout 1597 ms). Trace:
I20260709 13:05:44.876742 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:44.879007 14023 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:44.747051 14015 rpcz_store.cc:276] 0709 13:05:43.261617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.261811 (+   194us) service_pool.cc:224] Handling call
0709 13:05:43.262267 (+   456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:43.294010 (+ 31743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.294089 (+    79us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:43.303203 (+  9114us) log.cc:844] Serialized 52 byte log entry
0709 13:05:44.084228 (+781025us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:44.179083 (+ 94855us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.179552 (+   469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.179736 (+   184us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.425789 (+246053us) raft_consensus.cc:1685] finished
0709 13:05:44.425823 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.426099 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31205,"prepare.queue_time_us":262401,"raft.queue_time_us":182285,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":12087,"spinlock_wait_cycles":16774400,"thread_start_us":755449,"threads_started":2,"wal-append.queue_time_us":777407}
W20260709 13:05:44.873369 14019 rpcz_store.cc:269] 0709 13:05:43.259174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.259349 (+   175us) service_pool.cc:224] Handling call
0709 13:05:44.872420 (+1613071us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540023c654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:44.872679 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3130405760}
W20260709 13:05:44.872850 13867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.011s	sys 0.020s
I20260709 13:05:43.969740 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:44.931485 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:44.937562 13871 raft_consensus.cc:2410] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:05:44.937898 17817 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:44.938725 13769 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:44.942510 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:44.943223 14015 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:05:44.943979 13866 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:44.944933 13771 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57626 (request call id 246) took 26667 ms (client timeout 15000 ms). Trace:
I20260709 13:05:44.930773 17841 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:44.922919 17948 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.941540 13769 catalog_manager.cc:5697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:44.931713 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 135776us
W20260709 13:05:44.941695 17932 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.191s	user 0.001s	sys 0.007s
I20260709 13:05:44.918313 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2536) took 1113 ms. Trace:
I20260709 13:05:44.945031 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:44.923288 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 462) took 1138 ms. Trace:
W20260709 13:05:44.941718 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.919629 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2541) took 1058 ms. Trace:
I20260709 13:05:44.953109 14023 rpcz_store.cc:276] 0709 13:05:43.785176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.818762 (+1033586us) service_pool.cc:224] Handling call
0709 13:05:44.878667 (+ 59905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.879624 (+   957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.922991 (+ 43367us) spinlock_profiling.cc:243] Waited 57.3 ms on lock 0x7b540023c650. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:44.923195 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131893504}
I20260709 13:05:44.915867 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2546) took 1052 ms. Trace:
I20260709 13:05:44.951768 13870 rpcz_store.cc:276] 0709 13:05:43.804795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.804972 (+   177us) service_pool.cc:224] Handling call
0709 13:05:44.917975 (+1113003us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b54001aac54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:44.918199 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1771307264}
I20260709 13:05:44.926527 17925 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:44.963598 17948 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.963129 13769 catalog_manager.cc:5697] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 6 to 10, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 10 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:44.908866 17938 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.966740 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:44.966863 17938 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:44.968003 14023 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:44.969441 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:44.954461 13872 rpcz_store.cc:276] 0709 13:05:43.861551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.861824 (+   273us) service_pool.cc:224] Handling call
0709 13:05:44.919296 (+1057472us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b54002e4fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:44.919535 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1631647488}
I20260709 13:05:44.966542 13761 maintenance_manager.cc:643] P 91c63734839347c4aee340a59c8e3221: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 11.040s	user 0.207s	sys 0.122s Metrics: {"bytes_written":30244,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":461870,"dirs.run_cpu_time_us":611,"dirs.run_wall_time_us":26757,"drs_written":1,"lbm_read_time_us":73689,"lbm_reads_10-100_ms":4,"lbm_write_time_us":916408,"lbm_writes_10-100_ms":15,"lbm_writes_gt_100_ms":2,"lbm_writes_lt_1ms":17,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":404836,"threads_started":2,"update_count":9,"wal-append.queue_time_us":192404}
W20260709 13:05:44.907855 17694 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.064s	user 0.013s	sys 0.005s
I20260709 13:05:44.976742 13875 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:44.929790 17820 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:44.935446 13989 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:44.974291 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:44.915560 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:44.975371 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:44.974743 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:44.904814 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 462815us
I20260709 13:05:44.931092 17725 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:44.854588 14020 rpcz_store.cc:276] 0709 13:05:43.630241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.631129 (+   888us) service_pool.cc:224] Handling call
0709 13:05:43.631491 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.759636 (+128145us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.759672 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:43.760270 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:05:43.771825 (+ 11555us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:43.955966 (+184141us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.819331 (+863365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.832637 (+ 13306us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.844589 (+ 11952us) raft_consensus.cc:1685] finished
0709 13:05:44.844627 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.844800 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106032,"prepare.queue_time_us":390809,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":225526,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":21665,"thread_start_us":974651,"threads_started":4,"wal-append.queue_time_us":91801}
I20260709 13:05:45.010095 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:45.011113 13871 raft_consensus.cc:2368] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11.
I20260709 13:05:44.950980 17841 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:45.037418 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:45.038789 13870 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:45.044502 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2502) took 1592 ms. Trace:
I20260709 13:05:45.044683 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:45.044586 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:45.030406 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:44.980020 13769 catalog_manager.cc:5697] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:45.052302 17886 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:45.053177 17886 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.056061 17886 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:45.056985 13950 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:44.969995 17938 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.058043 17893 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.058914 14555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 532) took 1496 ms. Trace:
I20260709 13:05:45.059430 17930 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:45.060146 17930 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.060956 17893 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:45.063923 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2489) took 2074 ms. Trace:
I20260709 13:05:45.064420 13949 rpcz_store.cc:276] 0709 13:05:42.989660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.324762 (+335102us) service_pool.cc:224] Handling call
0709 13:05:43.325231 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.325740 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.325778 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:43.363679 (+ 37901us) log.cc:844] Serialized 26 byte log entry
0709 13:05:43.540644 (+176965us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:43.540692 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.540770 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.940473 (+1399703us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:45.063293 (+122820us) raft_consensus.cc:1685] finished
0709 13:05:45.063323 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.063671 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":999,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":50711808,"thread_start_us":130666,"threads_started":1,"wal-append.queue_time_us":274859}
W20260709 13:05:45.066426 17726 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.555s	user 0.011s	sys 0.011s
I20260709 13:05:45.067013 17726 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.068238 17920 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:45.069062 17920 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.070338 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:45.073657 17828 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:45.073995 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:45.074472 17828 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.075892 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:45.077090 17828 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:45.078114 17930 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:45.078444 17838 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:45.078802 17930 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:45.079367 17838 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:45.080857 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.053807 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:45.067097 17921 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:45.045053 13946 rpcz_store.cc:276] 0709 13:05:43.451610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.085817 (+634207us) service_pool.cc:224] Handling call
0709 13:05:44.381354 (+295537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.382045 (+   691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.005868 (+623823us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b5400088750. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:45.044266 (+ 38398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":678673280}
I20260709 13:05:45.088014 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:45.088807 14020 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:45.056062 13762 maintenance_manager.cc:419] P 91c63734839347c4aee340a59c8e3221: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9034 bytes on disk
I20260709 13:05:45.092648 14015 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:45.094365 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 115601us
I20260709 13:05:45.094995 17938 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:45.095139 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:44.915648 17747 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.085300 17726 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:44.958554 13867 rpcz_store.cc:276] 0709 13:05:43.863167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.934461 (+ 71294us) service_pool.cc:224] Handling call
0709 13:05:43.934889 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.935154 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:44.063548 (+128394us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.913247 (+849699us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.914727 (+  1480us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.914814 (+    87us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:44.915014 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.915159 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.915781 (+   622us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10150,"prepare.queue_time_us":419132,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":108,"raft.queue_time_us":10424,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":254,"thread_start_us":953615,"threads_started":2,"wal-append.queue_time_us":792266}
I20260709 13:05:45.119719 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:45.052342 17968 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:45.119979 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:45.121652 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2545) took 1253 ms. Trace:
I20260709 13:05:45.059463 14555 rpcz_store.cc:276] 0709 13:05:43.562207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.562434 (+   227us) service_pool.cc:224] Handling call
0709 13:05:43.562916 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.683571 (+120655us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:43.872353 (+188782us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.155601 (+283248us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.155676 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:44.155727 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.057688 (+901961us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x7b1800372be0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419
0709 13:05:45.058361 (+   673us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54005302d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:45.058438 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.058652 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7494,"prepare.queue_time_us":354665,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":561,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":9376,"spinlock_wait_cycles":452855424,"thread_start_us":437511,"threads_started":2,"wal-append.queue_time_us":606729}
W20260709 13:05:45.128448 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 300872us
W20260709 13:05:45.118745 17712 consensus_meta.cc:287] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.006s
I20260709 13:05:45.137595 13875 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:45.138327 13868 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:45.116641 13769 catalog_manager.cc:5697] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 8 to 9, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 9 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.137899 17712 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.139779 13838 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:45.142216 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:45.041569 17694 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.148470 17919 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:45.148597 17981 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.149264 17719 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.149471 17980 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.149806 17699 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.129642 17938 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:45.151544 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:45.152621 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:45.152985 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 219157us
I20260709 13:05:45.153700 17817 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:45.154929 17747 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:45.155584 17838 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:45.156350 17896 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.655s	user 0.015s	sys 0.001s
W20260709 13:05:45.156889 13839 leader_election.cc:336] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.703s (SENT)
W20260709 13:05:44.946549 13771 rpcz_store.cc:269] 0709 13:05:18.277592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.277875 (+   283us) service_pool.cc:224] Handling call
0709 13:05:44.944610 (+26666735us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:37.243137 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:37.243897 (+   760us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:37.243970 (+    73us) write_op.cc:435] Acquired schema lock
0709 13:05:37.243991 (+    21us) tablet.cc:662] Decoding operations
0709 13:05:37.245143 (+  1152us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:37.245258 (+   115us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:37.245287 (+    29us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:05:37.246336 (+  1049us) tablet.cc:701] Row locks acquired
0709 13:05:37.246389 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:05:37.246638 (+   249us) write_op.cc:270] Start()
0709 13:05:37.246783 (+   145us) write_op.cc:276] Timestamp: P: 1783602337246552 usec, L: 0
0709 13:05:37.246804 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:05:37.321694 (+ 74890us) log.cc:844] Serialized 3626 byte log entry
0709 13:05:38.559151 (+1237457us) op_driver.cc:464] REPLICATION: finished
0709 13:05:40.506734 (+1947583us) spinlock_profiling.cc:243] Waited 91.6 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c03290ac 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c6e69ef0
0709 13:05:41.076198 (+569464us) write_op.cc:301] APPLY: starting
0709 13:05:41.076302 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:41.077155 (+   853us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:41.077209 (+    54us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:41.154854 (+ 77645us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:41.385980 (+231126us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 13:05:41.402589 (+ 16609us) write_op.cc:312] APPLY: finished
0709 13:05:41.403014 (+   425us) log.cc:844] Serialized 71 byte log entry
0709 13:05:42.047061 (+644047us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:42.061937 (+ 14876us) write_op.cc:454] Released schema lock
0709 13:05:42.082184 (+ 20247us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:42.123885 (+ 41701us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:42.147790 (+ 23905us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":424072,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":2483309,"apply.run_cpu_time_us":69274,"apply.run_wall_time_us":1033315,"num_ops":9,"prepare.queue_time_us":147304,"prepare.run_cpu_time_us":35653,"prepare.run_wall_time_us":633133,"raft.queue_time_us":200029,"raft.run_cpu_time_us":45465,"raft.run_wall_time_us":2705496,"replication_time_us":1250768,"spinlock_wait_cycles":210759040,"thread_start_us":3795710,"threads_started":5,"wal-append.queue_time_us":1677975,"wal-append.run_cpu_time_us":21498,"wal-append.run_wall_time_us":2391599}]]}
I20260709 13:05:45.136493 17948 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.174870 14024 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:45.185971 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:45.196324 14020 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
W20260709 13:05:45.197503 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.180600 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:45.187407 13875 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 13:05:45.126269 13873 rpcz_store.cc:276] 0709 13:05:43.862861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.919259 (+ 56398us) service_pool.cc:224] Handling call
0709 13:05:43.951620 (+ 32361us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:44.114897 (+163277us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:44.632459 (+517562us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.819368 (+186909us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:45.006933 (+187565us) log.cc:844] Serialized 52 byte log entry
0709 13:05:45.031668 (+ 24735us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:45.031742 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.039469 (+  7727us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.082876 (+ 43407us) raft_consensus.cc:1685] finished
0709 13:05:45.082911 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.116195 (+ 33284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117911,"prepare.queue_time_us":295135,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":247212,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":21642,"thread_start_us":769828,"threads_started":3,"wal-append.queue_time_us":191858}
I20260709 13:05:45.215520 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:45.184634 17946 consensus_meta.cc:287] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.636s	user 0.012s	sys 0.000s
I20260709 13:05:45.169963 13761 maintenance_manager.cc:643] P 91c63734839347c4aee340a59c8e3221: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.047s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":682,"lbm_reads_lt_1ms":4}
I20260709 13:05:45.218189 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:45.177347 17950 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.673s	user 0.011s	sys 0.007s
I20260709 13:05:45.186025 17707 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:45.170640 17920 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:45.235711 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:45.244285 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:45.243703 14023 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
W20260709 13:05:45.248848 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 109339us
W20260709 13:05:45.250464 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 60721.5us
I20260709 13:05:45.256349 17712 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.258590 14552 raft_consensus.cc:2368] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10.
I20260709 13:05:45.259423 17699 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.260125 14020 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:05:45.261658 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:45.230969 13769 catalog_manager.cc:5697] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:45.267558 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 147898us
W20260709 13:05:45.251477 17926 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.540s	user 0.013s	sys 0.004s
I20260709 13:05:45.281131 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:45.282400 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:45.284373 13870 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:45.285976 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:45.283272 14024 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:05:45.295835 13769 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 8 to 9, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 9 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.298359 13769 catalog_manager.cc:5697] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:45.291479 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 111036us
W20260709 13:05:45.277606 17541 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.844s	user 0.012s	sys 0.008s
I20260709 13:05:45.312911 14020 raft_consensus.cc:2368] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10.
I20260709 13:05:45.304945 13949 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:45.231019 17981 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.323163 13947 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:45.323961 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:45.248554 17919 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:45.320722 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 105299us
I20260709 13:05:45.300489 17921 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:45.245018 17896 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.337484 14020 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:05:45.238837 17980 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.338654 13989 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:45.316478 17817 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:45.264259 17838 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.277979 17719 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.365263 14024 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:45.292565 17707 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.375279 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:45.351686 17921 raft_consensus.cc:697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:45.363448 17883 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:45.388571 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:45.388841 13949 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:45.366611 13769 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 5 to 8, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 8 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.374891 17817 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:45.400077 17694 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.418578 13769 catalog_manager.cc:5697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 8 to 11, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 11 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.422205 13769 catalog_manager.cc:5697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 4 to 6, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 6 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:45.397812 13988 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.799s (SENT)
I20260709 13:05:45.427119 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:45.427371 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2500) took 2003 ms. Trace:
I20260709 13:05:45.381466 17920 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.382611 17893 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:45.427983 13948 rpcz_store.cc:276] 0709 13:05:43.423853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.044115 (+620262us) service_pool.cc:224] Handling call
0709 13:05:44.090788 (+ 46673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.651174 (+560386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.399831 (+748657us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b5400085f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:45.427279 (+ 27448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284625792}
W20260709 13:05:45.435132 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 334480us
I20260709 13:05:45.393798 17699 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.436517 13869 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:05:45.441279 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1839) took 1675 ms. Trace:
I20260709 13:05:45.445060 13769 catalog_manager.cc:5697] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 5 to 8, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 8 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:45.472363 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 150652us
I20260709 13:05:45.443461 17950 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:45.448748 14022 rpcz_store.cc:276] 0709 13:05:43.632981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.416081 (+783100us) service_pool.cc:224] Handling call
0709 13:05:44.811534 (+395453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.828994 (+ 17460us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:44.894814 (+ 65820us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.304677 (+409863us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.304734 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:45.304788 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.304874 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.307983 (+  3109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48531,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":94,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":8908,"thread_start_us":388137,"threads_started":1,"wal-append.queue_time_us":519782}
I20260709 13:05:45.495076 13942 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:45.495352 17966 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.496172 17854 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 13:05:45.496260 17966 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.496447 13769 catalog_manager.cc:5697] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 4 to 5, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 5 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:45.498201 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.498579 13769 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:45.500275 17992 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 13:05:45.508108 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 244947us
W20260709 13:05:45.509786 17998 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 13:05:45.510800 17944 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:05:45.511020 13769 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 4 to 6, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 6 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.514467 13769 catalog_manager.cc:5697] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.516381 17707 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.516731 13769 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.518507 17838 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:45.519223 17926 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:45.518460 13769 catalog_manager.cc:5697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.519877 17926 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.520453 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:45.520679 13769 catalog_manager.cc:5697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:45.521445 14554 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
W20260709 13:05:45.524039 14552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 754) took 1663 ms (client timeout 2012 ms). Trace:
W20260709 13:05:45.524500 14552 rpcz_store.cc:269] 0709 13:05:43.861032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.867503 (+  6471us) service_pool.cc:224] Handling call
0709 13:05:45.523758 (+1656255us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54003372d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:45.523929 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2630221056}
I20260709 13:05:45.525095 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2544) took 1662 ms. Trace:
I20260709 13:05:45.525616 13874 rpcz_store.cc:276] 0709 13:05:43.862586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.916960 (+ 54374us) service_pool.cc:224] Handling call
0709 13:05:43.917367 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.917747 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.917778 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.924021 (+1006243us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b2c00000658. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:05:44.924750 (+   729us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.315107 (+390357us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:45.320634 (+  5527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.337591 (+ 16957us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.524533 (+186942us) raft_consensus.cc:1685] finished
0709 13:05:45.524563 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.524863 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29566,"raft.queue_time_us":70454,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":45241,"spinlock_wait_cycles":4982528,"thread_start_us":389881,"threads_started":1,"wal-append.queue_time_us":460895}
W20260709 13:05:45.528112 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.544296 17838 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:45.545224 17838 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.546859 17981 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:45.548404 13914 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:45.548148 14555 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.549686 13914 leader_election.cc:400] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10.
I20260709 13:05:45.549986 13914 leader_election.cc:403] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:05:45.550508 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 581) took 1421 ms. Trace:
I20260709 13:05:45.550923 13868 rpcz_store.cc:276] 0709 13:05:44.129084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.129254 (+   170us) service_pool.cc:224] Handling call
0709 13:05:44.129575 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.182404 (+1052829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.550164 (+367760us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b540026c0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:45.550432 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2188689792}
I20260709 13:05:45.551467 17980 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:45.463349 17814 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:45.556172 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 69219.9us
I20260709 13:05:45.571869 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:45.572835 13949 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:45.580240 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.548205 17838 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.582469 18002 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:45.583135 18002 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.583608 18002 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:45.584712 17728 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.585508 17728 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.588416 17728 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.589075 17921 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.590903 17719 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.590492 17817 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:45.593304 17841 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.003s
W20260709 13:05:45.593468 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:45.608711 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:45.581681 17992 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.080s	user 0.002s	sys 0.000s
I20260709 13:05:45.593690 17785 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.593809 17726 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:45.594285 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:45.594558 17974 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:45.611369 17974 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.612496 17726 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.595088 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:45.595100 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1853) took 1729 ms. Trace:
W20260709 13:05:45.604411 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 47286.1us
I20260709 13:05:45.605964 13948 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:05:45.607452 13988 leader_election.cc:336] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.813s (SENT)
I20260709 13:05:45.429453 17938 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.485270 17920 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.597731 17926 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.636018 13868 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:45.616335 17726 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:45.664849 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 361395us
I20260709 13:05:45.616669 14021 rpcz_store.cc:276] 0709 13:05:43.865491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.849493 (+984002us) service_pool.cc:224] Handling call
0709 13:05:44.859368 (+  9875us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:44.898202 (+ 38834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.898257 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:44.925971 (+ 27714us) log.cc:844] Serialized 52 byte log entry
0709 13:05:45.253319 (+327348us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:45.333557 (+ 80238us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.335707 (+  2150us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.362305 (+ 26598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.362503 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.540631 (+178128us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b48002115d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:05:45.594826 (+ 54195us) raft_consensus.cc:1685] finished
0709 13:05:45.594863 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.595033 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25204,"prepare.queue_time_us":95280,"prepare.run_cpu_time_us":223,"prepare.run_wall_time_us":143,"raft.queue_time_us":79661,"raft.run_cpu_time_us":3806,"raft.run_wall_time_us":169757,"spinlock_wait_cycles":9510528,"thread_start_us":326734,"threads_started":1,"wal-append.queue_time_us":411759}
W20260709 13:05:45.667570 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 90838.5us
I20260709 13:05:45.668258 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:45.673355 17838 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:45.634440 17919 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:45.630352 17974 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:45.676007 14015 raft_consensus.cc:1217] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[6.4-7.5]   Dedup: 6.4->[7.5-7.5]
I20260709 13:05:45.636749 17814 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:45.690680 17705 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10.
I20260709 13:05:45.691278 13866 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 10 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 13:05:45.691538 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.692505 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:45.693532 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:45.634721 17896 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.675863 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1852) took 1812 ms. Trace:
I20260709 13:05:45.680326 17712 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.682432 14015 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: [6.4-7.5]
I20260709 13:05:45.683470 17966 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:45.684132 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 107226us
I20260709 13:05:45.684602 13871 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:05:45.692606 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:45.697070 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:45.674777 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:45.697611 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 202918us
I20260709 13:05:45.704789 13868 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:05:45.710229 13839 leader_election.cc:336] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.764s (SENT)
I20260709 13:05:45.644107 17785 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.680964 17694 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.728710 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:45.717084 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 314549us
W20260709 13:05:45.722254 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.714746 17854 consensus_queue.cc:1243] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:45.730599 13874 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:45.731055 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:45.731760 13988 leader_election.cc:336] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.826s (SENT)
I20260709 13:05:45.681247 17838 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:45.734175 13948 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:45.741384 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:45.698945 14016 rpcz_store.cc:276] 0709 13:05:43.863968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.830303 (+966335us) service_pool.cc:224] Handling call
0709 13:05:44.831874 (+  1571us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:44.836611 (+  4737us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:44.838100 (+  1489us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.149990 (+311890us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:45.150818 (+   828us) log.cc:844] Serialized 52 byte log entry
0709 13:05:45.150992 (+   174us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:45.151258 (+   266us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.151458 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.151551 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.675558 (+524007us) raft_consensus.cc:1685] finished
0709 13:05:45.675594 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.675800 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70032,"prepare.queue_time_us":1648,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":103,"raft.queue_time_us":72940,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":351,"thread_start_us":242190,"threads_started":1,"wal-append.queue_time_us":464855}
I20260709 13:05:45.745854 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:45.750937 13872 raft_consensus.cc:2393] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:45.758204 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 62095.2us
I20260709 13:05:45.737950 17694 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.739796 17838 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:45.768126 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:45.736202 17966 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:45.735977 17712 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:45.779809 13839 leader_election.cc:336] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.062s (SENT)
I20260709 13:05:45.779289 13948 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:45.783010 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:45.783856 13950 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:45.784732 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:45.785342 13950 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:05:45.786448 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:45.787564 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:45.787921 13950 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:45.788540 13949 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:45.787649 17712 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.766671 17841 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:45.791112 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 50281.6us
I20260709 13:05:45.788371 13942 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
W20260709 13:05:45.777544 17747 consensus_meta.cc:287] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.533s	user 0.006s	sys 0.008s
I20260709 13:05:45.796087 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:45.769724 17974 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:05:45.808574 18004 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:05:45.814595 14524 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:45.814885 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1850) took 1968 ms. Trace:
I20260709 13:05:45.815279 14017 rpcz_store.cc:276] 0709 13:05:43.846576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.849894 (+1003318us) service_pool.cc:224] Handling call
0709 13:05:44.899442 (+ 49548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.009255 (+109813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.009294 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.099404 (+ 90110us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.433835 (+334431us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:45.433886 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.433977 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.814608 (+380631us) raft_consensus.cc:1685] finished
0709 13:05:45.814635 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.814814 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140788,"raft.queue_time_us":132393,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":4776,"thread_start_us":274404,"threads_started":1,"wal-append.queue_time_us":507618}
W20260709 13:05:45.810391 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 72891.8us
I20260709 13:05:45.818517 14023 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:05:45.819011 14023 pending_rounds.cc:85] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 4
I20260709 13:05:45.819281 14023 pending_rounds.cc:107] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:05:45.819576 14023 raft_consensus.cc:2889] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:45.820811 17893 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.821355 17893 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.821823 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:45.822185 13944 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:45.822680 13942 raft_consensus.cc:2393] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:45.822727 13947 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:45.823227 13947 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:45.823580 13944 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:45.824182 13947 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:45.825356 13947 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:45.794734 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:45.835168 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 103455us
I20260709 13:05:45.837626 13946 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:45.813906 18002 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.788998 17919 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:05:45.847302 13867 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.005s
I20260709 13:05:45.837710 13838 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:45.836882 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 57794.2us
I20260709 13:05:45.847858 13867 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:05:45.831434 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:45.858239 13949 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:45.826610 17841 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:45.861492 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:45.867115 17707 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:45.867756 17707 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:45.849164 17893 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:45.870352 13988 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.948s (SENT)
I20260709 13:05:45.870908 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:45.871027 17541 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:45.840860 17920 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:45.871593 17541 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.872077 17541 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:45.873334 17694 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:45.874619 17841 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:45.878065 17968 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:45.878544 17938 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:45.878978 17893 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:45.806542 17966 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.881289 17938 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:45.874040 13838 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:45.882300 13989 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:45.883436 14554 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:45.883852 13989 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:45.853581 17919 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.907258 13948 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
W20260709 13:05:45.913102 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:45.914664 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.909768 17926 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:45.916007 17926 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.916419 17926 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:45.917029 17785 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:45.890800 17925 consensus_meta.cc:287] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.921s	user 0.012s	sys 0.006s
I20260709 13:05:45.922794 18002 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:45.925761 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.869156 17707 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.934273 17707 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:45.906750 17974 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.938681 17974 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:45.906250 17919 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:45.884420 17966 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:45.925505 17726 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.999683 17726 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:46.000515 17974 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:45.925742 17981 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:46.363487 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:46.420256 17838 consensus_meta.cc:287] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.627s	user 0.008s	sys 0.004s
W20260709 13:05:46.435056 17926 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.518s	user 0.013s	sys 0.000s
W20260709 13:05:46.418212 17883 consensus_meta.cc:287] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.022s	user 0.014s	sys 0.000s
I20260709 13:05:45.965516 17841 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:46.442716 17921 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.168s	sys 0.304s
W20260709 13:05:46.447301 17921 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.169s	sys 0.304s
I20260709 13:05:46.444224 17883 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:46.443670 17838 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:46.444607 17926 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:46.475929 17838 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:46.479110 17841 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:46.468973 17926 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:46.484347 17541 consensus_meta.cc:287] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.611s	user 0.014s	sys 0.000s
I20260709 13:05:46.489423 17541 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:46.496562 17541 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:46.502943 17707 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.568s	user 0.004s	sys 0.008s
W20260709 13:05:46.502782 17893 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.003s	sys 0.003s
W20260709 13:05:46.504824 17893 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.003s	sys 0.003s
I20260709 13:05:46.508136 17919 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:46.511121 17919 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:46.519423 17981 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:47.328097 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2535) took 2387 ms. Trace:
I20260709 13:05:47.402617 14015 rpcz_store.cc:276] 0709 13:05:44.940301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.126278 (+185977us) service_pool.cc:224] Handling call
0709 13:05:45.675782 (+549504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:45.710915 (+ 35133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.327745 (+1616830us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b5400299fd0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:47.327992 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1193981824}
W20260709 13:05:47.417795 13949 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.512s	user 0.010s	sys 0.012s
I20260709 13:05:47.418483 13949 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
W20260709 13:05:47.419924 13949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 685) took 1611 ms (client timeout 2096 ms). Trace:
W20260709 13:05:47.420401 13949 rpcz_store.cc:269] 0709 13:05:45.808271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.831153 (+ 22882us) service_pool.cc:224] Handling call
0709 13:05:47.419765 (+1588612us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:47.432906 17726 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:47.443397 13941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.561s	user 0.032s	sys 0.038s
W20260709 13:05:47.443940 13941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.562s	user 0.032s	sys 0.038s
W20260709 13:05:47.444831 13942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.620s	user 0.035s	sys 0.115s
W20260709 13:05:47.445431 13942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.620s	user 0.036s	sys 0.116s
I20260709 13:05:47.446725 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 688) took 1637 ms. Trace:
I20260709 13:05:47.447255 13942 rpcz_store.cc:276] 0709 13:05:45.809369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.823307 (+ 13938us) service_pool.cc:224] Handling call
0709 13:05:45.823667 (+   360us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:45.824181 (+   514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.824208 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:45.824853 (+   645us) log.cc:844] Serialized 52 byte log entry
0709 13:05:47.445906 (+1621053us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:47.446026 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.446170 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:47.446250 (+    80us) raft_consensus.cc:1685] finished
0709 13:05:47.446265 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:47.446645 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":728,"raft.queue_time_us":8230,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":11088,"thread_start_us":1620517,"threads_started":1,"wal-append.queue_time_us":637286}
I20260709 13:05:47.484935 17888 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:05:47.485077 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 1.60114e+06us
W20260709 13:05:47.489029 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 1.60385e+06us
I20260709 13:05:47.489355 17946 consensus_queue.cc:1059] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:47.490304 17984 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:47.492476 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:47.494426 17893 raft_consensus.cc:3055] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:47.494968 17893 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:47.495098 17981 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:47.495975 17893 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.497007 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:47.497524 17893 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:47.497925 17981 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:47.499362 13989 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:47.500255 17981 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:47.485872 17888 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:47.501693 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:47.501751 17888 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.505391 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:47.506192 17841 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:47.506371 17981 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:47.506695 17841 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:47.507180 17841 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:47.500304 17919 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:47.510244 17919 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:47.511269 17919 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.512925 17919 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:47.525803 17981 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:47.529400 17946 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:47.530230 17946 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.532786 17946 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:47.533368 17946 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:47.539175 13947 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:47.540148 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1987) took 1780 ms. Trace:
I20260709 13:05:47.540551 13947 rpcz_store.cc:276] 0709 13:05:45.759246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.770213 (+ 10967us) service_pool.cc:224] Handling call
0709 13:05:45.788850 (+ 18637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:47.539736 (+1750886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.540059 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:47.533875 17946 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.546460 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1855) took 2366 ms. Trace:
I20260709 13:05:47.547073 14023 rpcz_store.cc:276] 0709 13:05:45.179583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.318343 (+138760us) service_pool.cc:224] Handling call
0709 13:05:45.341365 (+ 23022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.820089 (+478724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.546085 (+1725996us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b5400412b54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:47.546372 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1096243456}
I20260709 13:05:47.548655 17946 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:47.552320 17981 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:47.557885 17919 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.563059 17919 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.564358 17984 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:47.565683 17984 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.567409 17946 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:05:47.570175 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.563289 17893 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.574013 17888 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.562430 17981 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.576587 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1989) took 1816 ms. Trace:
I20260709 13:05:47.577332 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:47.568136 17984 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:47.578266 13947 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:47.568142 17946 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:47.574898 17888 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:47.561614 17841 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.585924 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.586875 13942 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:47.587810 17888 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:47.588231 17841 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:47.588944 17919 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:47.588881 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:47.594161 13944 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:47.594463 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:47.595364 13942 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:47.577844 13941 rpcz_store.cc:276] 0709 13:05:45.759903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.769737 (+  9834us) service_pool.cc:224] Handling call
0709 13:05:45.780623 (+ 10886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.792889 (+ 12266us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:45.874037 (+ 81148us) log.cc:844] Serialized 14 byte log entry
0709 13:05:47.571614 (+1697577us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:47.571673 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:47.576347 (+  4674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.576412 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:47.576532 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127154,"prepare.queue_time_us":53748,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":127541,"thread_start_us":1562371,"threads_started":1,"wal-append.queue_time_us":576469,"wal-append.run_cpu_time_us":1966,"wal-append.run_wall_time_us":1035563}
I20260709 13:05:45.883447 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:47.579666 17817 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.963s	user 0.008s	sys 0.011s
W20260709 13:05:47.596697 17817 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.983s	user 0.008s	sys 0.012s
W20260709 13:05:47.596814 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 1.71329e+06us
I20260709 13:05:47.598271 17726 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:47.598745 17726 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.599131 17726 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:47.592012 17984 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.599957 17984 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:47.579610 17896 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.602924 17896 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:47.574198 17893 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:47.597832 13838 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:47.612784 17893 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:47.613927 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.615046 17896 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:47.617635 13949 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:47.618454 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.618714 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.621013 13947 raft_consensus.cc:2410] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:47.644692 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.645941 14023 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:47.648028 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:47.648732 13949 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:47.649181 13948 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:47.650529 13941 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:05:47.627926 18003 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.954s	user 0.005s	sys 0.004s
W20260709 13:05:47.629961 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 1.85003e+06us
W20260709 13:05:47.653613 13839 connection.cc:256] RPC call timeout handler was delayed by -1.8066562940000495s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2489s for a response
I20260709 13:05:47.643392 13942 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:05:47.631031 13874 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.880s	user 0.019s	sys 0.003s
I20260709 13:05:47.634471 17981 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:47.660634 13867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2679) took 2685 ms (client timeout 1930 ms). Trace:
I20260709 13:05:47.650383 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.661613 13948 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:47.662889 13948 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:47.651466 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.664590 13941 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:47.665886 13874 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:47.665658 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.670233 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2554) took 1979 ms. Trace:
I20260709 13:05:47.670506 13950 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:05:47.626876 17984 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:47.661298 13867 rpcz_store.cc:269] 0709 13:05:44.975521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.151062 (+175541us) service_pool.cc:224] Handling call
0709 13:05:47.660342 (+2509280us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b540026b6d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:47.660494 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29563264}
W20260709 13:05:47.660743 13839 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.825s (SENT) [suppressed 35 similar messages]
I20260709 13:05:47.668772 14015 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:47.653740 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:47.669585 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:47.673341 13840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.795s	user 0.014s	sys 0.149s
I20260709 13:05:47.670714 13874 rpcz_store.cc:276] 0709 13:05:45.690822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.692306 (+  1484us) service_pool.cc:224] Handling call
0709 13:05:47.670162 (+1977856us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:47.673656 14017 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:47.674235 13840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.796s	user 0.014s	sys 0.150s
I20260709 13:05:47.676887 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:47.677855 17984 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:05:47.678872 13840 connection.cc:256] RPC call timeout handler was delayed by -1.6260612230000788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
I20260709 13:05:47.679811 13944 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:05:47.683396 13949 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:47.681339 17984 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:47.706879 17984 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.681870 17726 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.707722 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:47.708587 14019 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:47.707787 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.710789 17726 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:47.692901 17946 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:47.694706 18035 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:47.712533 18035 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.700080 14023 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:47.701839 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.713971 14015 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:47.711374 18034 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:47.715342 14019 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:05:47.715281 18034 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:47.715816 14019 pending_rounds.cc:85] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 6
I20260709 13:05:47.716221 14019 pending_rounds.cc:107] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:05:47.716549 14019 raft_consensus.cc:2889] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:47.717687 18034 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:47.697546 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.719175 13840 connection.cc:256] RPC call timeout handler was delayed by -1.3173955430000888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 13:05:47.719637 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 58932.3us
I20260709 13:05:47.720368 18035 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:47.721036 13988 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.848s	user 0.015s	sys 0.061s
W20260709 13:05:47.721517 13988 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.849s	user 0.016s	sys 0.062s
I20260709 13:05:47.722662 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:47.721858 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 1.8491e+06us
I20260709 13:05:47.723528 14015 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
W20260709 13:05:47.723815 13988 leader_election.cc:336] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.267s (SENT)
W20260709 13:05:47.689190 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 91415.1us
I20260709 13:05:47.715837 13943 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:47.724344 13988 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:47.721290 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.734551 14019 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:47.726852 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.727165 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:47.748383 13874 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:47.727492 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:47.747421 14015 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:47.728825 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:47.728871 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.729199 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.728408 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.757682 14023 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:47.729666 14017 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:47.757903 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:47.759824 13866 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:47.758953 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:47.733184 14022 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:47.763233 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:47.726258 18041 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.768115 18041 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.770864 13867 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:47.770918 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.771206 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.771775 14016 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:47.725113 18040 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.772167 14017 raft_consensus.cc:2410] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:05:47.775022 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:47.765332 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:47.748052 17817 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:47.775525 14022 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:47.728479 18038 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.776372 13838 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:47.778522 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 54796.3us
I20260709 13:05:47.782397 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:47.783820 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:47.784299 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:47.785456 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:47.786443 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:47.786648 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:47.787057 14016 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:47.787658 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:47.790436 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:47.791060 13875 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:47.724956 17726 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:47.798594 17726 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.801297 17726 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:47.809585 13872 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:47.832866 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1991) took 2072 ms. Trace:
W20260709 13:05:47.809890 13988 connection.cc:256] RPC call timeout handler was delayed by -2.1890818050000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3272s for a response
I20260709 13:05:47.823017 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.824430 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.846275 13943 rpcz_store.cc:276] 0709 13:05:45.760595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.769509 (+  8914us) service_pool.cc:224] Handling call
0709 13:05:45.786373 (+ 16864us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:47.747775 (+1961402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.832576 (+ 84801us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b54003b01d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:47.832785 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4436805376}
I20260709 13:05:47.821942 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:47.846539 13988 connection.cc:256] RPC call timeout handler was delayed by -2.1548118700000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3288s for a response
I20260709 13:05:47.848227 13944 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:47.848693 13871 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:47.819409 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.825335 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:47.849673 14017 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:47.824875 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.850207 13946 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:47.823958 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.850713 13947 raft_consensus.cc:2410] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:05:47.851019 13949 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:47.817289 14022 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:05:47.825927 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.852228 13941 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:47.826409 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.853331 13875 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:47.853458 13948 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:47.853834 14016 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:47.830401 13866 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:47.855795 13950 raft_consensus.cc:2410] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:05:47.858793 13915 connection.cc:256] RPC call timeout handler was delayed by -2.1998345100000734s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
I20260709 13:05:47.859961 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:47.861145 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:47.857286 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:47.810317 18038 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:47.918779 18046 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:05:47.778348 18040 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.863387 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.863481 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:47.864337 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.864292 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.820461 14552 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:47.865852 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:47.886034 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2555) took 2194 ms. Trace:
I20260709 13:05:47.861951 18041 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:47.895390 14021 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:47.896925 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:47.944638 13869 rpcz_store.cc:276] 0709 13:05:45.691190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.697727 (+  6537us) service_pool.cc:224] Handling call
0709 13:05:45.698040 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.729579 (+ 31539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.729633 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.730414 (+   781us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.852074 (+121660us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:45.852119 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.852191 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:47.885641 (+2033450us) raft_consensus.cc:1685] finished
0709 13:05:47.885673 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:47.885908 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14123,"raft.queue_time_us":15938,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"thread_start_us":121113,"threads_started":1,"wal-append.queue_time_us":259191,"wal-append.run_cpu_time_us":2460,"wal-append.run_wall_time_us":1458386}
I20260709 13:05:47.948230 13944 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:47.789835 18035 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:47.819660 18043 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:47.819914 18042 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:47.902345 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:47.903834 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.907974 18044 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:47.858587 13874 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:05:47.911267 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:47.918818 13839 connection.cc:256] RPC call timeout handler was delayed by -1.0062800590001189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
I20260709 13:05:47.919251 13868 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:47.934352 14015 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:47.943099 13872 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
W20260709 13:05:47.942934 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 166938us
W20260709 13:05:47.978595 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 146071us
W20260709 13:05:47.978816 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 129967us
I20260709 13:05:47.980504 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.985668 14017 raft_consensus.cc:2368] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6.
I20260709 13:05:47.986961 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.987183 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:47.983071 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 123112us
I20260709 13:05:47.986681 13873 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:47.988243 13875 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:47.988562 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.989130 14023 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:47.990585 14021 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:47.991520 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:47.993301 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.943051 18038 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:47.987074 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:48.003288 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:48.003912 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:48.004925 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:47.953892 18043 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:48.005909 14021 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:05:48.004098 14015 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:48.009126 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:48.006793 17946 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:48.007750 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:48.009979 14555 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:48.010442 13988 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.344s (SENT)
I20260709 13:05:48.010896 13988 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:05:48.013161 13988 connection.cc:256] RPC call timeout handler was delayed by -1.8650234209999326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2685s for a response
I20260709 13:05:48.012580 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:48.015417 13988 connection.cc:256] RPC call timeout handler was delayed by -1.2596618840000247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1653s for a response
I20260709 13:05:48.012984 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:48.015929 13988 connection.cc:256] RPC call timeout handler was delayed by -1.2146618840000656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
I20260709 13:05:48.016439 14015 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:48.017699 14015 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:48.015450 14021 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:48.018851 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:48.019940 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:48.013460 13944 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:05:48.022428 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:48.024920 13987 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:48.013118 17946 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:48.030522 18050 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:48.032102 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1846) took 4394 ms. Trace:
I20260709 13:05:48.044656 14018 rpcz_store.cc:276] 0709 13:05:43.638116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.764435 (+1126319us) service_pool.cc:224] Handling call
0709 13:05:44.765162 (+   727us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.816143 (+ 50981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.816185 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.887374 (+ 71189us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.349002 (+461628us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:45.349054 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.349154 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:47.627034 (+2277880us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:48.031736 (+404702us) raft_consensus.cc:1685] finished
0709 13:05:48.031770 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.032038 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48640,"raft.queue_time_us":50570,"raft.run_cpu_time_us":388,"raft.run_wall_time_us":11358,"spinlock_wait_cycles":48867072,"thread_start_us":370191,"threads_started":1,"wal-append.queue_time_us":723402}
I20260709 13:05:48.031064 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:48.070084 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 77702.6us
I20260709 13:05:48.020147 18043 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:48.076524 18054 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.990454 17817 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:05:48.077315 14017 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:48.007321 18034 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:48.029191 18041 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:48.081676 14020 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:48.047216 17946 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:48.033726 18035 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:48.074822 18040 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:48.103020 13839 leader_election.cc:336] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.489s (SENT)
I20260709 13:05:48.103744 13839 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:48.080603 18054 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:48.108475 18054 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:48.109071 18054 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.091486 13988 leader_election.cc:336] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.685s (SENT)
I20260709 13:05:48.110477 13988 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:48.083091 18042 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:48.113940 17817 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:48.089772 18034 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.031744 18049 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:05:48.082121 18038 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
W20260709 13:05:48.135080 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 110229us
I20260709 13:05:48.140933 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:48.141965 18057 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:48.077628 18050 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.145287 18050 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:48.133781 18034 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:48.146354 18042 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.109643 18054 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:48.092381 18041 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:48.164853 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 61920.6us
W20260709 13:05:48.186507 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.178732 14019 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
W20260709 13:05:48.187599 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46777.2us
I20260709 13:05:48.199836 18041 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:48.200223 18058 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:48.201505 18058 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.255833 18034 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.218104 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:48.229398 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 2.37181e+06us
I20260709 13:05:48.157055 18061 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.218246 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:48.217080 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:48.224248 13945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.462s	user 0.019s	sys 0.032s
I20260709 13:05:48.217523 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:48.211831 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:48.299767 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 264735us
I20260709 13:05:48.301035 13868 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:48.297582 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:48.300077 14553 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
W20260709 13:05:48.302682 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 97563.4us
I20260709 13:05:48.304308 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:48.310314 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:48.311349 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:48.313314 13945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.552s	user 0.023s	sys 0.039s
I20260709 13:05:48.298311 13872 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:05:48.318284 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.218415 17747 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:48.218169 17817 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.300331 14555 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:05:48.215619 18064 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.321462 18067 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:48.346333 18067 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.348834 18067 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:48.349399 18067 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:48.349800 14024 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:48.298792 13874 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:48.254621 18063 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:48.351075 18063 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.353550 18063 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:48.349800 17984 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:48.354527 17984 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:48.355053 18063 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:48.355932 18063 raft_consensus.cc:2764] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:05:48.355459 17984 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.356611 18063 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:48.356931 17984 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:48.357151 18063 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:48.324246 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:48.358577 13870 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:48.358225 18063 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:48.359691 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:48.361042 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:48.345459 17946 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.362149 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:48.363366 17946 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:48.369187 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.325019 18059 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:05:48.320499 17747 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.326010 17817 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:48.324122 13914 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:48.330279 18054 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:48.333130 13868 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:05:48.317211 18041 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:05:48.313403 18061 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:48.334574 18064 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:48.322614 18049 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:48.335959 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:48.338102 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:48.342406 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 253683us
I20260709 13:05:48.298350 18058 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:48.265971 18043 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:48.390647 13947 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:05:48.321655 18066 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:48.381131 17747 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:48.381986 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 57964us
I20260709 13:05:48.395206 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:48.393639 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:48.382539 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:48.396746 13942 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:48.388564 13869 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:48.393399 18072 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.380645 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:48.398361 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 94551.5us
W20260709 13:05:48.399256 13988 leader_election.cc:336] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 3.272s (SENT)
I20260709 13:05:48.399907 18046 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:48.400611 18061 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:48.401201 17838 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.401705 17838 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.402838 18043 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:48.403855 17838 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:48.404695 17838 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:48.399158 18044 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:48.405290 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:48.396513 13914 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:48.406903 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:48.407577 14556 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:05:48.407873 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:48.409441 17838 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.409842 17838 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:48.387812 18054 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.409252 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:48.414239 18057 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:48.418161 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:48.418354 17817 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:48.419001 17726 raft_consensus.cc:2764] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:05:48.409247 18049 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:48.412447 18058 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:48.421064 18041 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:48.421641 18058 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:48.422715 18058 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:48.421136 18049 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.423887 18049 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:48.426982 18064 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:48.413909 18054 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.428608 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:48.426975 17838 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:48.433369 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:48.433640 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:48.349879 18067 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:48.434312 14556 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:48.416021 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:48.437248 13866 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:48.438297 13870 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:48.438887 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:48.429776 18054 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:48.441747 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:48.442471 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:48.442523 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:48.442729 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:48.443341 13944 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:05:48.443357 13949 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:48.442176 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.442485 13946 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:05:48.445227 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:48.445844 13946 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:48.446797 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:48.403592 18072 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.448827 17747 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.458153 13949 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:05:48.458514 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:48.459477 13944 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:48.451002 18072 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:48.435911 18067 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:48.451390 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:48.451999 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 1990) took 2676 ms. Trace:
I20260709 13:05:48.447328 13946 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:05:48.443445 14551 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:48.394198 18066 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:48.465834 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:48.497213 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58417.4us
I20260709 13:05:48.465270 14556 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:05:48.465317 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:48.487972 13874 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:05:48.499215 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:48.441332 17838 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:48.499302 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:48.503860 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:48.504235 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:48.504624 13868 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:48.505640 13869 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:48.463402 17984 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:48.508265 17817 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:48.508708 17817 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:48.509083 17817 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:48.501098 18066 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:48.514636 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 63318.3us
I20260709 13:05:48.515666 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:48.464953 17747 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:48.470899 18061 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:48.466331 17981 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:48.554363 17981 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:48.554944 17981 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:48.561522 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:48.562345 13950 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:05:48.518537 18072 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:48.566672 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51157.3us
I20260709 13:05:48.567780 13915 leader_election.cc:400] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11.
I20260709 13:05:48.568251 13915 leader_election.cc:403] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:48.568439 18061 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:48.433677 18049 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.572254 18049 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:48.572700 17853 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.521370 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:48.575008 13871 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:48.528364 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:48.576521 13948 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.576958 17817 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.569576 18061 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.530521 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:48.579363 13949 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:48.534484 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:48.537213 18057 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:48.538043 17984 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:48.540336 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.464402 13945 rpcz_store.cc:276] 0709 13:05:45.760276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.760431 (+   155us) service_pool.cc:224] Handling call
0709 13:05:45.760829 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.761113 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:45.761454 (+   341us) log.cc:844] Serialized 14 byte log entry
0709 13:05:48.398721 (+2637267us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:48.398781 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:48.398838 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.398908 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.436179 (+ 37271us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1728,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":86959,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":2633262,"threads_started":2,"wal-append.queue_time_us":240344,"wal-append.run_cpu_time_us":1561,"wal-append.run_wall_time_us":1001590}
I20260709 13:05:48.506286 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:48.580319 13943 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
W20260709 13:05:48.582372 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 76163.7us
I20260709 13:05:48.583601 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:48.594030 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:48.573423 17853 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:48.616034 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:48.614929 18072 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11.
W20260709 13:05:48.618537 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 2.84208e+06us
I20260709 13:05:48.614900 17747 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:05:48.619436 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:48.617316 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:48.623342 18038 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:48.623885 18038 raft_consensus.cc:697] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:48.624176 18057 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:48.625233 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:48.627665 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:48.631263 14016 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:48.630497 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:48.632784 14017 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 9 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:05:48.590734 18067 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:48.614878 17888 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:48.634606 13944 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:48.592806 17946 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:48.635288 13946 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:48.634513 13941 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:48.637279 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:48.640014 14020 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:48.641690 17817 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:48.642328 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:48.642925 14018 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.581281 17984 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.636129 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:48.637786 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:48.637952 14021 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:48.638955 14022 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:48.649391 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:48.650033 14024 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:48.641569 17712 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:48.652181 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:48.652808 14023 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:05:48.624533 18035 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:48.651444 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:48.658931 13946 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:48.568094 17981 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.659332 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:48.662189 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:48.630652 18057 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.665772 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:48.666392 13946 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:48.666842 17853 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:48.651150 17888 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:48.667934 17888 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:48.625706 18072 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:05:48.648650 17946 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:48.673653 17946 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:48.677397 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:48.678902 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:48.679211 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:48.679509 13941 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:48.661731 18067 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:48.680819 17896 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:48.681228 17896 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:48.681695 17896 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:48.683252 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:48.684784 13948 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:05:48.633445 18049 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:05:48.664533 18035 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:48.687932 18035 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.690244 18044 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:48.692811 13946 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:48.698876 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:48.702237 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:48.703002 13948 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:48.704861 13946 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:48.705348 13946 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:48.703644 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:48.706743 18073 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:48.704972 13838 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:48.714008 13838 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:48.683270 18057 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:48.723719 13948 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:48.725059 13948 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:48.715132 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:48.709928 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:48.708636 18041 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:48.742084 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:48.752334 13942 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:48.742748 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:48.681242 18067 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:05:48.760500 14018 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
W20260709 13:05:48.762712 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 46412.8us
I20260709 13:05:48.694397 17896 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.727360 18043 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:48.739949 18049 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:48.707844 18042 consensus_queue.cc:1059] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:05:48.708165 17817 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:05:48.773967 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:48.708425 18050 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:05:48.716848 17726 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:05:48.716260 18044 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:05:48.750954 18046 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:48.779564 18068 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:48.768172 18058 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:48.785280 13945 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:48.751983 18041 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:48.742749 18066 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.769277 18067 consensus_queue.cc:1243] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:05:48.766552 18038 consensus_queue.cc:237] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:48.784332 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 43974us
I20260709 13:05:48.783344 18046 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.785892 18068 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:48.826923 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 117079us
I20260709 13:05:48.833298 18050 raft_consensus.cc:3055] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:48.790747 18041 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:48.821664 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:48.791306 18042 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.851472 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 58676.1us
I20260709 13:05:48.839823 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:48.885618 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.832052 18046 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:48.859548 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:48.821048 18058 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:48.839814 18068 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:48.903867 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:48.902281 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 49834us
I20260709 13:05:48.839318 18050 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:48.841074 14018 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:05:48.913549 18050 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.914222 14018 raft_consensus.cc:606] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:05:48.914636 14018 raft_consensus.cc:614] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:48.915041 18050 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:48.917702 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08002d1c40 after lost signal to thread 12728
W20260709 13:05:48.923125 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08003829a0 after lost signal to thread 13807
I20260709 13:05:48.918550 17981 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:48.919626 18082 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:05:48.920518 18078 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:05:48.894229 18042 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:48.925467 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08003995e0 after lost signal to thread 13899
W20260709 13:05:48.942595 14045 debug-util.cc:398] Leaking SignalData structure 0x7b0800348080 after lost signal to thread 13900
W20260709 13:05:48.944079 14045 debug-util.cc:398] Leaking SignalData structure 0x7b080038b060 after lost signal to thread 13974
I20260709 13:05:48.930092 13941 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:48.830891 18073 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:48.896154 18067 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:05:48.938584 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 79122.9us
W20260709 13:05:48.940757 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.956825 18058 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.958530 18090 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:48.959194 18090 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.959543 18090 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:48.971130 13871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:05:48.971354 18096 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:48.972337 18096 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:48.973479 18096 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:48.975607 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.958400 18066 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:48.982420 17725 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:48.983170 17725 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:48.990691 17919 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:48.991252 17919 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:48.992273 17919 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:48.993607 17919 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:49.000860 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.946870 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:49.006826 14045 debug-util.cc:398] Leaking SignalData structure 0x7b080038f6c0 after lost signal to thread 14048
W20260709 13:05:49.008981 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08003ded20 after lost signal to thread 14585
W20260709 13:05:49.009438 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08003cdf20 after lost signal to thread 17707
I20260709 13:05:49.014635 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:49.016934 13941 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
W20260709 13:05:49.021854 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 108899us
I20260709 13:05:49.025883 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:49.032804 17896 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:49.034874 18083 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:05:49.037317 18073 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:49.042937 17725 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:49.026095 18064 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:05:49.012588 18067 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:49.050433 13946 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:49.051395 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 503480us
I20260709 13:05:49.052276 13989 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:49.053242 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:49.051553 13946 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:49.061410 14522 connection.cc:256] RPC call timeout handler was delayed by -1.7386379920001218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2096s for a response
W20260709 13:05:49.061925 14522 connection.cc:256] RPC call timeout handler was delayed by -1.4006376869999713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1821s for a response
W20260709 13:05:49.062485 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.067624 13989 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:49.070428 14555 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:49.070791 17817 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:49.071269 17817 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.071638 17817 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:49.072340 13914 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:49.073364 18057 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:49.074270 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 454887us
I20260709 13:05:49.074715 18080 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:49.075201 18067 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:49.075974 17707 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:49.077991 18072 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:49.078343 18061 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:49.078887 17899 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:05:49.119818 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 52279.8us
I20260709 13:05:49.120784 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:49.121730 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:05:49.081537 18084 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 13:05:49.082962 18063 consensus_queue.cc:1059] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:05:49.089053 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.095772 13942 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:05:49.096954 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 71146.1us
I20260709 13:05:49.100116 18034 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:49.101086 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.106163 13946 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
W20260709 13:05:49.107466 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 156223us
I20260709 13:05:49.140760 13946 pending_rounds.cc:85] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 4
I20260709 13:05:49.141041 13946 pending_rounds.cc:107] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:05:49.141285 13946 raft_consensus.cc:2889] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:49.132359 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:49.122815 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 50554.4us
I20260709 13:05:49.126613 13871 raft_consensus.cc:606] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:05:49.126430 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.143270 13871 raft_consensus.cc:614] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:49.143579 14556 raft_consensus.cc:2368] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11.
I20260709 13:05:49.128593 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:49.131801 13947 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:49.138324 17707 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:49.142632 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.144779 13944 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:49.104945 18075 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:49.150707 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.151243 18063 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:49.157801 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:49.151885 17838 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:49.105576 17921 consensus_queue.cc:1059] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:05:49.103714 18072 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:49.119398 17899 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:49.130293 18074 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:49.161437 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:49.162338 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:49.172627 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:49.173517 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:49.175057 14018 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:49.177351 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:49.157663 17893 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:49.185246 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.164462 18063 raft_consensus.cc:697] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:49.197830 13989 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:49.166296 18075 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:49.170992 18061 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:49.176069 18066 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:49.201953 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.206749 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:49.178325 18059 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:49.212477 14552 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:49.179103 18091 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:49.182489 17893 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:49.193843 18064 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:49.188356 18074 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:49.193583 18054 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:49.197486 17896 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:49.212009 17888 consensus_meta.cc:287] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.003s
I20260709 13:05:49.190021 17981 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:05:49.220480 18073 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:49.233354 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.209211 18042 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:49.241741 14553 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 5.
I20260709 13:05:49.221140 17919 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:49.217624 18091 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:49.248103 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.203455 17747 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:05:49.221417 18061 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:49.256541 18098 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:49.254266 18066 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:49.260841 18102 log.cc:927] Time spent T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:05:49.238581 18054 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:49.287062 13944 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:49.289266 18066 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:49.290520 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 85278.2us
I20260709 13:05:49.292122 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.293052 17817 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:49.301265 14555 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:49.265880 17946 consensus_meta.cc:287] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.591s	user 0.015s	sys 0.000s
I20260709 13:05:49.326318 17946 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:49.334298 18074 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:49.316241 18057 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:49.228677 18072 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:49.336586 18072 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:49.339264 18093 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.377s	user 0.002s	sys 0.002s
I20260709 13:05:49.339725 18054 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:49.238193 17888 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:49.349063 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:49.350469 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:49.350636 18034 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:49.352159 18034 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:49.352754 18034 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:49.319689 18063 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:49.281631 18075 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:49.354866 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:49.355544 13944 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:49.252339 18091 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:49.324694 18061 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:49.343463 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:49.344902 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:49.366323 14015 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:49.346076 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.242901 17707 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:49.245925 17899 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:49.348354 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:49.258105 17893 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:49.352317 14023 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:49.317445 17919 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.361135 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:49.389973 13946 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:49.367920 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:49.391183 14555 raft_consensus.cc:2368] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11.
W20260709 13:05:49.391021 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 184392us
W20260709 13:05:49.369473 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 304709us
W20260709 13:05:49.378902 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.388329 14021 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:49.396376 13948 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:49.350807 17888 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:49.398586 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:49.404186 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:49.407873 13874 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:49.411273 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:49.412020 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:49.411185 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:49.418789 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:49.418081 14553 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:49.419973 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:49.420957 13875 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:49.424254 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:49.425351 13942 raft_consensus.cc:2410] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:05:49.426822 13869 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:05:49.416394 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 242933us
W20260709 13:05:49.438900 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:49.439445 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.443899 13950 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:49.445137 14024 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:05:49.446101 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:05:49.449556 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.400051 18068 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:49.458124 18103 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:05:49.468511 13874 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:49.411579 17893 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:49.408756 18113 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:49.476567 13839 leader_election.cc:336] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.722s (SENT)
I20260709 13:05:49.477288 13839 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:49.485270 13944 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:49.363044 18034 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:49.438009 17841 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:49.501756 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:49.502770 13946 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:05:49.503759 13948 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:49.504957 13949 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:05:49.516443 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.502112 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:49.405720 17919 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:49.471299 17899 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:49.527050 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.518486 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:49.540256 14024 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:49.565675 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.535020 18068 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:49.589780 18008 consensus_queue.cc:1059] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
W20260709 13:05:49.592633 18118 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:05:49.491926 18113 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:49.594379 18090 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.004s
I20260709 13:05:49.594872 18090 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:49.596144 18046 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.661s	user 0.006s	sys 0.006s
I20260709 13:05:49.596601 18046 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:49.598629 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:49.598812 18046 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:49.599829 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.600067 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.601016 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.601657 14561 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:49.526487 17921 consensus_queue.cc:1059] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:05:49.629621 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:49.630297 18113 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:49.630385 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:49.657862 14556 raft_consensus.cc:2368] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11.
I20260709 13:05:49.657929 14015 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:49.573971 18050 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:49.631072 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:49.632967 18098 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:49.639034 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.661474 14559 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:05:49.662576 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:49.663973 14024 raft_consensus.cc:2393] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:49.661513 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.655769 18090 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:49.659860 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 183335us
I20260709 13:05:49.666133 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.528920 17841 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:49.701792 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:49.660928 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:05:49.714023 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.717054 14016 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:49.717298 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:49.659137 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:49.719988 14015 raft_consensus.cc:2410] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:05:49.722417 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:49.723311 14015 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:05:49.724676 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:49.748915 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:49.756973 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:49.759275 13875 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:49.761023 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 193630us
W20260709 13:05:49.762742 13988 leader_election.cc:336] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.653s (SENT)
W20260709 13:05:49.782341 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 117793us
I20260709 13:05:49.816509 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:49.882330 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 65628.3us
I20260709 13:05:49.883103 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:49.907313 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 144606us
I20260709 13:05:49.787650 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:49.786643 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:49.786437 14552 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:49.827630 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:49.883575 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:49.932268 13875 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
W20260709 13:05:49.934602 18124 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:05:49.902866 18111 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:49.935484 18111 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:49.934808 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:49.938671 13875 raft_consensus.cc:2368] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
W20260709 13:05:49.730940 18041 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.837s	user 0.011s	sys 0.006s
I20260709 13:05:49.948019 18041 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:49.950453 18041 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:49.935886 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:49.830133 18112 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:49.939960 18072 consensus_meta.cc:287] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.603s	user 0.013s	sys 0.000s
I20260709 13:05:49.883123 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:49.952211 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:49.952791 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 69800.6us
W20260709 13:05:49.945132 18059 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.724s	user 0.007s	sys 0.001s
W20260709 13:05:49.930124 18120 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:05:49.909193 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:49.955204 13988 leader_election.cc:336] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.956s (SENT)
I20260709 13:05:49.945899 13870 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
W20260709 13:05:49.832543 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.787209 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:50.007395 14024 raft_consensus.cc:2368] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7.
W20260709 13:05:49.952301 18073 consensus_meta.cc:287] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.680s	user 0.012s	sys 0.000s
I20260709 13:05:50.034283 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:50.036423 13875 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:05:49.827378 18125 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:05:49.786509 14560 raft_consensus.cc:3055] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:49.882618 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:50.072605 18127 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:50.073541 18127 raft_consensus.cc:697] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:50.074826 18127 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:49.882591 17893 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:50.272768 13839 leader_election.cc:336] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.877s (SENT)
I20260709 13:05:50.273370 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:50.275311 14015 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:50.273516 13839 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:50.021112 18090 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.308506 13875 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:50.308218 18090 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.308954 17946 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:50.310653 13872 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:05:50.310783 18090 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:49.951965 18112 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.312234 13872 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:50.020601 18059 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:50.316329 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 361136us
W20260709 13:05:50.317144 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:50.318799 13988 leader_election.cc:336] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.908s (SENT)
I20260709 13:05:50.069751 14560 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:50.270454 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:50.321336 13869 raft_consensus.cc:2368] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7.
I20260709 13:05:50.322139 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:50.323271 13874 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
W20260709 13:05:50.324702 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.325470 13875 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
W20260709 13:05:50.325489 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.329154 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:50.334012 14024 raft_consensus.cc:2410] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:05:50.048919 18073 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:50.313482 18112 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:50.340767 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 68056.3us
W20260709 13:05:50.341660 13839 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.916s (SENT)
I20260709 13:05:50.342023 13839 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:50.337201 13874 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:05:50.337735 13915 connection.cc:256] RPC call timeout handler was delayed by -2.2192181169999685s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
I20260709 13:05:50.342829 13874 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:50.343001 13915 connection.cc:256] RPC call timeout handler was delayed by -2.1976181169999336s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 13:05:50.343572 13915 connection.cc:256] RPC call timeout handler was delayed by -2.1904181170000356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
W20260709 13:05:50.345371 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:50.345270 13915 connection.cc:256] RPC call timeout handler was delayed by -2.1742181170000094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 13:05:50.345803 13915 connection.cc:256] RPC call timeout handler was delayed by -2.1688181170000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
W20260709 13:05:50.346258 13915 connection.cc:256] RPC call timeout handler was delayed by -2.1670181169999978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
W20260709 13:05:50.346660 13915 connection.cc:256] RPC call timeout handler was delayed by -2.1571181169999818s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
W20260709 13:05:50.347048 13915 connection.cc:256] RPC call timeout handler was delayed by -2.11661811700003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260709 13:05:50.347517 13915 connection.cc:256] RPC call timeout handler was delayed by -2.0545181170000433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
W20260709 13:05:50.347882 13915 connection.cc:256] RPC call timeout handler was delayed by -1.953718116999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18720000000000003s for a response
W20260709 13:05:50.338804 13873 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.005s	sys 0.026s
W20260709 13:05:50.348637 13873 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.005s	sys 0.026s
W20260709 13:05:50.348251 13915 connection.cc:256] RPC call timeout handler was delayed by -1.9357181169999649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
W20260709 13:05:50.349866 13915 connection.cc:256] RPC call timeout handler was delayed by -1.7845181169999478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
W20260709 13:05:50.350287 13915 connection.cc:256] RPC call timeout handler was delayed by -1.7593181170000207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2088s for a response
W20260709 13:05:50.350666 13915 connection.cc:256] RPC call timeout handler was delayed by -1.724218116999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
W20260709 13:05:50.351016 13915 connection.cc:256] RPC call timeout handler was delayed by -1.7044181170000456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2149s for a response
W20260709 13:05:50.351379 13915 connection.cc:256] RPC call timeout handler was delayed by -1.6999181170000384s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2154s for a response
I20260709 13:05:50.030802 18041 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.304s
I20260709 13:05:50.358821 13869 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:05:50.336000 18091 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.941s	user 0.011s	sys 0.001s
I20260709 13:05:50.367681 13869 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:50.344434 14560 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:50.367466 13988 leader_election.cc:336] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.703s (SENT)
W20260709 13:05:50.355896 18038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.017s	sys 0.061s
I20260709 13:05:50.389670 14021 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:05:50.365486 13874 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.353574 18090 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:50.319809 17893 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.344151 18074 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:50.369937 18059 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.403266 13869 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:50.366044 18113 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.400831 13874 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:50.429476 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:50.403697 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:50.393662 18038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.017s	sys 0.061s
I20260709 13:05:50.425707 13869 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:50.441545 14560 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:50.388386 18112 consensus_queue.cc:1059] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:05:50.385913 18091 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.457782 13874 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:05:50.467607 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 125955us
W20260709 13:05:50.468947 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 65362.1us
W20260709 13:05:50.480255 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.430930 18074 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.431217 18095 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:05:50.464334 18090 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:50.436275 18059 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.438202 18113 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:50.453248 18100 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.045s	sys 0.051s
W20260709 13:05:50.511986 18144 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 13:05:50.504835 18100 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.045s	sys 0.052s
I20260709 13:05:50.514739 17853 raft_consensus.cc:3055] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:50.526105 17853 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:50.496659 18116 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:50.529909 13988 leader_election.cc:336] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.855s (SENT)
I20260709 13:05:50.530684 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:50.515686 18073 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:50.499327 18136 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.535681 18136 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:50.536381 18097 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.573s	user 0.018s	sys 0.040s
I20260709 13:05:50.538980 18132 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:50.539724 18132 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:50.540211 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2566) took 1038 ms. Trace:
I20260709 13:05:50.540119 17853 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:50.540917 18132 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.533879 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:50.540706 14021 rpcz_store.cc:276] 0709 13:05:49.501658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.501790 (+   132us) service_pool.cc:224] Handling call
0709 13:05:50.540012 (+1038222us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b540029a9d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:50.540141 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2003496320}
I20260709 13:05:50.541663 17853 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:50.557559 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:50.558959 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:50.562410 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:50.572541 13869 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:05:50.499516 17817 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:50.574821 13869 pending_rounds.cc:85] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 9
I20260709 13:05:50.576901 13869 pending_rounds.cc:107] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:05:50.577250 13869 raft_consensus.cc:2889] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:50.587153 14021 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:05:50.604506 14021 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:50.616030 14021 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:50.692278 17817 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.693089 17817 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.693856 17838 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.694695 17838 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.695669 17817 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:50.696347 17817 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:50.702523 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2594) took 1298 ms. Trace:
I20260709 13:05:50.703660 13868 rpcz_store.cc:276] 0709 13:05:49.404404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.442262 (+ 37858us) service_pool.cc:224] Handling call
0709 13:05:49.468854 (+ 26592us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:49.721118 (+252264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:49.721163 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:49.757273 (+ 36110us) log.cc:844] Serialized 52 byte log entry
0709 13:05:49.902505 (+145232us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:49.902569 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.902669 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:50.702185 (+799516us) raft_consensus.cc:1685] finished
0709 13:05:50.702219 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:50.702435 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90497,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":13059,"thread_start_us":319336,"threads_started":2,"wal-append.queue_time_us":206368}
W20260709 13:05:50.497185 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.533917 18116 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.543111 18073 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:50.737761 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 203961us
I20260709 13:05:50.741899 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:05:50.738898 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.702633 18075 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.743923 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:50.744819 14021 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:50.701421 18151 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:50.726253 18149 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.731755 18150 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.754176 13915 leader_election.cc:400] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6.
I20260709 13:05:50.758694 13915 leader_election.cc:403] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:05:50.753330 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2587) took 1357 ms. Trace:
I20260709 13:05:50.760293 13941 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:50.762439 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:50.764667 13944 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:05:50.766821 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.740903 18067 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.741192 17893 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:50.772547 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 779) took 1282 ms. Trace:
I20260709 13:05:50.773757 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:50.770375 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:50.767820 14524 connection.cc:256] RPC call timeout handler was delayed by -4.6460403720000159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
W20260709 13:05:50.775767 14524 connection.cc:256] RPC call timeout handler was delayed by -4.5281403720000526s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17300000000000001s for a response
W20260709 13:05:50.777667 14524 connection.cc:256] RPC call timeout handler was delayed by -4.1753403720000506s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
W20260709 13:05:50.778041 14524 connection.cc:256] RPC call timeout handler was delayed by -4.1492403720001221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
I20260709 13:05:50.782181 13941 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:50.774005 14551 rpcz_store.cc:276] 0709 13:05:49.490070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.491139 (+  1069us) service_pool.cc:224] Handling call
0709 13:05:49.494672 (+  3533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:49.494962 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:49.527410 (+ 32448us) log.cc:844] Serialized 14 byte log entry
0709 13:05:50.771992 (+1244582us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b18003fb9e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c0348a40 00007fb2c03434e4 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287 00007fb2c344e208
0709 13:05:50.772185 (+   193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:50.772238 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:50.772288 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.772360 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:50.772487 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9674,"prepare.queue_time_us":40297,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"spinlock_wait_cycles":8103808}
W20260709 13:05:50.786357 14524 connection.cc:256] RPC call timeout handler was delayed by -4.1051403720000508s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
I20260709 13:05:50.762564 18072 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:50.767151 17838 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:50.782990 18148 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.294s	user 0.001s	sys 0.001s
I20260709 13:05:50.791620 14015 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:50.813717 13944 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:50.776355 17817 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:50.749764 18075 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:50.791780 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2546) took 2172 ms. Trace:
I20260709 13:05:50.771709 18136 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:50.821851 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.778483 18127 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:50.792147 17899 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.249s	user 0.016s	sys 0.000s
I20260709 13:05:50.790506 18074 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:50.783277 18151 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 5. Current term is 6.
I20260709 13:05:50.821167 18059 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:50.854270 17919 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:50.785228 18114 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:50.762374 18149 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.884868 14560 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:50.830524 13945 rpcz_store.cc:276] 0709 13:05:48.619541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.719223 (+ 99682us) service_pool.cc:224] Handling call
0709 13:05:48.719588 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:48.789458 (+ 69870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.791486 (+2002028us) spinlock_profiling.cc:243] Waited 65.4 ms on lock 0x7b54001c7854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:50.791702 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150526720}
I20260709 13:05:50.837047 13942 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:05:50.771345 18150 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:50.859011 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 72001.5us
W20260709 13:05:50.860787 18077 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.964s	user 0.004s	sys 0.021s
I20260709 13:05:50.832240 17817 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:50.838963 18127 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.864369 18105 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.826380 17838 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:50.839213 18080 raft_consensus.cc:3055] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:50.776827 18067 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.870468 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:50.908349 14015 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 5.
I20260709 13:05:50.877379 13944 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:50.891966 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:50.788008 18091 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:50.895505 14560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 728) took 1297 ms. Trace:
I20260709 13:05:50.897305 18116 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:50.898355 18034 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:50.876448 18142 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.304s	user 0.000s	sys 0.002s
I20260709 13:05:50.926474 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2714) took 1032 ms. Trace:
I20260709 13:05:50.927248 18112 raft_consensus.cc:3055] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:50.902859 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:50.927726 18112 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:50.928275 13942 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:50.928678 18112 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.927175 13872 rpcz_store.cc:276] 0709 13:05:49.894431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.313054 (+418623us) service_pool.cc:224] Handling call
0709 13:05:50.313402 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.355924 (+ 42522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:50.355969 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:50.369572 (+ 13603us) log.cc:844] Serialized 26 byte log entry
0709 13:05:50.724543 (+354971us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:50.757336 (+ 32793us) log.cc:844] Serialized 14 byte log entry
0709 13:05:50.766196 (+  8860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.766282 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:50.926143 (+159861us) raft_consensus.cc:1685] finished
0709 13:05:50.926176 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:50.926386 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43190,"prepare.queue_time_us":32536,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":39366,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":17348,"thread_start_us":299424,"threads_started":1,"wal-append.queue_time_us":506615}
W20260709 13:05:50.933302 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.899802 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:50.799108 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 269200us
I20260709 13:05:50.907460 18127 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:50.909654 13950 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
W20260709 13:05:50.902328 18049 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:05:50.907938 18150 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:50.905313 18080 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:50.905092 17838 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:50.913543 18075 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:50.919624 18113 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:50.919416 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:50.986078 14015 raft_consensus.cc:2410] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:50.987092 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:50.987766 14015 raft_consensus.cc:2368] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11.
I20260709 13:05:50.909384 18105 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:50.970172 18116 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:50.990234 18105 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:50.990520 18116 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:50.992803 18116 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:50.998855 18080 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.000085 13872 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:51.000476 18080 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:50.759361 13873 rpcz_store.cc:276] 0709 13:05:49.395412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.395561 (+   149us) service_pool.cc:224] Handling call
0709 13:05:49.418550 (+ 22989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:49.418820 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:50.349142 (+930322us) log.cc:844] Serialized 14 byte log entry
0709 13:05:50.752937 (+403795us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:50.752994 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:50.753041 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.753110 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:50.753260 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":243520,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":3631,"raft.queue_time_us":3842,"thread_start_us":1289365,"threads_started":2}
I20260709 13:05:51.002266 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.002975 13869 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:50.895716 18114 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:51.009193 14016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2565) took 1508 ms (client timeout 1868 ms). Trace:
I20260709 13:05:51.011328 14022 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:51.000480 13872 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:50.948062 18112 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:51.015236 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 2.17379e+06us
W20260709 13:05:51.019454 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 115129us
I20260709 13:05:51.021170 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:51.021281 18111 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:51.022275 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.022794 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.023098 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.023391 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.023561 17817 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:51.023782 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.009783 14016 rpcz_store.cc:269] 0709 13:05:49.501313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.501469 (+   156us) service_pool.cc:224] Handling call
0709 13:05:51.008953 (+1507484us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540021cfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:51.009114 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394893056}
I20260709 13:05:50.911505 14560 rpcz_store.cc:276] 0709 13:05:49.597687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.720523 (+122836us) service_pool.cc:224] Handling call
0709 13:05:49.786345 (+ 65822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.885513 (+1099168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.895440 (+  9927us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3456}
I20260709 13:05:50.969090 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:50.936430 13988 leader_election.cc:336] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.125s (SENT)
I20260709 13:05:50.971937 18058 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:50.976990 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 203302us
I20260709 13:05:50.978004 13942 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:51.037667 13988 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:50.981884 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:51.039033 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 102657us
I20260709 13:05:50.983652 18149 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:50.984282 18073 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:50.980993 17838 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:51.013613 13872 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:51.039700 13988 leader_election.cc:336] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.241s (SENT)
I20260709 13:05:51.072692 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.072542 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.074745 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:51.060218 13838 connection.cc:256] RPC call timeout handler was delayed by -1.373226002000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17390000000000003s for a response
W20260709 13:05:51.075729 13838 connection.cc:256] RPC call timeout handler was delayed by -1.3471260020000955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1768s for a response
W20260709 13:05:51.075992 13838 connection.cc:256] RPC call timeout handler was delayed by -1.336326002000078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
I20260709 13:05:51.077538 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.077682 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:51.078084 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.039601 18122 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:51.079949 14553 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:05:51.083761 13868 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:51.094957 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.079751 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:51.092270 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.097713 14015 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:51.097146 13873 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:51.080302 18122 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.106859 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:51.110176 18044 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:51.110767 18044 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:51.111725 18044 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.113055 18150 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:51.113931 13838 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:51.114589 18067 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:51.072124 18116 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:51.115612 13838 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:51.078783 18075 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.123556 18075 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.116240 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.126562 13942 raft_consensus.cc:2376] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 9. Re-sending same reply.
I20260709 13:05:51.127600 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:51.128156 13942 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:05:51.070858 18049 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:51.130703 18049 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.132814 18049 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:51.133477 13869 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:51.081234 17853 consensus_queue.cc:1059] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:05:51.134390 17853 consensus_queue.cc:1243] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:51.134902 17853 raft_consensus.cc:3055] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:51.135223 17853 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:51.122033 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.136065 17853 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.119184 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.137277 17853 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:51.118144 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.137075 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.138257 14524 leader_election.cc:400] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 7 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 7. Current term is 8.
I20260709 13:05:51.138648 18041 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:51.044397 18058 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.125900 18075 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:51.121945 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:51.139698 18114 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:51.140405 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2021) took 2492 ms. Trace:
I20260709 13:05:51.140741 18157 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:51.140793 13947 rpcz_store.cc:276] 0709 13:05:48.647801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.968918 (+321117us) service_pool.cc:224] Handling call
0709 13:05:49.131617 (+162699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:49.144457 (+ 12840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.140217 (+1995760us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54001487d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:51.140347 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373311232}
I20260709 13:05:51.141557 18157 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:51.142563 18157 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.145828 18165 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:51.146937 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:51.147323 18105 raft_consensus.cc:3055] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:51.147822 18105 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:51.148768 18105 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.117729 18136 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:51.150336 18042 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:51.150512 18105 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:51.150955 18042 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:51.127380 18067 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:51.152261 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2557) took 1805 ms. Trace:
I20260709 13:05:51.115325 18122 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:51.153258 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 44461.6us
I20260709 13:05:51.153535 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:51.167239 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:51.168829 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:51.157022 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:51.158483 13942 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:51.035935 17817 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.170359 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:51.173457 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.163960 13949 rpcz_store.cc:276] 0709 13:05:49.347275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.365154 (+ 17879us) service_pool.cc:224] Handling call
0709 13:05:49.399790 (+ 34636us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:49.505463 (+105673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.152001 (+1646538us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540004c254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:51.152196 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240975744}
I20260709 13:05:51.150988 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1903) took 2443 ms. Trace:
I20260709 13:05:51.174680 14023 rpcz_store.cc:276] 0709 13:05:48.707481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.036679 (+329198us) service_pool.cc:224] Handling call
0709 13:05:49.036999 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:49.399447 (+362448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.150712 (+1751265us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b5400020654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:51.150921 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":703090944}
I20260709 13:05:51.149972 17707 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:51.217010 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:51.229204 14015 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:05:51.231215 17707 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:51.231678 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.232362 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.233007 13942 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:51.232542 17707 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.234045 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.233883 18042 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.234362 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:51.235020 13949 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:05:51.235214 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.235881 13942 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:51.140959 18058 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:51.240283 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71553.1us
I20260709 13:05:51.241453 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:51.243640 13989 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:51.243696 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.244836 13989 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:51.248975 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 74233.5us
I20260709 13:05:51.174955 18114 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:51.173485 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.253638 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:51.170552 18073 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:51.258328 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:51.258447 13947 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:51.261013 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.261337 13949 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:05:51.263176 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.216856 17984 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:51.265554 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.267743 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.250527 18116 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:51.162779 13872 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:51.236624 18034 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
W20260709 13:05:51.165108 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 125427us
I20260709 13:05:51.271157 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:51.276930 14554 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:05:51.284886 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.036711 18091 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:51.250840 17707 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:51.248222 18150 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:51.256234 18127 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:51.253744 18113 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:51.277031 13947 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:51.216137 17817 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.288429 14558 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:51.285985 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:51.305874 13949 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:51.265710 18114 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:51.308264 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:51.290992 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.316210 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.269244 18058 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:51.318552 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.292704 18127 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.272809 18161 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:51.303334 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:51.309015 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.331157 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.319809 13947 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:05:51.321235 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.334367 17726 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:05:51.334470 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:51.340440 14554 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:51.341722 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.292126 18091 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.342407 14554 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:05:51.343397 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.344599 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.345099 18091 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:51.323364 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.293154 18116 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.323004 13838 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:51.346092 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.347569 18116 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.348246 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:51.349116 18112 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:51.349231 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:51.349736 18112 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.350225 18112 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:51.350346 18041 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:51.350939 18041 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.351270 18043 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:51.351893 18043 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:51.352923 18043 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.348842 14554 raft_consensus.cc:2410] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:05:51.359004 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.300987 18151 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:51.351526 18041 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:51.362838 18086 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:05:51.364085 18112 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.365306 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1884) took 2737 ms. Trace:
I20260709 13:05:51.350373 18116 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:51.365723 14019 rpcz_store.cc:276] 0709 13:05:48.628219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.628354 (+   135us) service_pool.cc:224] Handling call
0709 13:05:48.628676 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:48.633577 (+  4901us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:48.659926 (+ 26349us) log.cc:844] Serialized 14 byte log entry
0709 13:05:48.792841 (+132915us) log.cc:844] Serialized 14 byte log entry
0709 13:05:48.793707 (+   866us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:48.841491 (+ 47784us) log.cc:844] Serialized 26 byte log entry
0709 13:05:48.855737 (+ 14246us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:48.893847 (+ 38110us) log.cc:844] Serialized 14 byte log entry
0709 13:05:48.995634 (+101787us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b180033a480. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:49.019282 (+ 23648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.027917 (+  8635us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:49.052774 (+ 24857us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b180033a480. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:51.139420 (+2086646us) spinlock_profiling.cc:243] Waited 8.56 ms on lock 0x7b180033a480. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:51.252691 (+113271us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b180033a480. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:51.365073 (+112382us) raft_consensus.cc:1685] finished
0709 13:05:51.365099 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.365240 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96225,"prepare.queue_time_us":114529,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":128,"raft.queue_time_us":30891,"raft.run_cpu_time_us":481,"raft.run_wall_time_us":19068,"spinlock_wait_cycles":144716032,"thread_start_us":201740,"threads_started":2,"wal-append.queue_time_us":126907}
W20260709 13:05:51.369594 18140 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.848s	user 0.000s	sys 0.002s
I20260709 13:05:51.284555 18059 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:05:51.371412 14558 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:05:51.318787 18114 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:51.377756 14554 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:51.377476 18041 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.325783 17921 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:05:51.378386 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.379773 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:51.380468 14558 raft_consensus.cc:2368] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11.
I20260709 13:05:51.381629 14558 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:51.376525 18112 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:51.383169 18041 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:51.383622 17817 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:51.385216 17838 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.387723 17838 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:51.388732 13945 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:51.389427 18066 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:51.389936 18066 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.390384 18066 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:51.280135 18160 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:05:51.330160 18127 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:51.330821 18161 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.403362 18161 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:51.405659 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.326706 17984 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:51.407425 17984 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.407790 17984 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:51.351128 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 80058us
I20260709 13:05:51.415963 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:51.417522 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.419579 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1901) took 2712 ms. Trace:
I20260709 13:05:51.420085 14018 rpcz_store.cc:276] 0709 13:05:48.706915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.915634 (+208719us) service_pool.cc:224] Handling call
0709 13:05:48.915974 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:49.175590 (+259616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.419266 (+2243676us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b540023b254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:51.419492 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":595215232}
I20260709 13:05:51.421980 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57836 (request call id 16) took 2446 ms. Trace:
I20260709 13:05:51.422449 13871 rpcz_store.cc:276] 0709 13:05:48.970714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.970877 (+   163us) service_pool.cc:224] Handling call
0709 13:05:51.404226 (+2433349us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54000a8f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c342d853 00007fb2c83fb60a 00007fb2c19817d4 00007fb2c1981762 00007fb2c1981691 00007fb2c198160c 00007fb2c19808c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:51.417033 (+ 12807us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355563264}
W20260709 13:05:51.429548 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.431470 13872 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:51.434660 13870 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:51.440582 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.446902 13947 raft_consensus.cc:1217] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Deduplicated request from leader. Original: 8.9->[10.10-10.10]   Dedup: 10.10->[]
I20260709 13:05:51.447628 13947 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:05:51.450114 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:51.451288 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:51.432785 18058 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:51.460330 13949 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:05:51.457525 13872 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 9 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:51.462472 14558 raft_consensus.cc:2376] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 11. Re-sending same reply.
I20260709 13:05:51.462743 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.465973 13950 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:51.470777 13870 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:05:51.472034 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:51.507069 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 57063.5us
I20260709 13:05:51.508314 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:05:51.509863 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.510270 13950 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:51.514750 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.516498 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.517618 14558 raft_consensus.cc:2410] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:05:51.522505 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.550377 13871 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:51.551240 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
W20260709 13:05:51.557871 18080 consensus_meta.cc:287] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.557s	user 0.012s	sys 0.000s
I20260709 13:05:51.559111 14560 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:05:51.561394 18121 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.830s	user 0.000s	sys 0.016s
I20260709 13:05:51.564152 18050 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:51.563913 18114 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.564615 18050 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:51.565172 13943 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:51.465549 17838 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:51.566459 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1896) took 2927 ms. Trace:
I20260709 13:05:51.566854 14020 rpcz_store.cc:276] 0709 13:05:48.639376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.099293 (+459917us) service_pool.cc:224] Handling call
0709 13:05:49.099757 (+   464us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:49.139133 (+ 39376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:49.139164 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:49.139794 (+   630us) log.cc:844] Serialized 52 byte log entry
0709 13:05:49.367285 (+227491us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:49.401166 (+ 33881us) log.cc:844] Serialized 14 byte log entry
0709 13:05:49.401396 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.401467 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:50.747557 (+1346090us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b180033b500. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:51.560736 (+813179us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:51.566204 (+  5468us) raft_consensus.cc:1685] finished
0709 13:05:51.566230 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.566395 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47589,"prepare.queue_time_us":37441,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":58500,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":9805,"spinlock_wait_cycles":28704384,"thread_start_us":212834,"threads_started":1,"wal-append.queue_time_us":456469}
I20260709 13:05:51.565507 18050 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.569767 18050 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:51.571414 18122 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.573731 18122 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:51.575354 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.575455 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 67304.9us
I20260709 13:05:51.576025 13947 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:51.581453 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.576249 13915 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:51.582877 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.585836 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1892) took 2947 ms. Trace:
I20260709 13:05:51.586297 14022 rpcz_store.cc:276] 0709 13:05:48.638406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.534641 (+896235us) service_pool.cc:224] Handling call
0709 13:05:51.011069 (+1476428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.011856 (+   787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.585630 (+573774us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b540021cfd0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:51.585771 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3394455808}
W20260709 13:05:51.588119 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 279895us
I20260709 13:05:51.504659 18116 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.594859 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:51.599473 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.602907 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:51.607201 13872 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:51.625609 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:51.626137 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:51.632431 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:51.592314 17838 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:51.634410 18177 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.284s	user 0.003s	sys 0.000s
I20260709 13:05:51.672377 14022 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:51.649258 18160 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.655072 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.658885 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.662990 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.667059 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.667872 13838 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:51.680230 13943 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:51.669438 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:51.668819 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.680801 13945 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:51.589337 18127 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:51.678807 13839 leader_election.cc:336] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.957s (SENT)
I20260709 13:05:51.681538 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.681708 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.687526 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:51.569677 17984 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.687911 13941 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:51.687978 13945 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:05:51.688891 13839 leader_election.cc:336] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.015s (SENT)
I20260709 13:05:51.683765 18042 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:05:51.684453 18147 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.152s	user 0.003s	sys 0.000s
I20260709 13:05:51.683807 18160 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.685047 17817 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.683766 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:51.697081 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:51.686039 18183 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:51.701454 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.682389 13949 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:51.633824 18151 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:51.688902 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.689064 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.689574 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:51.696033 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 119847us
I20260709 13:05:51.596570 18113 raft_consensus.cc:697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:51.597822 18116 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.734615 14558 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:51.735378 14017 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:51.672834 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:51.708873 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:51.741305 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:51.743901 14016 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:51.746220 13943 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:51.748109 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:51.743111 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:51.748785 13945 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:51.681010 18180 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:51.735981 18173 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.334s	user 0.002s	sys 0.000s
I20260709 13:05:51.692998 18042 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.755347 13941 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:51.768050 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.784583 13941 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:51.698973 18054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.163s	user 0.041s	sys 0.057s
I20260709 13:05:51.689910 18044 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:51.698411 18049 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:05:51.689885 18041 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:05:51.819965 14556 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:05:51.785640 13941 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.823506 13941 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:51.697984 18112 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:05:51.755903 18157 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:51.800325 18054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.277s	user 0.043s	sys 0.061s
I20260709 13:05:51.831872 13941 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:05:51.847211 13941 pending_rounds.cc:85] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 6
I20260709 13:05:51.847621 13941 pending_rounds.cc:107] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:05:51.847929 13941 raft_consensus.cc:2889] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:51.783708 18114 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:51.868168 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.840408 18161 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.844251 18116 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:51.769011 18150 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:51.810274 18042 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:51.763064 13915 leader_election.cc:400] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7.
I20260709 13:05:51.701360 18136 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
I20260709 13:05:51.742163 18183 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.875722 13941 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:51.814960 18159 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.814100 18073 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.901676 13915 leader_election.cc:403] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:51.743822 17817 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.861718 18049 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.912927 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 150060us
I20260709 13:05:51.908483 18159 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.914785 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:51.918599 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.899156 18150 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:05:51.907878 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 103737us
I20260709 13:05:51.920743 14018 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:51.920046 18114 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7.
I20260709 13:05:51.879388 18066 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.922330 18054 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:51.921747 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:51.897850 18043 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.010s	sys 0.006s
W20260709 13:05:51.938716 18043 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.010s	sys 0.006s
I20260709 13:05:51.938410 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:51.940016 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.941543 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:51.947626 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:51.945930 14022 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:51.927565 18150 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:51.932875 14558 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
W20260709 13:05:51.938143 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.960923 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:51.910593 18073 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:51.961812 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:51.918571 14017 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:05:51.946703 18075 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:51.955231 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:51.932809 18054 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.969652 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:51.938181 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 249254us
I20260709 13:05:51.914392 18049 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:51.966118 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:51.974221 13839 leader_election.cc:336] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.085s (SENT)
I20260709 13:05:51.980242 14018 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:05:51.957931 18114 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:51.964102 18165 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.954221 18179 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:05:51.946877 18116 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:05:51.961112 18160 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:52.037354 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.968156 17853 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:51.987465 18174 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.298s	user 0.001s	sys 0.001s
W20260709 13:05:52.029613 18176 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
I20260709 13:05:52.074942 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:51.976315 18098 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:05:52.076807 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 115112us
I20260709 13:05:51.976557 18127 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:51.978153 18054 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:52.103747 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 695) took 2708 ms. Trace:
I20260709 13:05:52.104426 13948 rpcz_store.cc:276] 0709 13:05:49.395106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.401559 (+  6453us) service_pool.cc:224] Handling call
0709 13:05:49.503498 (+101939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:49.504231 (+   733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.101878 (+2597647us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400381650. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:52.103646 (+  1768us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233658112}
I20260709 13:05:52.107092 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:52.077941 13839 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:52.882180 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 804306us
I20260709 13:05:52.883231 13839 leader_election.cc:400] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11.
I20260709 13:05:52.090440 13950 raft_consensus.cc:2410] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:52.112154 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:52.478523 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2031) took 1459 ms. Trace:
I20260709 13:05:52.953115 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:52.953410 13947 rpcz_store.cc:276] 0709 13:05:51.019022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.453316 (+434294us) service_pool.cc:224] Handling call
0709 13:05:51.575761 (+122445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.588766 (+ 13005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.478195 (+889429us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400091350. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:52.478425 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253410688}
I20260709 13:05:52.491696 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:52.954881 13944 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:52.506580 18206 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:52.510390 18208 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.524232 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:52.607822 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:52.635336 13945 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.886s	user 0.009s	sys 0.002s
I20260709 13:05:52.959450 13945 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:52.959619 13949 raft_consensus.cc:2410] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:52.959992 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2048) took 1277 ms. Trace:
I20260709 13:05:52.960332 13945 rpcz_store.cc:276] 0709 13:05:51.682422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.747825 (+ 65403us) service_pool.cc:224] Handling call
0709 13:05:52.959928 (+1212103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:52.961359 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:52.669368 13866 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.979s	user 0.007s	sys 0.020s
W20260709 13:05:52.962247 13866 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.271s	user 0.009s	sys 0.022s
I20260709 13:05:52.962323 18208 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:52.963038 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2613) took 1372 ms. Trace:
I20260709 13:05:52.963263 18214 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.963804 18214 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:52.678519 13943 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.932s	user 0.011s	sys 0.003s
W20260709 13:05:52.170301 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.248284 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:53.004441 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 477) took 1024 ms. Trace:
I20260709 13:05:53.004889 14020 rpcz_store.cc:276] 0709 13:05:51.980310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.201215 (+220905us) service_pool.cc:224] Handling call
0709 13:05:53.004360 (+803145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:52.163794 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:53.009555 14024 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:53.010147 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1921) took 1267 ms. Trace:
I20260709 13:05:53.010504 14024 rpcz_store.cc:276] 0709 13:05:51.742638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.131965 (+389327us) service_pool.cc:224] Handling call
0709 13:05:53.010041 (+878076us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:52.528515 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:52.498447 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 352171us
W20260709 13:05:52.136730 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.016966 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:52.552073 14560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 782) took 1506 ms (client timeout 1586 ms). Trace:
W20260709 13:05:53.020732 14560 rpcz_store.cc:269] 0709 13:05:51.045771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.077267 (+ 31496us) service_pool.cc:224] Handling call
0709 13:05:52.551805 (+1474538us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b54007148d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:52.551991 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1104993536}
W20260709 13:05:52.480623 17838 consensus_meta.cc:287] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.837s	user 0.004s	sys 0.010s
I20260709 13:05:53.024556 17838 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:52.148646 17817 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:52.177102 17841 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:52.126820 17853 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:52.034245 17984 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:52.197317 18049 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:53.034449 13772 catalog_manager.cc:5697] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 6 to 8, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 8 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:52.473869 18050 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.904s	user 0.013s	sys 0.000s
W20260709 13:05:53.037469 13769 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54072 (request call id 138) took 21106 ms (client timeout 15000 ms). Trace:
I20260709 13:05:52.482570 18054 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.039788 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:53.041620 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:52.175261 18073 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:52.167704 18066 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:53.043864 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:53.044281 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:53.045157 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:53.045840 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:53.033227 17853 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:05:53.041810 18054 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:53.047344 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:51.976213 18075 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.057535 18075 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:53.060700 13873 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:52.026602 17946 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:05:51.923939 18113 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.084434 17853 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:53.084928 17853 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:53.085842 17853 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:53.089713 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.065101 18114 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:53.092875 18114 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:53.094230 13989 leader_election.cc:336] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.504s (SENT)
I20260709 13:05:53.094825 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:53.096264 17946 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:52.238511 18042 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:52.082871 18043 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:05:52.531708 18061 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:53.101613 18061 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.102975 18066 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.103590 18066 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:52.134660 18122 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:05:52.245872 18150 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:52.177227 18151 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:52.609521 18157 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:53.112640 18157 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:52.532510 18161 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:52.708210 18184 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.839s	user 0.001s	sys 0.002s
I20260709 13:05:52.027006 18135 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:53.119153 18135 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.119686 18135 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:53.119915 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 788) took 1828 ms. Trace:
I20260709 13:05:53.120393 14552 rpcz_store.cc:276] 0709 13:05:51.291313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.303106 (+ 11793us) service_pool.cc:224] Handling call
0709 13:05:51.303575 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:51.315018 (+ 11443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.315053 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:51.328778 (+ 13725us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.591626 (+262848us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:51.591669 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.591746 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:52.707448 (+1115702us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:53.119411 (+411963us) raft_consensus.cc:1685] finished
0709 13:05:53.119438 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.119766 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11221,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":14773,"spinlock_wait_cycles":48923904,"thread_start_us":153831,"threads_started":1,"wal-append.queue_time_us":319864}
I20260709 13:05:52.117239 18159 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:52.064466 18165 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.123970 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:53.125458 18165 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:52.619374 18187 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.537s	user 0.003s	sys 0.000s
W20260709 13:05:52.200428 18189 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.122s	user 0.001s	sys 0.001s
I20260709 13:05:51.954577 18180 consensus_queue.cc:1059] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 13:05:53.130506 18180 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:53.130854 18180 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:53.131690 18180 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:52.141947 18182 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 13:05:52.086840 18181 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
I20260709 13:05:52.152287 18183 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:53.137501 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2025) took 2120 ms. Trace:
W20260709 13:05:53.082377 13873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2617) took 1489 ms (client timeout 1504 ms). Trace:
I20260709 13:05:53.137916 13942 rpcz_store.cc:276] 0709 13:05:51.017141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.280199 (+263058us) service_pool.cc:224] Handling call
0709 13:05:51.280562 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:51.288073 (+  7511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.288104 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:51.293046 (+  4942us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.491898 (+198852us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:51.491949 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.492035 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:53.137245 (+1645210us) raft_consensus.cc:1685] finished
0709 13:05:53.137267 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.137424 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10706,"raft.queue_time_us":7792,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":10497,"thread_start_us":145386,"threads_started":1,"wal-append.queue_time_us":327730}
I20260709 13:05:53.140458 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:53.141151 13942 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
W20260709 13:05:52.167129 18185 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.247s	user 0.001s	sys 0.001s
W20260709 13:05:53.138132 13873 rpcz_store.cc:269] 0709 13:05:51.593435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.593587 (+   152us) service_pool.cc:224] Handling call
0709 13:05:53.082123 (+1488536us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54002e68d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:53.082289 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3231256960}
I20260709 13:05:53.143898 14015 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:52.166921 18186 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:05:52.122818 18192 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:53.146553 14522 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:52.788385 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:53.147706 14022 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:53.148233 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1926) took 1324 ms. Trace:
I20260709 13:05:53.148567 14022 rpcz_store.cc:276] 0709 13:05:51.824035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.788047 (+964012us) service_pool.cc:224] Handling call
0709 13:05:53.148155 (+360108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:52.865012 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 465) took 2361 ms. Trace:
I20260709 13:05:52.873167 18112 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:53.150729 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 475) took 2371 ms. Trace:
I20260709 13:05:53.151193 14015 rpcz_store.cc:276] 0709 13:05:50.779572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.263041 (+483469us) service_pool.cc:224] Handling call
0709 13:05:51.282476 (+ 19435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.144460 (+1861984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.150456 (+  5996us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b540004b354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:53.150660 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4280282112}
I20260709 13:05:53.151222 18112 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:52.875151 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.156375 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:52.876567 17726 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.157807 18050 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:52.877310 18008 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:53.158483 17726 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:52.892220 13841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.083s	user 0.041s	sys 0.052s
I20260709 13:05:53.159065 18050 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:05:53.159989 13841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.350s	user 0.041s	sys 0.052s
I20260709 13:05:53.160082 18050 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:53.160993 17726 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:53.161605 17726 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:05:52.904990 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.162251 18043 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:05:53.162606 13839 connection.cc:256] RPC call timeout handler was delayed by -1.3938473859999476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2006s for a response
W20260709 13:05:53.163043 13839 connection.cc:256] RPC call timeout handler was delayed by -1.3692133969999531s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:05:53.163606 13839 connection.cc:256] RPC call timeout handler was delayed by -1.2362352629999123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1897s for a response
I20260709 13:05:53.164016 18132 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.164572 18132 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.165226 18041 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:53.165789 18041 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.166227 14551 raft_consensus.cc:3055] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:53.166642 14551 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:53.166674 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:52.904812 18211 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:53.167658 14551 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.162637 18043 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:53.168989 14551 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:53.169348 18043 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.170696 18043 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:53.166239 13869 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:53.171411 18211 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:53.171928 18211 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:52.912489 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1917) took 1234 ms. Trace:
I20260709 13:05:52.924674 18215 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:53.173550 18050 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:53.150296 14021 rpcz_store.cc:276] 0709 13:05:50.503368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.620382 (+117014us) service_pool.cc:224] Handling call
0709 13:05:50.620789 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.745405 (+124616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.864681 (+2119276us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54002eb654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:52.864933 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284558080}
I20260709 13:05:52.958451 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:53.174690 18050 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.175280 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:53.175519 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1928) took 1312 ms. Trace:
I20260709 13:05:53.175940 13947 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:05:53.177080 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:53.177713 13947 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:52.953177 13948 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:53.177819 14022 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:53.178532 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1922) took 1431 ms. Trace:
I20260709 13:05:53.178795 14022 rpcz_store.cc:276] 0709 13:05:51.747470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.149310 (+1401840us) service_pool.cc:224] Handling call
0709 13:05:53.149664 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.178309 (+ 28645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.178483 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:52.964164 13941 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:05:52.964948 18208 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:52.965310 13943 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:53.182224 18159 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:53.182703 13943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2047) took 1500 ms (client timeout 1894 ms). Trace:
W20260709 13:05:53.183153 13943 rpcz_store.cc:269] 0709 13:05:51.682126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.694484 (+ 12358us) service_pool.cc:224] Handling call
0709 13:05:53.182627 (+1488143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:52.592039 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:52.961952 13945 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:05:52.966368 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:53.187507 13950 raft_consensus.cc:2376] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9. Re-sending same reply.
W20260709 13:05:53.187629 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 595620us
I20260709 13:05:53.187757 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:53.188478 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:53.188711 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:53.189069 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:53.189088 13950 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:05:53.189713 13945 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:52.963446 13866 rpcz_store.cc:276] 0709 13:05:51.590528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.592481 (+  1953us) service_pool.cc:224] Handling call
0709 13:05:51.627727 (+ 35246us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:51.628247 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.628292 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:51.635003 (+  6711us) log.cc:844] Serialized 26 byte log entry
0709 13:05:52.962685 (+1327682us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:52.962735 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.962811 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:52.962854 (+    43us) raft_consensus.cc:1685] finished
0709 13:05:52.962865 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.962984 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":126,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7826,"thread_start_us":1271764,"threads_started":1,"wal-append.queue_time_us":823336}
I20260709 13:05:52.970587 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:53.190855 18183 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:53.191068 18112 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:52.971449 18214 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:52.976405 13944 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
W20260709 13:05:52.978469 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.192734 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2587) took 1110 ms. Trace:
I20260709 13:05:53.193120 13944 rpcz_store.cc:276] 0709 13:05:52.082674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.491136 (+408462us) service_pool.cc:224] Handling call
0709 13:05:53.192665 (+701529us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.193667 13838 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:53.194401 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:52.979358 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2619) took 1383 ms. Trace:
I20260709 13:05:53.194988 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:53.195669 18113 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:53.194990 13944 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:53.196273 13987 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:53.196846 18215 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:53.197088 17984 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:52.981909 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.197520 17984 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.191262 18183 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:53.198272 17984 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:53.198473 18183 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:52.986452 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.006343 18160 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.200428 18160 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:53.201876 13914 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.586s (SENT) [suppressed 39 similar messages]
I20260709 13:05:53.184518 18192 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.996398 18195 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.121s	user 0.004s	sys 0.005s
I20260709 13:05:53.202888 18192 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.006865 17919 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.197746 18161 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:53.204241 17919 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.205529 18192 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:53.014894 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1918) took 1337 ms. Trace:
I20260709 13:05:53.227900 14020 rpcz_store.cc:276] 0709 13:05:51.692806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.005850 (+1313044us) service_pool.cc:224] Handling call
0709 13:05:53.014007 (+  8157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.014500 (+   493us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:53.014533 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:53.014587 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:53.014639 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.014709 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.014831 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84}
I20260709 13:05:53.012033 14024 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:05:53.017230 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.486786 14554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59630 (request call id 551) took 1146 ms. Trace:
I20260709 13:05:53.283509 18214 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.018832 18010 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:53.042405 14560 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:53.296936 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:53.037957 13769 rpcz_store.cc:269] 0709 13:05:31.954371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.975984 (+ 21613us) service_pool.cc:224] Handling call
0709 13:05:53.037348 (+21061364us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:45.882371 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:45.883019 (+   648us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:45.883090 (+    71us) write_op.cc:435] Acquired schema lock
0709 13:05:45.883109 (+    19us) tablet.cc:662] Decoding operations
0709 13:05:45.920108 (+ 36999us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:45.920244 (+   136us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:45.920298 (+    54us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:05:45.924693 (+  4395us) tablet.cc:701] Row locks acquired
0709 13:05:45.924765 (+    72us) write_op.cc:260] PREPARE: finished
0709 13:05:45.925053 (+   288us) write_op.cc:270] Start()
0709 13:05:45.942328 (+ 17275us) write_op.cc:276] Timestamp: P: 1783602345924959 usec, L: 0
0709 13:05:45.946360 (+  4032us) op_driver.cc:348] REPLICATION: starting
0709 13:05:46.447075 (+500715us) log.cc:844] Serialized 7679 byte log entry
0709 13:05:46.513723 (+ 66648us) op_driver.cc:464] REPLICATION: finished
0709 13:05:46.523505 (+  9782us) write_op.cc:301] APPLY: starting
0709 13:05:46.523609 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:46.539134 (+ 15525us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:46.539304 (+   170us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:46.675038 (+135734us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:46.675659 (+   621us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:05:46.675738 (+    79us) write_op.cc:312] APPLY: finished
0709 13:05:46.676553 (+   815us) log.cc:844] Serialized 172 byte log entry
0709 13:05:46.694277 (+ 17724us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:46.694752 (+   475us) write_op.cc:454] Released schema lock
0709 13:05:46.694878 (+   126us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:46.695573 (+   695us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:46.695642 (+    69us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2607093,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":9406,"apply.run_cpu_time_us":32724,"apply.run_wall_time_us":172025,"cfile_cache_hit":74,"cfile_cache_hit_bytes":77219,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6214,"cfile_init":2,"lbm_read_time_us":1421,"lbm_reads_lt_1ms":12,"num_ops":19,"prepare.queue_time_us":312922,"prepare.run_cpu_time_us":12096,"prepare.run_wall_time_us":611644,"raft.queue_time_us":8432,"raft.run_cpu_time_us":1626,"raft.run_wall_time_us":1984,"replication_time_us":567102,"thread_start_us":201402,"threads_started":4,"wal-append.queue_time_us":52843,"wal-append.run_cpu_time_us":12460,"wal-append.run_wall_time_us":1177753}]]}
I20260709 13:05:53.048141 13772 catalog_manager.cc:5697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 4 to 7, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 7 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:53.048990 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:53.299284 14558 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:53.050103 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:53.051364 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:53.300345 14556 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:05:53.299640 13772 catalog_manager.cc:5697] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 reported cstate change: term changed from 4 to 7, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 7 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:53.051525 14553 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
W20260709 13:05:53.300872 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 249600us
I20260709 13:05:53.301887 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:05:53.053781 13988 leader_election.cc:336] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.553s (SENT)
I20260709 13:05:53.055302 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:53.305167 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 251401us
I20260709 13:05:53.302042 13772 catalog_manager.cc:5697] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:53.302297 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:53.306942 13988 leader_election.cc:336] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.641s (SENT)
I20260709 13:05:53.078634 18075 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.303402 14524 leader_election.cc:400] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11.
I20260709 13:05:53.309270 14524 leader_election.cc:403] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:05:53.098378 17853 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:53.100989 18042 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:53.103962 18061 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:52.089119 18063 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:05:53.122146 18150 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:53.125509 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:53.113613 18157 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.322403 18075 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:53.323035 18157 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:53.324563 18161 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:53.105955 18066 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:53.128029 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2615) took 1536 ms. Trace:
I20260709 13:05:53.326476 13870 rpcz_store.cc:276] 0709 13:05:51.592027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.592156 (+   129us) service_pool.cc:224] Handling call
0709 13:05:51.592515 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:51.596458 (+  3943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.596495 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:51.603008 (+  6513us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.926093 (+323085us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:51.926156 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.926253 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:52.618625 (+692372us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:53.127670 (+509045us) raft_consensus.cc:1685] finished
0709 13:05:53.127698 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.127953 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4096,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":26235,"spinlock_wait_cycles":8824064,"thread_start_us":271635,"threads_started":1,"wal-append.queue_time_us":316136}
I20260709 13:05:53.328399 13950 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:53.329681 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 703) took 1647 ms. Trace:
I20260709 13:05:53.129335 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2616) took 1536 ms. Trace:
I20260709 13:05:53.330041 13950 rpcz_store.cc:276] 0709 13:05:51.681842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.189975 (+1508133us) service_pool.cc:224] Handling call
0709 13:05:53.190291 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.328858 (+138567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.329417 (+   559us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b5400091854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:53.329615 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317129472}
I20260709 13:05:53.330411 13868 rpcz_store.cc:276] 0709 13:05:51.592810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.592925 (+   115us) service_pool.cc:224] Handling call
0709 13:05:51.593247 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:51.597071 (+  3824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.597101 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:51.625725 (+ 28624us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.908157 (+282432us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:51.908253 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.908329 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:53.129072 (+1220743us) raft_consensus.cc:1685] finished
0709 13:05:53.129112 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.129250 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3709,"raft.queue_time_us":3730,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":26092,"thread_start_us":236888,"threads_started":1,"wal-append.queue_time_us":294520}
I20260709 13:05:53.133399 18180 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:53.332562 18220 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:52.911396 18059 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.168087 18041 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:53.172982 18132 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:52.625186 13915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.004s	sys 0.047s
I20260709 13:05:53.179819 14022 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:53.346717 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1929) took 1447 ms. Trace:
I20260709 13:05:53.347170 14022 rpcz_store.cc:276] 0709 13:05:51.898997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.179320 (+1280323us) service_pool.cc:224] Handling call
0709 13:05:53.179584 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.346369 (+166785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.346646 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.349198 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1933) took 1428 ms. Trace:
W20260709 13:05:53.165110 13839 leader_election.cc:336] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.897s (SENT)
W20260709 13:05:53.350942 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 185856us
I20260709 13:05:53.182842 18159 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.170711 18212 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.185269 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:53.355098 13943 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:53.161980 17726 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:52.967543 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 468) took 2189 ms. Trace:
I20260709 13:05:52.967741 14023 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:53.190388 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 702) took 1508 ms. Trace:
I20260709 13:05:53.197413 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:53.357339 14016 rpcz_store.cc:276] 0709 13:05:50.777902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.034925 (+257023us) service_pool.cc:224] Handling call
0709 13:05:51.035277 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.746652 (+711375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.967286 (+1220634us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b5400411254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:52.967481 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629135104}
I20260709 13:05:53.357883 13941 rpcz_store.cc:276] 0709 13:05:51.681533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.906046 (+224513us) service_pool.cc:224] Handling call
0709 13:05:52.963914 (+1057868us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:53.180179 (+216265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.190199 (+ 10020us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000b9d50. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:53.190330 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2431825280}
W20260709 13:05:53.358496 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 161132us
I20260709 13:05:53.359364 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:53.360790 13987 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:53.197654 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2050) took 1514 ms. Trace:
I20260709 13:05:53.195587 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:53.206363 17919 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:53.007292 14019 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:53.367915 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1915) took 1690 ms. Trace:
I20260709 13:05:53.368315 14019 rpcz_store.cc:276] 0709 13:05:51.677021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.677148 (+   127us) service_pool.cc:224] Handling call
0709 13:05:51.684168 (+  7020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.007120 (+1322952us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b5400295f20. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33aadc0 00007fb2c34351c0 00007fb2c3436231 00007fb2c3432d8f 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c
0709 13:05:53.367573 (+360453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.367826 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7214848}
I20260709 13:05:53.368848 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 474) took 2589 ms. Trace:
I20260709 13:05:53.207000 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2561) took 2463 ms. Trace:
I20260709 13:05:53.369318 14023 rpcz_store.cc:276] 0709 13:05:50.779355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.175864 (+396509us) service_pool.cc:224] Handling call
0709 13:05:52.967458 (+1791594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.357504 (+390046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.368643 (+ 11139us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b5400411250. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:53.368786 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3948707712}
I20260709 13:05:53.369504 13946 rpcz_store.cc:276] 0709 13:05:50.743716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.743828 (+   112us) service_pool.cc:224] Handling call
0709 13:05:50.744163 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.974614 (+230451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:50.974647 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:51.685749 (+711102us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b2c000e5d98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:05:51.686189 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.879885 (+193696us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:51.879945 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.628932 (+748987us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b5400053f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:52.629000 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:52.995676 (+366676us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:53.206746 (+211070us) raft_consensus.cc:1685] finished
0709 13:05:53.206768 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.206913 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":19397,"spinlock_wait_cycles":627261952,"thread_start_us":193270,"threads_started":1,"wal-append.queue_time_us":188260}
I20260709 13:05:53.356048 14022 rpcz_store.cc:276] 0709 13:05:51.920768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.348139 (+1427371us) service_pool.cc:224] Handling call
0709 13:05:53.348524 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.348781 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.348817 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:53.348869 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:53.348927 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.348990 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.349128 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.372491 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2053) took 1686 ms. Trace:
I20260709 13:05:53.013401 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1923) took 1256 ms. Trace:
I20260709 13:05:53.375567 14018 rpcz_store.cc:276] 0709 13:05:51.756755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.528147 (+771392us) service_pool.cc:224] Handling call
0709 13:05:53.013318 (+485171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.202920 18160 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:53.210315 18216 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.195125 13875 rpcz_store.cc:276] 0709 13:05:51.596328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.596495 (+   167us) service_pool.cc:224] Handling call
0709 13:05:51.631569 (+ 35074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.631853 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:51.902422 (+270569us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.473705 (+571283us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b18000badc0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:52.904854 (+431149us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b18000badc0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c0348a40 00007fb2c03434e4 00007fb2c0343455 00007fb2c6e2b3c7 00007fb2c343d9a7 00007fb2c344d0bf 00007fb2c344d038
0709 13:05:52.905203 (+   349us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.978891 (+ 73688us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:52.978959 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:52.979020 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.979103 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.979275 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":838394,"prepare.queue_time_us":1280534,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":4448,"raft.queue_time_us":40929,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":239,"spinlock_wait_cycles":50158208,"thread_start_us":388109,"threads_started":3,"wal-append.queue_time_us":216912}
I20260709 13:05:53.212402 18049 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:53.213610 18211 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:53.213814 18044 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:53.175908 14021 rpcz_store.cc:276] 0709 13:05:51.863090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.174605 (+1311515us) service_pool.cc:224] Handling call
0709 13:05:53.174950 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.175164 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.175192 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:53.175238 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:53.175293 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.175351 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.175457 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.197463 18215 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.216240 17984 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.222352 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:53.230835 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.230023 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1920) took 1531 ms. Trace:
I20260709 13:05:53.239084 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.244360 13947 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:53.393654 14024 rpcz_store.cc:276] 0709 13:05:51.698811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.011262 (+1312451us) service_pool.cc:224] Handling call
0709 13:05:53.011662 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.229666 (+218004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.229942 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.245925 14551 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:53.395153 13873 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:05:53.249529 13948 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:05:53.396896 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2586) took 1340 ms. Trace:
I20260709 13:05:53.397313 13948 rpcz_store.cc:276] 0709 13:05:52.056980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.111777 (+ 54797us) service_pool.cc:224] Handling call
0709 13:05:53.396825 (+1285048us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.249977 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1931) took 1343 ms. Trace:
I20260709 13:05:53.400461 14020 rpcz_store.cc:276] 0709 13:05:51.907064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.229334 (+1322270us) service_pool.cc:224] Handling call
0709 13:05:53.249178 (+ 19844us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.249436 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.249463 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:53.249556 (+    93us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:53.249614 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.249709 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.249885 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:53.267015 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 65149.1us
W20260709 13:05:53.271663 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 78075.8us
I20260709 13:05:53.377223 18180 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:53.017649 18206 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.293654 14554 rpcz_store.cc:276] 0709 13:05:51.340012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.358692 (+ 18680us) service_pool.cc:224] Handling call
0709 13:05:52.486561 (+1127869us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b5400584b54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:52.486709 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41406208}
I20260709 13:05:53.301180 14553 pending_rounds.cc:85] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting all ops after (but not including) 9
I20260709 13:05:53.053443 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:53.380870 17919 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:53.410499 14555 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:53.069115 14561 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:05:53.290252 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 95582.2us
I20260709 13:05:53.078464 18219 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:05:53.307956 13988 leader_election.cc:336] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.658s (SENT)
I20260709 13:05:53.310935 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2049) took 1628 ms. Trace:
I20260709 13:05:53.303848 18208 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:53.414793 13949 rpcz_store.cc:276] 0709 13:05:51.682723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.191235 (+1508512us) service_pool.cc:224] Handling call
0709 13:05:53.191534 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.191777 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:53.253465 (+ 61688us) log.cc:844] Serialized 14 byte log entry
0709 13:05:53.310537 (+ 57072us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:53.310599 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:53.310666 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.310742 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.310868 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":111287,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":667,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":887,"spinlock_wait_cycles":1355520,"thread_start_us":117985,"threads_started":2,"wal-append.queue_time_us":61083}
I20260709 13:05:53.314852 18122 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11.
I20260709 13:05:53.302448 18010 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:53.418215 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:53.332242 13950 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:53.421231 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2051) took 1738 ms. Trace:
I20260709 13:05:53.421341 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:53.421604 13950 rpcz_store.cc:276] 0709 13:05:51.683277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.331311 (+1648034us) service_pool.cc:224] Handling call
0709 13:05:53.331883 (+   572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.420953 (+ 89070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.421174 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.421607 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:53.334115 13772 catalog_manager.cc:5697] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:53.342172 18059 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.173403 14017 rpcz_store.cc:276] 0709 13:05:51.677554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.862308 (+1184754us) service_pool.cc:224] Handling call
0709 13:05:52.862725 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:52.862962 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:52.874388 (+ 11426us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.911975 (+ 37587us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:52.912038 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:52.912115 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.912218 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.912408 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7756,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":32060,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":128,"thread_start_us":47522,"threads_started":2,"wal-append.queue_time_us":35347}
W20260709 13:05:53.349793 13915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.400s	user 0.009s	sys 0.048s
I20260709 13:05:53.425303 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:53.425732 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 1.47815e+06us
I20260709 13:05:53.386680 18161 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:05:53.354014 18159 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:53.343794 18183 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.335980 18135 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.430665 14552 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:53.420729 18010 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:53.431279 18010 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:53.356371 18212 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.373736 14016 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:53.361382 13944 rpcz_store.cc:276] 0709 13:05:51.683010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.196813 (+1513803us) service_pool.cc:224] Handling call
0709 13:05:53.197126 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.197317 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.197344 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:53.197384 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:53.197435 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.197497 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.197597 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.433826 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1934) took 1508 ms. Trace:
I20260709 13:05:53.434249 14016 rpcz_store.cc:276] 0709 13:05:51.925010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.358713 (+1433703us) service_pool.cc:224] Handling call
0709 13:05:53.359038 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.433543 (+ 74505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.433762 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.372866 13946 rpcz_store.cc:276] 0709 13:05:51.686441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.371657 (+1685216us) service_pool.cc:224] Handling call
0709 13:05:53.371968 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.372172 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.372195 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:53.372236 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:53.372282 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.372335 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.372437 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.377421 14023 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:53.379076 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:53.381671 13943 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:53.435586 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 476) took 1455 ms. Trace:
I20260709 13:05:53.435731 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2052) took 1752 ms. Trace:
I20260709 13:05:53.435952 14023 rpcz_store.cc:276] 0709 13:05:51.979925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.370422 (+1390497us) service_pool.cc:224] Handling call
0709 13:05:53.377020 (+  6598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.435327 (+ 58307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.435529 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.436080 13943 rpcz_store.cc:276] 0709 13:05:51.686127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.368873 (+1682746us) service_pool.cc:224] Handling call
0709 13:05:53.381321 (+ 12448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.435453 (+ 54132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.435670 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.437022 18122 raft_consensus.cc:3055] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:53.437427 18122 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:53.437778 17726 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.438359 13950 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:53.439044 13944 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:53.439343 17726 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:53.439914 13950 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:53.440420 13944 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:05:53.441246 13950 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:53.441566 13944 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:53.417646 18050 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:05:53.442932 18050 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:53.443357 18050 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:53.438374 18122 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.445000 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:53.384167 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2054) took 1697 ms. Trace:
I20260709 13:05:53.445801 13941 rpcz_store.cc:276] 0709 13:05:51.686754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.383288 (+1696534us) service_pool.cc:224] Handling call
0709 13:05:53.383601 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.383808 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.383832 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:53.383874 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:53.383915 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.383973 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.384093 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.385308 13945 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:53.447391 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2055) took 1677 ms. Trace:
I20260709 13:05:53.447727 13945 rpcz_store.cc:276] 0709 13:05:51.770176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.384766 (+1614590us) service_pool.cc:224] Handling call
0709 13:05:53.385057 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.447100 (+ 62043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.447327 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.388494 14019 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:53.448953 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1935) took 1513 ms. Trace:
I20260709 13:05:53.449309 14019 rpcz_store.cc:276] 0709 13:05:51.935755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.369303 (+1433548us) service_pool.cc:224] Handling call
0709 13:05:53.388282 (+ 18979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.448673 (+ 60391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.448868 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:53.383957 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.386394 18066 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:05:53.403725 13838 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:53.285218 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1925) took 1464 ms. Trace:
I20260709 13:05:53.451375 13949 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:05:53.405318 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62640.9us
I20260709 13:05:53.407902 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:53.377065 17946 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:53.413765 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:53.308554 18214 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:53.455389 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 51723.3us
I20260709 13:05:53.414603 13988 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:53.408413 18008 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:05:53.427068 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:53.409582 18180 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:53.410001 14553 pending_rounds.cc:107] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.10
I20260709 13:05:53.474890 14553 raft_consensus.cc:2917] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Aborting config change with OpId 10.10: Aborted: Op aborted by new leader
I20260709 13:05:53.475777 14553 raft_consensus.cc:2889] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:53.430190 18211 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:05:53.445338 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.390734 18044 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:53.450520 14015 rpcz_store.cc:276] 0709 13:05:51.820452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.152408 (+1331956us) service_pool.cc:224] Handling call
0709 13:05:53.223972 (+ 71564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.224327 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:53.250965 (+ 26638us) log.cc:844] Serialized 14 byte log entry
0709 13:05:53.284370 (+ 33405us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:53.284771 (+   401us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:53.284854 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.284955 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.285149 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27169,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":106,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":32885,"threads_started":1,"wal-append.queue_time_us":33319}
I20260709 13:05:53.420202 18220 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:53.425398 18132 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:53.472395 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 164453us
W20260709 13:05:53.479554 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 100540us
I20260709 13:05:53.479770 13838 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:53.480651 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.484318 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:53.485139 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.514461 13874 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:53.486037 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:53.486812 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.472075 13772 catalog_manager.cc:5697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:53.515717 13867 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:53.517086 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:53.495997 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.388338 18216 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.456050 17946 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.520138 14553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49438 (request call id 17) took 7896 ms. Trace:
W20260709 13:05:53.522814 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.523295 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.525713 18116 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:53.526012 13987 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:53.526329 18063 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.475021 18049 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:53.433894 18225 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:53.473080 18113 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:53.537413 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 81025.9us
I20260709 13:05:53.538189 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.539548 13987 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:53.574220 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 119569us
W20260709 13:05:53.527868 13988 leader_election.cc:336] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.939s (SENT)
I20260709 13:05:53.574733 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.487619 18180 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:53.647171 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 156831us
I20260709 13:05:53.475184 18041 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:53.578140 13772 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 7 to 9, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 9 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:53.643584 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:53.444424 18219 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:05:53.414536 18206 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.488744 18211 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:53.495421 17919 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:53.649857 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:53.609905 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 80903.3us
I20260709 13:05:53.516867 18220 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:53.683425 13874 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:53.658473 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.658758 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.688881 13870 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:05:53.575440 17946 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:53.639487 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 135543us
I20260709 13:05:53.660938 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.536101 14553 rpcz_store.cc:276] 0709 13:05:45.580181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.586171 (+  5990us) service_pool.cc:224] Handling call
0709 13:05:47.817627 (+2231456us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b58005bafd0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c338f0aa 00007fb2c3411447 00007fb2c342e70a 00007fb2c342f76a 00007fb2c342e91b 00007fb2c343a03c 00007fb2c34396b3 00007fb2c34319fa 00007fb2c83f94c9 00007fb2c1975c54
0709 13:05:47.997570 (+179943us) log.cc:844] Serialized 501 byte log entry
0709 13:05:53.476354 (+5478784us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":302,"mutex_wait_us":1050759,"raft.queue_time_us":1055774,"raft.run_cpu_time_us":654,"raft.run_wall_time_us":653,"spinlock_wait_cycles":300414848,"thread_start_us":75476,"threads_started":2,"wal-append.queue_time_us":28604,"wal-append.run_cpu_time_us":2324,"wal-append.run_wall_time_us":1004954}
I20260709 13:05:53.670614 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:53.663542 13867 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:05:53.444243 18050 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.535624 18160 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:53.680992 13772 catalog_manager.cc:5697] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:53.578094 18157 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:53.578289 18061 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:53.470609 18150 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.651325 13988 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:53.708272 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:53.708938 14551 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:53.582747 18049 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:53.580523 18112 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.111s
W20260709 13:05:53.682538 18181 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:05:53.535772 18122 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:53.581842 18208 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.579456 17984 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:53.703330 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 54523.5us
I20260709 13:05:53.682314 18211 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:53.644870 18232 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.127s
I20260709 13:05:53.651520 18231 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:53.684952 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.724900 13874 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:53.742803 18061 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:53.781957 13915 connection.cc:256] RPC call timeout handler was delayed by -2.6010936719999336s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
I20260709 13:05:53.682973 17919 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:53.754004 13868 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:53.683971 17853 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:05:53.686831 18225 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:53.776726 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.777990 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.718318 18157 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:53.729034 18113 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:53.744076 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.779098 13772 catalog_manager.cc:5697] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 4 to 11, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 11 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:53.782709 13915 connection.cc:256] RPC call timeout handler was delayed by -2.5947936720000371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260709 13:05:53.774329 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 234909us
W20260709 13:05:53.807732 13915 connection.cc:256] RPC call timeout handler was delayed by -2.4858936719999747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
W20260709 13:05:53.808139 13915 connection.cc:256] RPC call timeout handler was delayed by -2.3688936720000129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
I20260709 13:05:53.808212 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:53.808455 13915 connection.cc:256] RPC call timeout handler was delayed by -2.2887936719999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 13:05:53.808866 13915 connection.cc:256] RPC call timeout handler was delayed by -2.182593671999939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 13:05:53.809204 13915 connection.cc:256] RPC call timeout handler was delayed by -2.0799936720000005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
W20260709 13:05:53.809530 13915 connection.cc:256] RPC call timeout handler was delayed by -2.0250936720000254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21610000000000001s for a response
I20260709 13:05:53.793381 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.810389 13868 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:05:53.810709 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:53.811389 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.581053 18063 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.728721 18049 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:53.725911 18135 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:53.728093 18041 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:53.809216 13874 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
W20260709 13:05:53.809866 13915 connection.cc:256] RPC call timeout handler was delayed by -1.9683936719999338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
W20260709 13:05:53.783185 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 132420us
I20260709 13:05:53.775559 18054 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.795583 17919 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:53.775216 18165 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:53.829037 13915 connection.cc:256] RPC call timeout handler was delayed by -1.9647936720000416s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
I20260709 13:05:53.738291 18150 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:53.829447 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.809664 18157 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:53.806684 18114 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.844345 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:53.756788 18208 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.782647 18159 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:05:53.812134 14551 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:53.883140 14554 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:05:53.860548 14553 raft_consensus.cc:2410] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:53.819375 18050 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:53.845520 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 122816us
I20260709 13:05:53.847664 13772 catalog_manager.cc:5697] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:53.873080 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:53.875098 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:53.879060 13875 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:53.797643 18112 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:05:53.884227 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:53.898535 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.899523 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:53.900265 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:53.901772 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:53.936728 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:53.792135 18231 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.830346 18116 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:53.936784 13875 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:05:53.904870 14553 raft_consensus.cc:2376] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 14. Re-sending same reply.
W20260709 13:05:53.958360 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 194300us
I20260709 13:05:53.939013 14554 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:05:53.940454 13915 leader_election.cc:400] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:05:53.986351 14558 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:05:53.878563 18059 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:54.009840 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:53.997876 14558 pending_rounds.cc:85] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting all ops after (but not including) 7
I20260709 13:05:53.997972 14559 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:54.005414 13772 catalog_manager.cc:5697] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 3 to 6, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 6 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:53.997706 13915 leader_election.cc:403] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:53.775987 18044 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:54.012480 13772 catalog_manager.cc:5697] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 5, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 5 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:53.996567 18226 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:53.898362 18215 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:54.016985 14558 pending_rounds.cc:107] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting uncommitted NO_OP operation due to leader change: 8.8
W20260709 13:05:54.015071 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 119222us
I20260709 13:05:54.040115 14558 raft_consensus.cc:2889] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:54.040596 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.016500 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 76269.4us
I20260709 13:05:54.017750 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:54.018401 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08003065e0 after lost signal to thread 12728
I20260709 13:05:53.847385 18054 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:53.904381 18242 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.805684 18220 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:53.942757 18241 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:53.884642 18114 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:54.041679 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.042936 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:54.043226 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08001b54a0 after lost signal to thread 13807
W20260709 13:05:54.088402 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08001cfce0 after lost signal to thread 13899
W20260709 13:05:54.088837 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08001d7420 after lost signal to thread 13900
I20260709 13:05:54.043470 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:05:54.040115 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 229499us
W20260709 13:05:54.090806 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 47480.2us
I20260709 13:05:54.091674 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:05:54.092641 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08002f2040 after lost signal to thread 13974
W20260709 13:05:54.092898 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.093408 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.997310 18218 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:54.044646 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.070442 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:54.084523 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 87327.6us
I20260709 13:05:54.099753 13772 catalog_manager.cc:5697] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 4, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 4 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:53.938108 18061 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:54.218390 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 148057us
I20260709 13:05:54.230479 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:54.233224 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:54.018512 18243 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:54.071818 18216 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:54.044762 18113 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:54.024762 18244 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:05:54.094141 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08003b3340 after lost signal to thread 14048
W20260709 13:05:54.241705 13971 debug-util.cc:398] Leaking SignalData structure 0x7b08002b1b00 after lost signal to thread 14585
I20260709 13:05:54.032107 18132 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:05:54.096640 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:54.250598 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 149720us
I20260709 13:05:54.024817 18042 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.259800 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:54.261088 13989 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:54.242681 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:54.083443 18151 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:05:54.073495 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 545fc4aeb71d491985e296a0721d905d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Aborted: Op aborted by new leader
I20260709 13:05:54.079809 18212 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:54.004999 18225 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:54.101655 18226 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:54.256903 18259 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:54.080271 18215 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.222s
I20260709 13:05:54.239914 18206 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:54.349821 13772 catalog_manager.cc:5697] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 1 to 5, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 5 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:05:54.044862 18041 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:54.239248 18159 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:05:54.101898 18248 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.253s
I20260709 13:05:54.099793 18242 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:54.214702 18218 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:54.321671 18237 log.cc:927] Time spent T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.518s	user 0.000s	sys 0.001s
I20260709 13:05:54.322978 18243 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:05:54.349630 18259 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:54.312458 18112 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.106s
W20260709 13:05:54.391942 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 300370us
W20260709 13:05:54.388865 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 127835us
I20260709 13:05:54.372021 18253 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:54.445860 13772 catalog_manager.cc:5697] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 5 to 8, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 8 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:54.308982 13758 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 512543us
I20260709 13:05:54.491516 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:54.492274 13772 catalog_manager.cc:5697] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:54.493633 13915 leader_election.cc:400] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 6. Current term is 7.
I20260709 13:05:54.382130 18066 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:54.427656 17984 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:54.274959 18042 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:54.377545 17838 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:54.385924 18073 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:54.267459 18116 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:54.513988 18073 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.514488 18073 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:54.514760 18114 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:54.515407 18114 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:54.516209 18114 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:54.517341 18263 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.197s
W20260709 13:05:54.458462 18236 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.598s	user 0.000s	sys 0.001s
I20260709 13:05:54.502993 17984 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:05:54.512513 18216 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:54.513271 18180 consensus_meta.cc:287] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.794s	user 0.009s	sys 0.014s
I20260709 13:05:54.507929 18066 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:54.518404 18251 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:54.518942 18235 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.563s	user 0.000s	sys 0.001s
W20260709 13:05:54.519754 18246 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.410s	user 0.000s	sys 0.001s
I20260709 13:05:54.520516 18262 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.135s
I20260709 13:05:54.521826 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2624) took 1039 ms. Trace:
W20260709 13:05:54.522324 18249 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:05:54.524040 18008 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:54.528607 17726 consensus_meta.cc:287] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.089s	user 0.006s	sys 0.006s
I20260709 13:05:54.463059 18150 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:54.534443 18165 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.536545 13875 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:54.378500 18240 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:54.453114 18113 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:54.361707 18226 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:54.573164 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 795) took 1165 ms. Trace:
I20260709 13:05:54.489362 18063 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:54.539342 18157 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.641s	user 0.010s	sys 0.000s
I20260709 13:05:54.489037 18208 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:54.544713 18234 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.556s	user 0.000s	sys 0.001s
W20260709 13:05:54.642752 13988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.902s	user 0.017s	sys 0.013s
W20260709 13:05:54.643364 13988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.017s	sys 0.014s
W20260709 13:05:54.644160 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 1.11622e+06us
W20260709 13:05:54.644971 13988 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.945s (SENT)
I20260709 13:05:54.645408 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:54.647074 13988 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:54.596529 13913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.015s	sys 0.005s
W20260709 13:05:54.591848 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 337324us
W20260709 13:05:54.548084 18247 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.290s	user 0.003s	sys 0.000s
I20260709 13:05:54.598292 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2056) took 2754 ms. Trace:
W20260709 13:05:54.657523 13913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.015s	sys 0.005s
I20260709 13:05:54.657783 13947 rpcz_store.cc:276] 0709 13:05:51.844249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.394678 (+1550429us) service_pool.cc:224] Handling call
0709 13:05:53.403696 (+  9018us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:53.409886 (+  6190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.409921 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:53.415735 (+  5814us) log.cc:844] Serialized 26 byte log entry
0709 13:05:53.688728 (+272993us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:53.688770 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.688845 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:54.598019 (+909174us) raft_consensus.cc:1685] finished
0709 13:05:54.598042 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:54.598214 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7559,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":2678,"thread_start_us":243840,"threads_started":1,"wal-append.queue_time_us":366977}
I20260709 13:05:54.513266 17838 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:54.541613 18054 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:54.549810 18075 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:54.616936 13871 rpcz_store.cc:276] 0709 13:05:53.482337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.482494 (+   157us) service_pool.cc:224] Handling call
0709 13:05:53.482945 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:53.483259 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.483285 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:53.484034 (+   749us) log.cc:844] Serialized 26 byte log entry
0709 13:05:53.546634 (+ 62600us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:53.546702 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.546822 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:54.521502 (+974680us) raft_consensus.cc:1685] finished
0709 13:05:54.521528 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:54.521737 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":62015,"threads_started":1,"wal-append.queue_time_us":235603}
I20260709 13:05:54.548127 18214 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:54.621598 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:54.626185 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.596117 17984 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:54.732278 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2058) took 1770 ms. Trace:
I20260709 13:05:54.631222 13872 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:54.494307 13772 catalog_manager.cc:5697] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:54.700987 13875 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
W20260709 13:05:54.639643 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 142532us
I20260709 13:05:54.617616 18231 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:54.631825 18165 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:54.673794 13916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.010s	sys 0.017s
I20260709 13:05:54.615506 18180 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:54.629458 18008 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:54.739792 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2588) took 2139 ms. Trace:
I20260709 13:05:54.743793 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2589) took 2143 ms. Trace:
I20260709 13:05:54.756335 13943 rpcz_store.cc:276] 0709 13:05:52.961736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.436908 (+475172us) service_pool.cc:224] Handling call
0709 13:05:53.437484 (+   576us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:53.443347 (+  5863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.443380 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:53.443910 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 13:05:53.810145 (+366235us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:53.810191 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.810269 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:54.537989 (+727720us) spinlock_profiling.cc:243] Waited 6.36 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:54.731849 (+193860us) raft_consensus.cc:1685] finished
0709 13:05:54.731878 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:54.732189 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":158000,"raft.queue_time_us":6538,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":20412,"spinlock_wait_cycles":14629120,"thread_start_us":365801,"threads_started":1,"wal-append.queue_time_us":584503}
I20260709 13:05:54.762818 13942 rpcz_store.cc:276] 0709 13:05:52.600806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.411739 (+810933us) service_pool.cc:224] Handling call
0709 13:05:53.412058 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:53.412410 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.412439 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:53.417569 (+  5130us) log.cc:844] Serialized 26 byte log entry
0709 13:05:53.670183 (+252614us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:53.670229 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.670311 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:54.743442 (+1073131us) raft_consensus.cc:1685] finished
0709 13:05:54.743474 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:54.743692 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7506,"raft.queue_time_us":101,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":5706,"thread_start_us":227799,"threads_started":1,"wal-append.queue_time_us":312263}
I20260709 13:05:54.762236 13948 rpcz_store.cc:276] 0709 13:05:52.600422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.398286 (+797864us) service_pool.cc:224] Handling call
0709 13:05:53.398725 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:53.399059 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.399097 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:53.399795 (+   698us) log.cc:844] Serialized 26 byte log entry
0709 13:05:53.721275 (+321480us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:53.721674 (+   399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.721766 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:54.739412 (+1017646us) raft_consensus.cc:1685] finished
0709 13:05:54.739446 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:54.739696 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":320935,"threads_started":1,"wal-append.queue_time_us":427037}
I20260709 13:05:54.688189 13988 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:54.637099 14551 rpcz_store.cc:276] 0709 13:05:53.407477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.407656 (+   179us) service_pool.cc:224] Handling call
0709 13:05:54.572864 (+1165208us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b5400531bd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:54.573065 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590280576}
I20260709 13:05:54.554281 18212 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
W20260709 13:05:54.756933 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.768153 14522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.003s	sys 0.009s
W20260709 13:05:54.758387 14524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.013s	sys 0.025s
W20260709 13:05:54.768859 14524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.014s	sys 0.025s
W20260709 13:05:54.644456 17946 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.924s	user 0.013s	sys 0.000s
W20260709 13:05:54.629979 18122 consensus_meta.cc:287] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.870s	user 0.006s	sys 0.006s
W20260709 13:05:54.768290 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.757893 13916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.011s	sys 0.017s
W20260709 13:05:54.769569 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 889243us
W20260709 13:05:54.770370 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 82260.1us
I20260709 13:05:54.761226 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:05:54.733304 18250 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:05:54.676257 18063 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:54.776013 13868 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:54.779614 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:05:54.780722 13988 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:54.758132 18066 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:54.542621 18259 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:54.779263 13914 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:54.785737 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2057) took 1824 ms. Trace:
W20260709 13:05:54.789163 13840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.002s	sys 0.012s
I20260709 13:05:54.788247 13946 rpcz_store.cc:276] 0709 13:05:52.961474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.435398 (+473924us) service_pool.cc:224] Handling call
0709 13:05:53.435736 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:53.652223 (+216487us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:53.652274 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:53.669651 (+ 17377us) log.cc:844] Serialized 26 byte log entry
0709 13:05:53.810539 (+140888us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:53.810580 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.575898 (+765318us) spinlock_profiling.cc:243] Waited 85.5 ms on lock 0x7b54000f5354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:54.575977 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:54.757441 (+181464us) raft_consensus.cc:1685] finished
0709 13:05:54.757467 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:54.785656 (+ 28189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":130370,"raft.queue_time_us":131162,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":12498,"spinlock_wait_cycles":196713728,"thread_start_us":103047,"threads_started":1,"wal-append.queue_time_us":388831}
W20260709 13:05:54.789623 13840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.002s	sys 0.012s
I20260709 13:05:54.675186 18240 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:54.744330 18075 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:54.790692 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:54.790836 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:54.792426 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:54.794384 13871 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:54.795660 13870 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:54.796412 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2637) took 1218 ms. Trace:
I20260709 13:05:54.797150 13870 rpcz_store.cc:276] 0709 13:05:53.578168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.769499 (+1191331us) service_pool.cc:224] Handling call
0709 13:05:54.796326 (+ 26827us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:54.771281 14522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.004s	sys 0.010s
W20260709 13:05:54.792822 14555 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.009s	sys 0.005s
W20260709 13:05:54.804204 14555 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.009s	sys 0.005s
I20260709 13:05:54.665289 18157 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:54.808504 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 565860us
I20260709 13:05:54.809134 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:54.811141 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:54.811504 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:54.811875 14551 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:54.777294 18160 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:54.812913 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.823997 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:54.824383 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.815964 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:54.817853 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2718) took 1327 ms. Trace:
I20260709 13:05:54.818279 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:54.820179 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:54.870787 14021 raft_consensus.cc:2368] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 9.
I20260709 13:05:54.870929 14022 raft_consensus.cc:2410] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:54.871768 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:54.872354 14021 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:54.823494 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:05:54.844549 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.874311 13868 rpcz_store.cc:276] 0709 13:05:53.490771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.775296 (+1284525us) service_pool.cc:224] Handling call
0709 13:05:54.775741 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:54.786287 (+ 10546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.817791 (+ 31504us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:54.790779 18116 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:54.879835 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:54.785200 17946 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:54.800598 18150 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:54.852634 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:54.814007 18160 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:54.906008 13942 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:54.825938 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:54.784689 18261 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:05:54.857407 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:54.858021 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:54.858878 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:54.805864 18243 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:54.918027 14522 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:54.875756 14018 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
W20260709 13:05:54.875978 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 52576.4us
I20260709 13:05:54.876191 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:54.919780 14024 raft_consensus.cc:2368] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 11.
I20260709 13:05:54.877377 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:54.824724 13948 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:05:54.881052 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:54.810449 18253 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.794550 18122 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:54.879383 18278 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:54.879660 18061 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.890632 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:54.890960 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:54.893661 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:54.896159 18240 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:54.897338 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.898432 14554 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:05:54.873813 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:54.805729 18272 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:54.919971 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:54.963251 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:54.922919 14522 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:54.923974 14021 raft_consensus.cc:2368] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13.
I20260709 13:05:54.880407 18242 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:54.924479 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:54.965917 14018 raft_consensus.cc:2368] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 11.
I20260709 13:05:54.826521 18263 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 13:05:54.881620 18218 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:54.880302 18116 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:54.915546 18160 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:54.962954 17919 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.085s	user 0.011s	sys 0.002s
I20260709 13:05:54.882382 18063 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.971637 14551 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
W20260709 13:05:54.968214 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 43869.9us
W20260709 13:05:54.978308 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 55512.9us
I20260709 13:05:54.974330 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:54.989465 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:54.970870 14559 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:54.966549 18276 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.992646 14554 pending_rounds.cc:85] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting all ops after (but not including) 6
I20260709 13:05:54.905100 18112 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:05:54.988821 18271 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.000782 18268 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:55.000953 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:55.001166 18271 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.001371 18268 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.003628 18271 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:54.964591 18272 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.004580 18272 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.005537 18277 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:55.004647 18276 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.006631 18283 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:55.007123 18283 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.007464 18283 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:55.008205 18276 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:55.008800 14024 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:54.918558 18180 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:54.937052 18122 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:55.009730 18122 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.010268 18252 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.010650 18073 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.010922 18252 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.012861 18073 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:54.999224 14554 pending_rounds.cc:107] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:05:55.005556 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 573) took 1159 ms. Trace:
I20260709 13:05:55.011379 17946 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:55.013885 14554 raft_consensus.cc:2889] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:55.016273 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:54.912914 17726 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:05:55.025070 18282 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.013330 18252 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:54.993570 18116 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:54.964181 18243 raft_consensus.cc:697] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:54.992439 18248 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:55.025731 18269 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 13:05:55.028798 18180 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:05:55.025791 18282 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.013463 18226 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:54.963059 18061 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:54.969838 18240 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.032398 18208 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.003661 18268 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:54.909333 18075 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:54.963199 18253 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.044482 18248 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.053352 18208 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.058840 18264 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:55.059381 18264 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.059778 18264 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:55.060652 18157 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:55.060879 18208 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:55.057003 18248 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:55.063002 18183 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:55.063637 18240 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:55.052739 18165 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.068397 18282 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:55.074908 18253 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:55.075429 18281 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:55.077603 18272 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.080353 18278 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:55.080955 18272 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:55.081521 18278 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:54.900105 13839 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:54.898940 13838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.011s	sys 0.020s
W20260709 13:05:55.125768 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 225522us
W20260709 13:05:55.126672 13838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.012s	sys 0.020s
W20260709 13:05:55.127290 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 873536us
I20260709 13:05:55.128077 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:55.133235 13839 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:54.909801 13871 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:55.135463 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2629) took 1649 ms. Trace:
I20260709 13:05:55.135569 13870 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:55.136070 13871 rpcz_store.cc:276] 0709 13:05:53.485320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.767462 (+1282142us) service_pool.cc:224] Handling call
0709 13:05:55.135139 (+367677us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:55.137213 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 144340us
I20260709 13:05:55.138664 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:55.139058 13943 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:05:55.143190 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:55.144740 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 114578us
I20260709 13:05:54.998353 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:55.013965 14552 rpcz_store.cc:276] 0709 13:05:53.845936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.846092 (+   156us) service_pool.cc:224] Handling call
0709 13:05:53.883628 (+ 37536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.266434 (+382806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.266471 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:54.322565 (+ 56094us) log.cc:844] Serialized 26 byte log entry
0709 13:05:54.570303 (+247738us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:54.676386 (+106083us) log.cc:844] Serialized 14 byte log entry
0709 13:05:54.676766 (+   380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.676913 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:55.005320 (+328407us) raft_consensus.cc:1685] finished
0709 13:05:55.005345 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.005486 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":161622,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":501927,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":125054,"thread_start_us":399843,"threads_started":2,"wal-append.queue_time_us":420979}
I20260709 13:05:55.145979 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:55.155800 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.152640 13950 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:55.157783 14555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 572) took 1312 ms. Trace:
W20260709 13:05:55.148012 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 227911us
I20260709 13:05:55.156750 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:55.163378 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:55.162582 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:55.197330 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 64320.2us
I20260709 13:05:55.164064 18165 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:55.160466 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:55.071694 18243 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.206485 13944 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:55.206862 17726 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:55.207681 14553 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.208460 14552 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:55.211072 18132 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:55.149101 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:55.212880 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:55.214277 17838 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.215183 17838 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.211517 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:55.046077 18049 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.214s	user 0.010s	sys 0.005s
I20260709 13:05:55.213937 13942 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:55.219209 18049 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.221613 13869 raft_consensus.cc:2393] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:05:55.167789 18211 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.374s	user 0.016s	sys 0.000s
I20260709 13:05:55.230571 18211 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.232396 13867 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:55.163221 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:55.233566 18211 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:55.225026 18008 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:55.156637 13988 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:55.229751 18226 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:55.230616 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:55.224684 14016 raft_consensus.cc:2368] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11.
I20260709 13:05:55.238739 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:55.239878 18061 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:55.014775 18063 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.211522 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:55.259088 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:55.280844 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 69547.9us
I20260709 13:05:55.282666 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:55.284556 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:55.288543 18266 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:55.288751 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 132340us
I20260709 13:05:55.286041 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:55.289516 18266 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:55.292254 18295 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:55.301095 14553 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:05:55.304111 14021 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:55.258927 18226 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:55.319638 14015 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:55.329885 13866 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:55.331290 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:55.079155 18283 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.319480 18226 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:54.999676 17919 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.337524 13948 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:55.338357 13871 raft_consensus.cc:2410] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:55.338718 18253 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:55.318282 18296 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:05:55.347496 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:55.351142 18010 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.919s	user 0.015s	sys 0.003s
I20260709 13:05:55.354966 18266 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.358022 18278 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:55.358461 18211 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:05:55.358165 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.336823 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.385066 13867 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:05:55.327766 18214 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:55.323498 18073 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:55.423282 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 75956.9us
I20260709 13:05:55.387486 18253 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.414486 13915 leader_election.cc:400] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 9.
W20260709 13:05:55.437394 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.437711 13915 leader_election.cc:403] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:55.415343 18271 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.416005 18289 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.196832 14555 rpcz_store.cc:276] 0709 13:05:53.845552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.845692 (+   140us) service_pool.cc:224] Handling call
0709 13:05:53.896747 (+ 51055us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.807233 (+910486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.807268 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:54.807781 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 13:05:54.936196 (+128415us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:54.941140 (+  4944us) log.cc:844] Serialized 14 byte log entry
0709 13:05:54.967166 (+ 26026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.967248 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:55.157180 (+189932us) raft_consensus.cc:1685] finished
0709 13:05:55.157226 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.157521 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65625,"prepare.queue_time_us":4715,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":3252,"raft.queue_time_us":586861,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":28471,"spinlock_wait_cycles":16063232,"thread_start_us":937448,"threads_started":2,"wal-append.queue_time_us":161859}
I20260709 13:05:55.415325 18218 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.403856 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:55.428717 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2731) took 1900 ms. Trace:
I20260709 13:05:55.430004 18066 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:55.436038 18283 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:55.324036 18061 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:05:55.448500 13875 rpcz_store.cc:276] 0709 13:05:53.527857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.095095 (+567238us) service_pool.cc:224] Handling call
0709 13:05:55.428190 (+1333095us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54002e7cd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:55.428442 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659453824}
I20260709 13:05:55.454887 13942 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
W20260709 13:05:55.481590 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 152127us
I20260709 13:05:55.415421 18251 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.488806 14020 raft_consensus.cc:2368] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 13.
I20260709 13:05:55.443857 18253 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:55.436093 17919 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:55.506747 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:55.455435 18049 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:55.436794 18278 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.402410 18010 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.517854 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:05:55.531978 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 117760us
I20260709 13:05:55.535642 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 599) took 1878 ms. Trace:
I20260709 13:05:55.551010 13874 rpcz_store.cc:276] 0709 13:05:53.519970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.909959 (+1389989us) service_pool.cc:224] Handling call
0709 13:05:54.910418 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.910942 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.910983 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:54.911670 (+   687us) log.cc:844] Serialized 26 byte log entry
0709 13:05:55.325498 (+413828us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:55.361113 (+ 35615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.390042 (+ 28929us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:55.395160 (+  5118us) raft_consensus.cc:1685] finished
0709 13:05:55.395178 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.398100 (+  2922us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1026,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":21725,"thread_start_us":85218,"threads_started":1,"wal-append.queue_time_us":131294}
I20260709 13:05:55.550334 18214 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:55.554265 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:55.560588 14556 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:55.562661 18010 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:55.540473 18271 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:55.542243 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.543056 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 53575.3us
I20260709 13:05:55.542845 18251 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.544701 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:55.518033 13948 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:55.539294 14023 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:55.541409 18159 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 13:05:55.565061 13947 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 5.
I20260709 13:05:55.508432 18066 raft_consensus.cc:697] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:55.553900 13874 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:55.581549 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:55.585337 13948 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:55.514402 18073 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.518157 18218 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:55.615835 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 170058us
I20260709 13:05:55.620000 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:55.625684 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 52315.1us
I20260709 13:05:55.607152 18278 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:55.629137 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:55.624966 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:55.599223 18066 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.628539 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:55.637868 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:55.649253 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 63028.4us
W20260709 13:05:55.657109 18050 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.718s	user 0.012s	sys 0.008s
I20260709 13:05:55.658504 13950 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:55.640906 18251 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:55.666329 13948 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:55.666330 13871 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:55.685822 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:55.685596 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:55.688661 14555 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 13:05:55.683480 17984 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.910s	user 0.017s	sys 0.004s
W20260709 13:05:55.702237 18160 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.686s	user 0.013s	sys 0.002s
I20260709 13:05:55.701634 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:55.700764 18214 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 9.
W20260709 13:05:55.699769 18122 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.689s	user 0.006s	sys 0.011s
W20260709 13:05:55.705437 18248 consensus_meta.cc:287] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.641s	user 0.009s	sys 0.009s
I20260709 13:05:55.711503 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:55.707712 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:55.721446 14522 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:55.728610 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:55.729372 13950 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:55.732452 18312 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:05:55.732787 18264 consensus_meta.cc:287] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.013s
I20260709 13:05:55.518431 18289 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.733510 18264 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.730976 18214 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:55.736411 18289 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:55.719238 18122 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.737160 13942 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:55.738584 14015 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:55.739266 18122 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:55.740392 14559 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:55.742587 18135 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.725294 18248 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.743623 18218 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:55.743351 18135 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.745333 14023 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:55.733122 18312 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:55.768194 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46840.2us
I20260709 13:05:55.715481 18160 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.770547 18063 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:55.771831 14019 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 12: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:55.772501 18160 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:55.758592 18312 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:55.773310 18165 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:55.774501 18135 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:55.736116 18264 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:55.714617 17984 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.783221 18073 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:55.798341 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.799539 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 121716us
I20260709 13:05:55.794389 18295 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:55.796682 18122 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:55.774013 18165 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.802735 18271 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:55.807150 18310 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.275s	user 0.001s	sys 0.002s
I20260709 13:05:55.807876 18122 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.765399 18226 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:55.828559 18116 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.761s	user 0.019s	sys 0.000s
I20260709 13:05:55.829324 18116 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.830308 14017 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:55.832913 18116 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:55.839615 18245 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.453s	user 0.000s	sys 0.003s
I20260709 13:05:55.841614 13915 leader_election.cc:400] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 11.
I20260709 13:05:55.844302 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2620) took 2368 ms. Trace:
I20260709 13:05:55.844729 13873 rpcz_store.cc:276] 0709 13:05:53.475877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.476018 (+   141us) service_pool.cc:224] Handling call
0709 13:05:53.535707 (+ 59689us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:53.536013 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.536047 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:53.536594 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:05:53.884638 (+348044us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:53.884701 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.884799 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:55.838723 (+1953924us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:55.843751 (+  5028us) raft_consensus.cc:1685] finished
0709 13:05:55.843776 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.844085 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64590592,"thread_start_us":206162,"threads_started":1,"wal-append.queue_time_us":599783}
I20260709 13:05:55.818434 18271 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:55.846417 18271 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.849124 18271 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:55.819496 18122 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:55.863185 18165 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:55.865622 13867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2639) took 2175 ms (client timeout 1702 ms). Trace:
W20260709 13:05:55.867086 13867 rpcz_store.cc:269] 0709 13:05:53.690592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.707925 (+ 17333us) service_pool.cc:224] Handling call
0709 13:05:55.865257 (+2157332us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54002e86d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:55.865442 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3502563584}
I20260709 13:05:55.870872 13948 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:55.869655 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2592) took 1047 ms. Trace:
I20260709 13:05:55.876116 18248 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:55.871706 14024 rpcz_store.cc:276] 0709 13:05:54.821905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.001934 (+180029us) service_pool.cc:224] Handling call
0709 13:05:55.004364 (+  2430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.009194 (+  4830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.868734 (+859540us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b5400298bd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:55.869082 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9764608}
I20260709 13:05:55.877781 14558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49438 (request call id 18) took 1125 ms. Trace:
I20260709 13:05:55.878039 13915 leader_election.cc:400] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13.
I20260709 13:05:55.872674 14023 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
W20260709 13:05:55.880417 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 62579.4us
I20260709 13:05:55.882177 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:55.878211 14558 rpcz_store.cc:276] 0709 13:05:54.752670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.753215 (+   545us) service_pool.cc:224] Handling call
0709 13:05:55.877161 (+1123946us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b540018c9d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3439735 00007fb2c34319fa 00007fb2c83f94c9 00007fb2c1975c54 00007fb2c1975be2 00007fb2c1975b11 00007fb2c1975a8c 00007fb2c1974d42 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:55.877536 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72328448}
I20260709 13:05:55.887590 13915 leader_election.cc:400] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 11.
I20260709 13:05:55.877985 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:55.888877 14024 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:05:55.875905 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:55.894572 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.895356 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:55.849820 18271 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 13:05:55.909938 13758 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 4dd265d61fc141a48ca9d3d5c8f52921 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:55.882850 17984 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:55.917207 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:55.926468 18316 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:05:55.926977 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 479) took 1111 ms. Trace:
I20260709 13:05:55.927568 14019 rpcz_store.cc:276] 0709 13:05:54.815283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.815419 (+   136us) service_pool.cc:224] Handling call
0709 13:05:55.926551 (+1111132us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b54000ac154. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:55.926842 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1969533184}
I20260709 13:05:55.929198 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:55.931876 18043 raft_consensus.cc:3055] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:55.932441 18043 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:55.932798 13915 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:55.933808 18043 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:55.936892 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:55.935487 14019 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:55.938920 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:55.934808 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:55.949681 13950 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:55.950624 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:55.952782 13948 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:55.956570 14024 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:55.952199 18043 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:55.934114 18226 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:55.965744 18042 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:55.966503 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:55.967559 18315 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:05:55.966205 18264 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:55.954408 18066 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:05:55.969857 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:55.970126 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2634) took 2476 ms. Trace:
I20260709 13:05:55.970299 18066 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:55.970634 13945 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:55.970770 18066 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:55.972051 18066 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:55.973500 18066 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:55.983431 13838 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:55.983953 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:55.984553 17726 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:55.985009 17726 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:55.985394 17726 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:55.967069 18122 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:55.988903 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.975442 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:55.992084 18259 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:55.992679 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:55.993429 13947 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:55.994886 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.970578 13870 rpcz_store.cc:276] 0709 13:05:53.493269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.800851 (+1307582us) service_pool.cc:224] Handling call
0709 13:05:54.852093 (+ 51242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.136528 (+284435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.969710 (+833182us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54000a7b54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:55.969996 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649995008}
W20260709 13:05:55.979296 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.998617 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.998812 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:55.999688 13873 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:55.999621 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.002264 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 711) took 1131 ms. Trace:
I20260709 13:05:55.982591 18312 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.002677 13943 rpcz_store.cc:276] 0709 13:05:54.874998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.875156 (+   158us) service_pool.cc:224] Handling call
0709 13:05:54.898290 (+ 23134us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.139694 (+241404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.001902 (+862208us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54001469d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:56.002180 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551850752}
I20260709 13:05:56.004191 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.006724 18041 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.007283 18041 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.010447 18314 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:56.010885 18314 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:56.012590 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2716) took 2540 ms. Trace:
I20260709 13:05:56.013433 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2582) took 1196 ms. Trace:
I20260709 13:05:56.013048 13866 rpcz_store.cc:276] 0709 13:05:53.471735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.480830 (+  9095us) service_pool.cc:224] Handling call
0709 13:05:53.481336 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.330746 (+1849410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.012192 (+681446us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b540026b6d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:56.012479 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4246642944}
I20260709 13:05:56.004936 13943 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:56.014034 14020 rpcz_store.cc:276] 0709 13:05:54.816654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.817336 (+   682us) service_pool.cc:224] Handling call
0709 13:05:56.013169 (+1195833us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b54003250d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:56.013344 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1197677312}
W20260709 13:05:56.015720 18322 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:05:56.016366 18312 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.017597 18075 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:56.018043 18122 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:56.019734 14016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2580) took 1203 ms (client timeout 1521 ms). Trace:
W20260709 13:05:56.020205 14016 rpcz_store.cc:269] 0709 13:05:54.815928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.825665 (+  9737us) service_pool.cc:224] Handling call
0709 13:05:56.019481 (+1193816us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b540030cad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:56.019665 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":600241920}
I20260709 13:05:56.022825 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2640) took 1110 ms. Trace:
I20260709 13:05:56.023231 13871 rpcz_store.cc:276] 0709 13:05:54.912186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.481503 (+569317us) service_pool.cc:224] Handling call
0709 13:05:56.022635 (+541132us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400461ed4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:56.022733 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236928896}
I20260709 13:05:56.026327 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 598) took 2506 ms. Trace:
I20260709 13:05:56.026816 13868 rpcz_store.cc:276] 0709 13:05:53.519667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.898600 (+1378933us) service_pool.cc:224] Handling call
0709 13:05:54.899091 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.899463 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.899490 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:54.900450 (+   960us) log.cc:844] Serialized 501 byte log entry
0709 13:05:56.025344 (+1124894us) spinlock_profiling.cc:243] Waited 6.02 ms on lock 0x7b540046d5a0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339d090 00007fb2c33aa6b8 00007fb2c343392b 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c
0709 13:05:56.025458 (+   114us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:56.025528 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.025707 (+   179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.025752 (+    45us) raft_consensus.cc:1685] finished
0709 13:05:56.025768 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.026096 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13839104,"thread_start_us":166397,"threads_started":1,"wal-append.queue_time_us":408860}
I20260709 13:05:56.031766 18281 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:56.032891 18281 raft_consensus.cc:3055] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:56.033252 18281 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:56.035477 14556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 575) took 2161 ms. Trace:
I20260709 13:05:56.029865 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:56.035913 14556 rpcz_store.cc:276] 0709 13:05:53.874158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.874488 (+   330us) service_pool.cc:224] Handling call
0709 13:05:53.874846 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.561349 (+1686503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.035124 (+473775us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54005302d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:56.035372 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3875747072}
I20260709 13:05:56.011991 18314 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:56.021785 13872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2730) took 2494 ms (client timeout 2731 ms). Trace:
W20260709 13:05:56.038147 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 117683us
I20260709 13:05:56.038515 18314 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:56.039417 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:56.039966 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:05:56.037534 13872 rpcz_store.cc:269] 0709 13:05:53.527610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.938756 (+411146us) service_pool.cc:224] Handling call
0709 13:05:56.021421 (+2082665us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54000d9654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:56.021596 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426981760}
I20260709 13:05:56.041065 18278 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:56.041568 18278 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:56.041280 18281 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.043042 18281 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:56.046617 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 111954us
W20260709 13:05:56.047116 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.041340 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:56.028158 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.001061 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.053259 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:56.054337 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:56.057053 14024 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:05:56.058668 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:56.041301 18241 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:56.065816 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:56.075675 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:56.077980 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:56.078388 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:56.053155 18271 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:56.077814 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.105413 13872 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:56.104276 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:56.109030 14019 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:56.110313 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:56.066932 18241 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:56.078533 18010 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:56.142972 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 76204.9us
I20260709 13:05:56.113900 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:56.112859 18059 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.765s	user 0.014s	sys 0.023s
W20260709 13:05:56.146731 18059 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.848s	user 0.015s	sys 0.023s
I20260709 13:05:56.146399 18278 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.148379 18278 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.152060 18010 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.153074 18150 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:56.154742 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 89051.2us
I20260709 13:05:56.158295 18061 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:56.158926 18061 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.159122 18150 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.159394 18061 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:56.166204 14560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 577) took 2224 ms. Trace:
I20260709 13:05:56.167160 18151 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:56.166662 14560 rpcz_store.cc:276] 0709 13:05:53.875932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.876877 (+   945us) service_pool.cc:224] Handling call
0709 13:05:53.903701 (+ 26824us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.108040 (+204339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.108083 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:54.214379 (+106296us) log.cc:844] Serialized 26 byte log entry
0709 13:05:54.471553 (+257174us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:55.836449 (+1364896us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b180023a340. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:05:55.886587 (+ 50138us) spinlock_profiling.cc:243] Waited 74 ms on lock 0x7b180023a340. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c6e2b3c7 00007fb2c343d9a7 00007fb2c344d0bf 00007fb2c344d038
0709 13:05:55.918014 (+ 31427us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.054875 (+136861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.074588 (+ 19713us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.093897 (+ 19309us) raft_consensus.cc:1685] finished
0709 13:05:56.093929 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.100564 (+  6635us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33134,"prepare.queue_time_us":72455,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":103,"raft.queue_time_us":583428,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":16413,"spinlock_wait_cycles":246945920,"thread_start_us":363728,"threads_started":3,"wal-append.queue_time_us":379337}
I20260709 13:05:56.166989 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:56.079114 18218 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:05:56.181646 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:56.173938 13869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2719) took 2682 ms (client timeout 1764 ms). Trace:
I20260709 13:05:56.174635 13874 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:56.175524 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2596) took 1311 ms. Trace:
I20260709 13:05:56.175800 14016 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:56.159937 18150 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.154829 18220 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:56.182865 13869 rpcz_store.cc:269] 0709 13:05:53.491045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.576522 (+ 85477us) service_pool.cc:224] Handling call
0709 13:05:56.173614 (+2597092us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54003acad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:56.173848 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3523933440}
I20260709 13:05:56.185989 14024 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:56.194137 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2600) took 1223 ms. Trace:
I20260709 13:05:56.183923 14023 rpcz_store.cc:276] 0709 13:05:54.864090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.096436 (+1232346us) service_pool.cc:224] Handling call
0709 13:05:56.174891 (+ 78455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.175165 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.175189 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:56.175238 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:56.175278 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.175334 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.175443 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.189538 13943 raft_consensus.cc:2368] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11.
I20260709 13:05:56.195423 13945 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:56.195668 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.202265 18180 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:56.203076 18218 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:56.213502 18218 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:56.214450 14017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2581) took 1398 ms (client timeout 1580 ms). Trace:
I20260709 13:05:56.214618 18218 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.216369 18218 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:56.217339 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2590) took 1396 ms. Trace:
I20260709 13:05:56.142561 18241 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:56.186941 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:56.214960 14017 rpcz_store.cc:269] 0709 13:05:54.816301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.817244 (+   943us) service_pool.cc:224] Handling call
0709 13:05:56.214182 (+1396938us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b540023bc54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:56.214381 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2031804672}
I20260709 13:05:56.210340 14024 rpcz_store.cc:276] 0709 13:05:54.969391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.077978 (+1108587us) service_pool.cc:224] Handling call
0709 13:05:56.185726 (+107748us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.192560 (+  6834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.192835 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.210762 18295 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:56.221155 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 77332us
I20260709 13:05:56.221558 13948 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:56.227267 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.222262 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1936) took 1058 ms. Trace:
I20260709 13:05:56.217756 14021 rpcz_store.cc:276] 0709 13:05:54.821009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.998899 (+177890us) service_pool.cc:224] Handling call
0709 13:05:54.999225 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.304870 (+305645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.217016 (+912146us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54002981d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:56.217254 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":699635968}
I20260709 13:05:56.225644 14553 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:05:56.233381 14016 rpcz_store.cc:276] 0709 13:05:55.149932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.165532 (+1015600us) service_pool.cc:224] Handling call
0709 13:05:56.165996 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.191945 (+ 25949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.207879 (+ 15934us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.175541 17838 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.235345 13870 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:56.238382 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.242146 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:56.242821 13945 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:56.245694 14019 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:05:56.246315 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2606) took 1102 ms. Trace:
I20260709 13:05:56.246640 14019 rpcz_store.cc:276] 0709 13:05:55.143487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.053708 (+910221us) service_pool.cc:224] Handling call
0709 13:05:56.246253 (+192545us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.251626 13868 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:05:56.252054 14021 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:56.252898 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1949) took 1095 ms. Trace:
I20260709 13:05:56.253134 18150 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:56.253648 14019 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:05:56.255393 14019 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:56.253314 14021 rpcz_store.cc:276] 0709 13:05:55.157753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.233637 (+1075884us) service_pool.cc:224] Handling call
0709 13:05:56.251840 (+ 18203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.252620 (+   780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.252822 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.257174 13942 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:56.262584 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.263378 13943 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:56.268276 18276 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:56.268105 17726 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.269089 18276 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:56.270361 18276 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:56.271308 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 201680us
I20260709 13:05:56.273307 18010 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.241356 18061 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.275195 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.276070 13943 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 12: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:56.275974 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.276482 18061 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:56.277086 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:56.277776 13943 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:56.270768 13915 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.521s (SENT)
I20260709 13:05:56.277258 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.279840 13942 raft_consensus.cc:2410] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:56.223804 18180 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:56.286689 13915 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.580s (SENT)
W20260709 13:05:56.296196 13915 leader_election.cc:336] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.650s (SENT)
I20260709 13:05:56.296697 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:56.287281 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:56.298954 18295 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:56.299696 18295 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.299360 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:56.301813 18073 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:56.302456 18073 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:56.290005 14016 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:56.291050 18259 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.303437 18073 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.311345 18259 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.291649 18010 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:56.314397 18010 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.301393 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:56.316593 18010 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.316762 14019 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:05:56.318809 13838 leader_election.cc:400] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11.
W20260709 13:05:56.319083 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.319250 13838 leader_election.cc:403] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:05:56.320312 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2646) took 1195 ms. Trace:
I20260709 13:05:56.320739 13874 rpcz_store.cc:276] 0709 13:05:55.124957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.636580 (+511623us) service_pool.cc:224] Handling call
0709 13:05:56.174416 (+537836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.188544 (+ 14128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.320052 (+131508us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54003acad0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:56.320213 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1205766528}
I20260709 13:05:56.322667 14522 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:56.326191 18159 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.327023 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1951) took 1040 ms. Trace:
I20260709 13:05:56.326836 18159 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.327746 18277 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:05:56.330487 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.289050 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:56.288607 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 187849us
I20260709 13:05:56.327428 14016 rpcz_store.cc:276] 0709 13:05:55.286375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.241247 (+954872us) service_pool.cc:224] Handling call
0709 13:05:56.279021 (+ 37774us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.326723 (+ 47702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.326957 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.295994 18180 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.336833 14023 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:56.337301 13945 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:56.337823 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1940) took 1184 ms. Trace:
I20260709 13:05:56.338785 14522 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:56.338855 14023 rpcz_store.cc:276] 0709 13:05:55.153440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.208700 (+1055260us) service_pool.cc:224] Handling call
0709 13:05:56.248726 (+ 40026us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.337459 (+ 88733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.337764 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.342545 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:56.344933 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 43659.9us
W20260709 13:05:56.345170 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.345034 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.346325 13942 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:56.263010 18041 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.350742 18206 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:56.313880 18259 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:56.356861 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:56.350513 18244 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.848s	user 0.024s	sys 0.061s
I20260709 13:05:56.363040 13874 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:56.357532 13948 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:56.367273 13868 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:56.341284 18122 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.342310 18010 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:05:56.374485 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 55848.4us
I20260709 13:05:56.366792 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.373605 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.379935 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.354597 18157 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.356436 18066 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:56.364445 18244 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.015s	user 0.025s	sys 0.061s
I20260709 13:05:56.375680 13872 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:56.393102 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.354697 18206 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:56.399955 13945 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:05:56.402603 13838 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:56.367919 18295 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:56.399657 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:56.415566 13942 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:05:56.367904 18226 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.385339 18041 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11.
I20260709 13:05:56.422887 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:56.423753 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45423.3us
W20260709 13:05:56.424932 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.360908 18264 consensus_queue.cc:1059] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
W20260709 13:05:56.393443 18253 consensus_meta.cc:287] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.868s	user 0.014s	sys 0.002s
I20260709 13:05:56.404371 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:56.432849 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46938.4us
I20260709 13:05:56.435452 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.401752 18010 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:56.450421 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.452899 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.455047 13871 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:56.399504 18244 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:56.455976 13874 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:56.457226 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:56.464692 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.471182 18010 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.473904 13866 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:56.478820 18244 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.441956 18226 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.482499 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.484123 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.487711 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:56.450773 18161 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:56.490095 18206 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.442342 18253 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.493247 18253 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:56.495335 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.497249 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2598) took 1345 ms. Trace:
I20260709 13:05:56.497718 13949 rpcz_store.cc:276] 0709 13:05:55.151740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.153104 (+  1364us) service_pool.cc:224] Handling call
0709 13:05:55.153761 (+   657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.154506 (+   745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:55.154551 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.003857 (+849306us) spinlock_profiling.cc:243] Waited 77.1 ms on lock 0x7b2c000e5d98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:05:56.004388 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.103256 (+ 98868us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:56.103295 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.103374 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.496747 (+393373us) raft_consensus.cc:1685] finished
0709 13:05:56.496773 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.497053 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":59988,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":278,"spinlock_wait_cycles":177217920,"thread_start_us":98404,"threads_started":1,"wal-append.queue_time_us":357326}
I20260709 13:05:56.499954 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.497282 18135 raft_consensus.cc:3055] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:56.500694 18135 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:56.501595 18135 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.503168 18135 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:56.497283 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.504001 13867 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:05:56.504107 18243 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.484275 18226 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:56.446624 18043 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:56.485714 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:56.383726 18122 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.513232 13942 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:56.504724 18243 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.522143 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2612) took 1056 ms. Trace:
I20260709 13:05:56.529861 13949 rpcz_store.cc:276] 0709 13:05:55.449930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.499429 (+1049499us) service_pool.cc:224] Handling call
0709 13:05:56.499770 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.505501 (+  5731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.505533 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:56.505583 (+    50us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:56.505616 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.505682 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.505796 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.527501 13874 raft_consensus.cc:2410] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:05:56.525182 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 58121.3us
W20260709 13:05:56.537796 13947 consensus_meta.cc:287] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.003s
W20260709 13:05:56.542600 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.542419 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2610) took 1131 ms. Trace:
I20260709 13:05:56.505878 18159 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.547101 13942 rpcz_store.cc:276] 0709 13:05:55.401530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.441058 (+1039528us) service_pool.cc:224] Handling call
0709 13:05:56.467633 (+ 26575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.529210 (+ 61577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.532527 (+  3317us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.523932 18043 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.546146 13947 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:05:56.554136 13874 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:56.558297 18112 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.558123 18295 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:56.558897 18112 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.562139 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1955) took 1206 ms. Trace:
I20260709 13:05:56.562625 14019 rpcz_store.cc:276] 0709 13:05:55.356257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.256087 (+899830us) service_pool.cc:224] Handling call
0709 13:05:56.256433 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.317196 (+ 60763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.561684 (+244488us) spinlock_profiling.cc:243] Waited 60 ms on lock 0x7b540022d654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:56.561988 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138046848}
I20260709 13:05:56.566745 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:56.573364 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:56.574962 14522 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:56.576166 13949 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:56.576289 18231 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:56.576936 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2077) took 1057 ms. Trace:
I20260709 13:05:56.577359 18219 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:56.578377 18219 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:56.579887 18219 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.451043 18157 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.589205 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:56.557426 18043 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:56.595618 18161 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.596465 18161 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:56.543584 18041 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:56.577315 13949 rpcz_store.cc:276] 0709 13:05:55.519625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.550774 (+1031149us) service_pool.cc:224] Handling call
0709 13:05:56.575964 (+ 25190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.576623 (+   659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.576854 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.604768 13950 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:05:56.605129 13871 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:05:56.608704 18243 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.547559 17726 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.611475 18043 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.620460 13838 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:56.623027 13943 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
W20260709 13:05:56.637729 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 339636us
I20260709 13:05:56.576936 18231 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.641531 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.589262 18157 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.644410 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.596469 18122 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.626014 17726 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:56.660364 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:56.661414 13949 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:56.649431 18231 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:56.688758 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.663218 17726 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.485560 18281 raft_consensus.cc:2955] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Committing config change with OpId 14.13: config changed from index 7 to 13, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:05:56.658313 18008 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:56.692875 13915 leader_election.cc:336] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.887s (SENT)
I20260709 13:05:56.700131 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:56.684325 18295 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:56.701035 13946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 708) took 1815 ms (client timeout 1943 ms). Trace:
I20260709 13:05:56.693922 17726 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:56.697988 18008 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.664597 18041 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:56.718696 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 243995us
W20260709 13:05:56.708797 13946 rpcz_store.cc:269] 0709 13:05:54.809551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.811237 (+  1686us) service_pool.cc:224] Handling call
0709 13:05:56.619284 (+1808047us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400086454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:56.624439 (+  5155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3707874048}
I20260709 13:05:56.741557 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1945) took 1585 ms. Trace:
I20260709 13:05:56.744746 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.747128 13950 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:05:56.748091 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2083) took 1190 ms. Trace:
I20260709 13:05:56.748497 13950 rpcz_store.cc:276] 0709 13:05:55.557904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.746250 (+1188346us) service_pool.cc:224] Handling call
0709 13:05:56.746784 (+   534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.747719 (+   935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.748017 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.728509 18066 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:56.742424 14017 rpcz_store.cc:276] 0709 13:05:55.155940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.219561 (+1063621us) service_pool.cc:224] Handling call
0709 13:05:56.241803 (+ 22242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.252844 (+ 11041us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:56.435037 (+182193us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.741022 (+305985us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:56.741108 (+    86us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:56.741191 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.741292 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.741460 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20512,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"thread_start_us":433648,"threads_started":2}
I20260709 13:05:56.751151 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:56.762712 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 69869.3us
I20260709 13:05:56.746899 18218 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:56.737387 18295 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.778183 18114 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:56.797919 18295 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:56.851950 18343 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:05:56.852939 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:56.854013 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 68992.7us
I20260709 13:05:56.854347 18344 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:05:56.855005 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:56.855285 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:56.855509 18344 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:56.855979 18010 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:56.844194 18066 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.856514 18010 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:56.856980 18010 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:56.862388 17984 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:56.862860 17984 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.863204 17984 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:56.865672 18333 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:05:56.866317 18244 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:56.866981 18244 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:56.867769 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2602) took 1657 ms. Trace:
I20260709 13:05:56.867926 18244 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.868234 13944 rpcz_store.cc:276] 0709 13:05:55.210008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.224262 (+ 14254us) service_pool.cc:224] Handling call
0709 13:05:55.224849 (+   587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.225817 (+   968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:55.225874 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.046255 (+820381us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b2c000e5d98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:05:56.056644 (+ 10389us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.422771 (+366127us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:56.422922 (+   151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.422993 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.867399 (+444406us) raft_consensus.cc:1685] finished
0709 13:05:56.867427 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.867694 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":921,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":283,"spinlock_wait_cycles":12362112,"thread_start_us":287643,"threads_started":1,"wal-append.queue_time_us":530758}
I20260709 13:05:56.872949 18010 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.874825 18116 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:56.875705 18116 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:56.876768 18329 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:56.877125 18329 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:56.878700 18342 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:05:56.878077 18329 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.880422 18344 raft_consensus.cc:3055] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:56.880800 18344 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:56.855996 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:56.869451 18244 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:56.899108 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2608) took 1509 ms. Trace:
I20260709 13:05:56.899838 13948 rpcz_store.cc:276] 0709 13:05:55.389391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.397690 (+1008299us) service_pool.cc:224] Handling call
0709 13:05:56.421584 (+ 23894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.443427 (+ 21843us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:56.735230 (+291803us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.898567 (+163337us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:56.898651 (+    84us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:56.898752 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.898849 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.899006 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":406524,"thread_start_us":385884,"threads_started":2}
I20260709 13:05:56.902032 13948 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:56.915962 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2089) took 1248 ms. Trace:
I20260709 13:05:56.917312 13948 rpcz_store.cc:276] 0709 13:05:55.663016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.901470 (+1238454us) service_pool.cc:224] Handling call
0709 13:05:56.901856 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.910938 (+  9082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.911145 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.922525 18054 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.923432 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:56.924186 14553 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:56.926940 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:56.927479 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2080) took 1406 ms. Trace:
I20260709 13:05:56.927965 13949 rpcz_store.cc:276] 0709 13:05:55.520738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.662446 (+1141708us) service_pool.cc:224] Handling call
0709 13:05:56.686588 (+ 24142us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.691710 (+  5122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.691745 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.704669 (+ 12924us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.846559 (+141890us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:56.846619 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.846693 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.927241 (+ 80548us) raft_consensus.cc:1685] finished
0709 13:05:56.927265 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.927402 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4911,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":4675,"thread_start_us":140985,"threads_started":1,"wal-append.queue_time_us":220302}
W20260709 13:05:56.937470 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.937050 17984 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.897814 18220 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:56.923235 18054 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.964026 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.964968 13870 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
W20260709 13:05:56.965955 18349 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.114s	user 0.001s	sys 0.001s
W20260709 13:05:56.966574 18346 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.171s	user 0.002s	sys 0.000s
I20260709 13:05:56.968189 14561 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:56.941795 13949 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:56.947100 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:56.969851 14015 raft_consensus.cc:2393] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:56.970788 13948 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:56.948002 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:56.946652 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:56.963084 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:56.881840 18344 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:56.945571 14017 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:05:56.948904 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:56.952271 18218 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:56.953612 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:56.973348 13873 raft_consensus.cc:2376] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11. Re-sending same reply.
I20260709 13:05:56.954979 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2093) took 1237 ms. Trace:
W20260709 13:05:56.975229 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.975731 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2092) took 1263 ms. Trace:
I20260709 13:05:56.976310 13949 rpcz_store.cc:276] 0709 13:05:55.712425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.929698 (+1217273us) service_pool.cc:224] Handling call
0709 13:05:56.941426 (+ 11728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.975455 (+ 34029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.975661 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.946722 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:56.977367 14023 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:05:56.958007 18350 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 13:05:56.964354 18351 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.112s	user 0.004s	sys 0.000s
I20260709 13:05:56.966454 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.978454 13874 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:56.959177 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:56.980073 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.962646 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.947654 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:56.980845 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:56.945478 14022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2587) took 2125 ms (client timeout 1887 ms). Trace:
I20260709 13:05:56.974432 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2091) took 1263 ms. Trace:
I20260709 13:05:56.950769 18008 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.914388 18010 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.990132 14018 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:56.965710 18054 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:56.956439 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.996666 13867 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:56.997085 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:56.984277 14022 rpcz_store.cc:269] 0709 13:05:54.820077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.876961 (+ 56884us) service_pool.cc:224] Handling call
0709 13:05:56.945062 (+2068101us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54002ebb54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:56.945250 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4243223808}
I20260709 13:05:56.980036 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.975466 13945 rpcz_store.cc:276] 0709 13:05:55.717968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.953859 (+1235891us) service_pool.cc:224] Handling call
0709 13:05:56.954276 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.954569 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.954598 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:56.954661 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:56.954716 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.954784 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.954913 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64768}
I20260709 13:05:56.993182 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:56.993503 18344 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:56.960582 18329 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:56.994225 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:56.984355 13948 rpcz_store.cc:276] 0709 13:05:55.710724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.918242 (+1207518us) service_pool.cc:224] Handling call
0709 13:05:56.970589 (+ 52347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.974180 (+  3591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.974374 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:57.001081 13868 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 12: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:57.001761 13873 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:56.998829 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:56.963048 18348 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.112s	user 0.002s	sys 0.000s
I20260709 13:05:56.994963 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:56.995390 18042 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:05:56.993256 18218 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:56.993784 18073 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:57.006423 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:57.006994 14522 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:57.010439 13949 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:57.009855 14553 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:57.014976 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:57.015841 13870 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:57.018159 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:57.018391 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.019232 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:57.020041 13867 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:57.025300 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:57.005826 18241 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.783s	user 0.006s	sys 0.009s
I20260709 13:05:57.032294 14015 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:57.000145 17726 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.029240 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:57.030441 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.031548 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.034595 14020 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 12: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:57.031746 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:57.035316 14016 raft_consensus.cc:2393] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:57.044518 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.040150 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.048264 14019 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:57.041708 14023 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 12. Current term is 13. Ops: [11.10-12.11]
I20260709 13:05:57.005802 18008 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:57.049504 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:57.053639 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:57.049764 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2078) took 1529 ms. Trace:
I20260709 13:05:57.054313 13949 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:57.051118 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.054934 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.053975 13945 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:05:57.015209 18054 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:57.014492 18073 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:57.060184 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.057955 13868 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:57.062182 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:57.054297 13947 rpcz_store.cc:276] 0709 13:05:55.520001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.555324 (+1035323us) service_pool.cc:224] Handling call
0709 13:05:56.555838 (+   514us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:56.613458 (+ 57620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.613505 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:56.614418 (+   913us) log.cc:844] Serialized 52 byte log entry
0709 13:05:56.754590 (+140172us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:56.754656 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.754768 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.963700 (+208932us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:57.040509 (+ 76809us) raft_consensus.cc:1685] finished
0709 13:05:57.040533 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.049694 (+  9161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2298,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":149,"spinlock_wait_cycles":31850368,"thread_start_us":139572,"threads_started":1,"wal-append.queue_time_us":237255}
I20260709 13:05:57.066243 13875 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:57.060876 14020 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:05:57.040158 18159 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:57.068365 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:57.068854 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:57.040426 18180 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:57.071348 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:57.072208 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.005779 18295 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:05:57.082429 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.044909 18010 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:57.076884 13945 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:57.087065 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2094) took 1340 ms. Trace:
I20260709 13:05:57.087561 13872 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:57.087464 13945 rpcz_store.cc:276] 0709 13:05:55.746296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.056440 (+1310144us) service_pool.cc:224] Handling call
0709 13:05:57.076550 (+ 20110us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.086743 (+ 10193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.086990 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:57.084417 13874 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:57.070744 18157 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:05:57.067937 18054 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.065412 18073 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:57.091326 18054 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:57.076150 18159 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:57.077769 18253 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:57.094260 18066 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:57.078712 18180 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.095575 18180 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:57.094976 18159 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:57.096632 18159 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:57.097728 13866 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:05:57.098979 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.100198 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.103703 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:57.104589 14559 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 12: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:57.104938 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:57.105816 14556 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:05:57.040257 18241 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.108431 18241 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:57.109192 14024 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:57.110652 14553 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:57.115238 18043 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:57.117841 18112 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:57.120193 13870 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:05:57.120993 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1956) took 1764 ms. Trace:
I20260709 13:05:57.127825 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2079) took 1607 ms. Trace:
W20260709 13:05:57.129539 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.128294 13942 rpcz_store.cc:276] 0709 13:05:55.520385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.650610 (+1130225us) service_pool.cc:224] Handling call
0709 13:05:56.651112 (+   502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.656932 (+  5820us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.656985 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.699425 (+ 42440us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.897185 (+197760us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:56.897247 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.897346 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:57.127591 (+230245us) raft_consensus.cc:1685] finished
0709 13:05:57.127616 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.127759 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2831,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":15891,"thread_start_us":152982,"threads_started":1,"wal-append.queue_time_us":321402}
I20260709 13:05:57.054351 18042 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.132975 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.128723 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:57.048946 18314 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:05:57.142872 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.143000 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.156487 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.157078 14020 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:57.157182 13870 raft_consensus.cc:2410] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:05:57.158782 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.159143 13945 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:57.159569 13945 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:05:57.160038 13870 raft_consensus.cc:2393] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:57.160230 13873 raft_consensus.cc:2410] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:05:57.163781 13942 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:57.169147 13869 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:57.169333 13989 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:57.170540 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:57.162313 13947 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:57.153013 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.173834 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2623) took 1404 ms. Trace:
I20260709 13:05:57.152067 13867 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:57.154122 14015 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6.
I20260709 13:05:57.174269 13947 rpcz_store.cc:276] 0709 13:05:55.776177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.065106 (+1288929us) service_pool.cc:224] Handling call
0709 13:05:57.131230 (+ 66124us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.173553 (+ 42323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.173773 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:57.175704 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2097) took 1188 ms. Trace:
I20260709 13:05:57.176214 13942 rpcz_store.cc:276] 0709 13:05:55.987620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.130294 (+1142674us) service_pool.cc:224] Handling call
0709 13:05:57.163599 (+ 33305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.175378 (+ 11779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.175631 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:57.169256 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:57.065933 18226 consensus_queue.cc:1059] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:05:57.171039 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.064541 18231 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:57.154799 18116 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:57.201763 13868 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:57.204592 13942 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:57.185312 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:57.121362 14021 rpcz_store.cc:276] 0709 13:05:55.356758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.257477 (+900719us) service_pool.cc:224] Handling call
0709 13:05:56.270478 (+ 13001us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:56.279067 (+  8589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.279101 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:56.293137 (+ 14036us) log.cc:844] Serialized 52 byte log entry
0709 13:05:56.366581 (+ 73444us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18003bcf20. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:05:56.687332 (+320751us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:56.701614 (+ 14282us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.701933 (+   319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.119317 (+417384us) spinlock_profiling.cc:243] Waited 8.65 ms on lock 0x7b18003bcf20. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419
0709 13:05:57.120668 (+  1351us) spinlock_profiling.cc:243] Waited 8.11 ms on lock 0x7b5400297cd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:57.120716 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:57.120763 (+    47us) raft_consensus.cc:1685] finished
0709 13:05:57.120776 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.120930 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11974,"prepare.queue_time_us":19750,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":3036,"raft.queue_time_us":2574,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":30914,"spinlock_wait_cycles":63865856,"thread_start_us":320460,"threads_started":1,"wal-append.queue_time_us":460544}
I20260709 13:05:57.188970 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:57.189304 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.185635 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.208447 14019 raft_consensus.cc:2393] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:57.208624 13867 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:57.143951 18010 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:57.195566 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.206591 13870 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:57.189621 18135 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:57.196277 13945 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.168569 18112 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.225719 13947 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:05:57.215109 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2098) took 1165 ms. Trace:
I20260709 13:05:57.202229 18220 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:57.165261 18218 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:57.230497 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:57.144022 18161 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.233582 13945 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:57.235427 13867 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: [6.6-9.7]
I20260709 13:05:57.226878 13942 rpcz_store.cc:276] 0709 13:05:56.049779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.177142 (+1127363us) service_pool.cc:224] Handling call
0709 13:05:57.204273 (+ 27131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:57.214570 (+ 10297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.215018 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:57.228008 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.226220 13947 pending_rounds.cc:85] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 6
I20260709 13:05:57.243281 13874 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:57.249989 13874 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:57.256839 13874 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:57.259262 13943 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:57.260074 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2621) took 1529 ms. Trace:
I20260709 13:05:57.260530 13943 rpcz_store.cc:276] 0709 13:05:55.730975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.953872 (+1222897us) service_pool.cc:224] Handling call
0709 13:05:57.258687 (+304815us) spinlock_profiling.cc:243] Waited 9.35 ms on lock 0x7b5800202e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030eeee 0000000000375939 00007fb2c83821e0 00007fb2c844b0c4 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:05:57.258870 (+   183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.259742 (+   872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.259992 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21521152}
W20260709 13:05:57.263013 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.253603 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:57.262228 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2088) took 1629 ms. Trace:
I20260709 13:05:57.239476 13947 pending_rounds.cc:107] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:05:57.275930 13947 raft_consensus.cc:2889] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:57.185849 18278 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:05:57.238509 18220 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:57.335682 13867 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:57.317310 18180 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.259016 13874 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:57.282418 18010 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:57.291193 18357 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:05:57.291448 18041 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:57.294806 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.377900 14019 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:05:57.296959 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1947) took 2140 ms. Trace:
I20260709 13:05:57.379108 14024 rpcz_store.cc:276] 0709 13:05:55.156838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.220536 (+1063698us) service_pool.cc:224] Handling call
0709 13:05:56.265162 (+ 44626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.109640 (+844478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.296455 (+186815us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b54003bb0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:57.296855 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1940288768}
I20260709 13:05:57.300189 18281 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49438 (request call id 16) took 11792 ms. Trace:
W20260709 13:05:57.303638 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.313796 17726 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:57.394172 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.277926 17984 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:57.266067 13944 rpcz_store.cc:276] 0709 13:05:55.638116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.888065 (+1249949us) service_pool.cc:224] Handling call
0709 13:05:56.911819 (+ 23754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.938302 (+ 26483us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:56.938726 (+   424us) log.cc:844] Serialized 14 byte log entry
0709 13:05:57.222565 (+283839us) log.cc:844] Serialized 14 byte log entry
0709 13:05:57.232303 (+  9738us) log.cc:844] Serialized 14 byte log entry
0709 13:05:57.261757 (+ 29454us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:57.261836 (+    79us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:57.261907 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.261997 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.262162 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13728,"prepare.queue_time_us":66635,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":89,"raft.queue_time_us":29211,"thread_start_us":280494,"threads_started":1}
I20260709 13:05:57.275019 18112 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.314729 13875 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:05:57.315146 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2082) took 1757 ms. Trace:
W20260709 13:05:57.315670 18341 log.cc:927] Time spent T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.521s	user 0.001s	sys 0.003s
I20260709 13:05:57.328637 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2099) took 1235 ms. Trace:
I20260709 13:05:57.183347 18042 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.227454 18116 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:05:57.345314 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 112060us
I20260709 13:05:57.359443 13949 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:05:57.361655 13874 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:57.386451 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:57.390117 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.376451 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:57.307605 18161 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:57.393406 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.307870 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:57.376621 18314 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:57.415419 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.415354 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:05:57.416345 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.403128 13946 rpcz_store.cc:276] 0709 13:05:55.557461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.735333 (+1177872us) service_pool.cc:224] Handling call
0709 13:05:56.747441 (+ 12108us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.757202 (+  9761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.757236 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.778930 (+ 21694us) log.cc:844] Serialized 26 byte log entry
0709 13:05:57.060399 (+281469us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:57.060502 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.060592 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:57.314866 (+254274us) raft_consensus.cc:1685] finished
0709 13:05:57.314889 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.315057 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":749,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":3022,"thread_start_us":264040,"threads_started":1,"wal-append.queue_time_us":426170}
I20260709 13:05:57.419991 14017 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:57.393301 13945 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:05:57.360512 18054 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:57.376333 18276 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.023s	sys 0.058s
I20260709 13:05:57.404188 13947 rpcz_store.cc:276] 0709 13:05:56.093477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.200796 (+1107319us) service_pool.cc:224] Handling call
0709 13:05:57.223105 (+ 22309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.328203 (+105098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.328534 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:57.421123 13871 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:05:57.425534 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 367712us
I20260709 13:05:57.423763 14522 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:57.428124 18276 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.024s	sys 0.058s
W20260709 13:05:57.430593 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.430941 13945 pending_rounds.cc:85] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 7
I20260709 13:05:57.431545 13945 pending_rounds.cc:107] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:05:57.432009 13945 raft_consensus.cc:2889] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:57.433652 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:57.418026 18206 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.008s	sys 0.017s
W20260709 13:05:57.435163 18206 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.008s	sys 0.017s
I20260709 13:05:57.433262 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2096) took 1446 ms. Trace:
I20260709 13:05:57.435884 14560 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:05:57.426771 13872 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:57.427532 14522 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:57.436975 13866 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:05:57.437793 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2086) took 1859 ms. Trace:
I20260709 13:05:57.437405 13945 rpcz_store.cc:276] 0709 13:05:55.987266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.088393 (+1101127us) service_pool.cc:224] Handling call
0709 13:05:57.158994 (+ 70601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.432843 (+273849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.433163 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:57.438282 13950 rpcz_store.cc:276] 0709 13:05:55.578496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.750998 (+1172502us) service_pool.cc:224] Handling call
0709 13:05:56.751266 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.754875 (+  3609us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:57.019338 (+264463us) log.cc:844] Serialized 14 byte log entry
0709 13:05:57.436756 (+417418us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:57.436811 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:57.437556 (+   745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.437623 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.437737 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":162351,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"thread_start_us":646625,"threads_started":2}
I20260709 13:05:57.443631 13870 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:57.446555 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:57.447325 13943 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:57.448486 13874 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:05:57.449280 13874 pending_rounds.cc:85] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 8
I20260709 13:05:57.449787 13874 pending_rounds.cc:107] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:05:57.450115 13874 raft_consensus.cc:2889] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:57.476986 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 50483.9us
I20260709 13:05:57.467528 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:57.480787 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:57.465683 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:57.392807 18066 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:05:57.487751 14024 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 13:05:57.423323 18054 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.427435 17726 raft_consensus.cc:3055] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:05:57.490559 14524 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.738s (SENT)
I20260709 13:05:57.504992 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:57.445960 18241 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:57.495159 14024 pending_rounds.cc:85] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 10
W20260709 13:05:57.521785 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.521960 14024 pending_rounds.cc:107] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:05:57.522336 14024 raft_consensus.cc:2889] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:57.447960 18219 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:57.524235 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:57.528288 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 47661.8us
I20260709 13:05:57.419987 18314 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.493134 18206 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:57.502624 18054 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:57.436599 18008 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 13:05:57.564410 13943 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:57.568840 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.519101 18180 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:57.527665 18278 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:57.595983 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.612205 14022 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:57.596396 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.580755 14019 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:05:57.613535 14522 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:57.605749 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:57.521451 18220 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:57.504827 17726 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:57.573441 18042 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:57.557986 18157 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.577937 18008 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:57.558501 18206 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:57.579103 18365 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:05:57.390842 18281 rpcz_store.cc:276] 0709 13:05:45.507524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.507717 (+   193us) service_pool.cc:224] Handling call
0709 13:05:45.547090 (+ 39373us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b580054b0d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c338f0aa 00007fb2c3411447 00007fb2c342e70a 00007fb2c342f76a 00007fb2c342e91b 00007fb2c343a03c 00007fb2c34396b3 00007fb2c34319fa 00007fb2c83f94c9 00007fb2c1975c54
0709 13:05:45.551197 (+  4107us) log.cc:844] Serialized 501 byte log entry
0709 13:05:47.828061 (+2276864us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b1800198160. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c0348a40 00007fb2c03434e4 00007fb2c0343455 00007fb2c338cb0b 00007fb2c3411b63 00007fb2c3439752 00007fb2c34319fa
0709 13:05:57.279273 (+9451212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3187828,"raft.queue_time_us":2782555,"raft.run_cpu_time_us":1801,"raft.run_wall_time_us":78589,"spinlock_wait_cycles":87234048,"thread_start_us":237175,"threads_started":1,"wal-append.queue_time_us":355229,"wal-append.run_cpu_time_us":2399,"wal-append.run_wall_time_us":1075566}
I20260709 13:05:57.579768 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2604) took 2354 ms. Trace:
I20260709 13:05:57.594745 18112 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:57.597796 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.658177 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.642144 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:57.598449 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.660923 14016 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 6.
I20260709 13:05:57.599246 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.600087 18314 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:57.600407 13949 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:05:57.624930 18295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.035s	sys 0.036s
I20260709 13:05:57.603246 13945 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:57.603435 18231 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:57.604717 18043 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:05:57.605003 18159 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.508s	user 0.015s	sys 0.000s
I20260709 13:05:57.558930 18241 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:57.561215 18219 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:57.605705 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 46165.3us
I20260709 13:05:57.645056 13941 rpcz_store.cc:276] 0709 13:05:55.211140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.214265 (+  3125us) service_pool.cc:224] Handling call
0709 13:05:55.214806 (+   541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.215575 (+   769us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:55.215613 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.301435 (+1085822us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b2c000e5d98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:05:56.302121 (+   686us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.707453 (+405332us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:56.707497 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.707580 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:57.273405 (+565825us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:05:57.565092 (+291687us) raft_consensus.cc:1685] finished
0709 13:05:57.565116 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.565246 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":275,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":170,"spinlock_wait_cycles":74327552,"thread_start_us":404820,"threads_started":1,"wal-append.queue_time_us":491892}
W20260709 13:05:57.593447 18329 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.000s
I20260709 13:05:57.606593 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.608217 18226 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:57.615101 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:57.587056 18362 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:05:57.692236 14020 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:57.596783 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.641192 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.641598 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.646639 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.641645 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:57.638507 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 277111us
I20260709 13:05:57.602505 18371 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:57.662511 14018 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:57.593618 18116 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.640856 18206 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:57.646806 18344 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.689875 13943 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:57.689986 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:57.678352 18295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.035s	sys 0.036s
W20260709 13:05:57.624244 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 199625us
I20260709 13:05:57.736792 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:57.622722 18220 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:57.691865 18226 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:57.746270 18372 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:57.746904 14018 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:57.691092 18219 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:57.748485 18370 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:57.699841 14022 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
W20260709 13:05:57.702364 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.707118 14024 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:57.707422 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.750537 18370 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:05:57.722186 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.751303 18370 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:57.751698 18008 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:57.752113 18008 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:57.752525 18370 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.753077 18008 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:57.754522 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.757035 18180 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:57.757102 18054 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.722543 18281 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:05:57.722357 18314 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:57.761569 18314 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.696525 17726 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.763482 17726 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:57.764091 18314 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:57.766489 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:57.767103 14022 raft_consensus.cc:2376] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 9. Re-sending same reply.
I20260709 13:05:57.768136 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:57.724889 14016 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:57.740694 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:57.769738 13866 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:05:57.722465 14021 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:57.751995 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.776595 14019 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:57.775928 18241 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.622315 18157 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:57.778756 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:57.779381 14018 raft_consensus.cc:2376] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 13. Re-sending same reply.
I20260709 13:05:57.779613 18253 raft_consensus.cc:3055] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:57.780179 18253 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:57.784370 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:57.785260 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 143725us
I20260709 13:05:57.749364 18219 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:05:57.788990 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.741837 18243 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:57.791626 18243 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.792642 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.706785 18231 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:57.794580 18112 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:57.794885 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.795722 14016 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:57.795035 18112 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:57.797027 18073 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.002s
I20260709 13:05:57.796828 18112 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.798116 18112 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:57.799693 18244 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.887s	user 0.014s	sys 0.000s
I20260709 13:05:57.743038 18344 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.801349 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.802006 14019 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:05:57.802960 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.803648 13872 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:57.804045 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:57.804812 14021 raft_consensus.cc:2376] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 12. Re-sending same reply.
I20260709 13:05:57.810475 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.812799 13839 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:05:57.810475 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.815443 18114 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.816073 18114 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:57.817293 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.780467 18157 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:57.818450 18157 consensus_queue.cc:1059] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:57.757894 18180 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:57.820935 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.822578 18157 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:57.778585 18220 raft_consensus.cc:697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:57.824865 18220 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:57.827445 18375 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.828662 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.829384 14015 raft_consensus.cc:2410] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:05:57.745589 18226 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.804118 13872 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:57.827260 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.826562 18008 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:57.837679 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.844795 13943 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:05:57.827219 18372 raft_consensus.cc:3055] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:57.856702 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:57.860966 14015 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:05:57.860087 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 194287us
I20260709 13:05:57.861990 13866 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:57.838385 14018 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:57.865054 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:57.862161 13872 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.831967 18314 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.880334 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:05:57.884079 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 325591us
W20260709 13:05:57.885536 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.888513 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.838992 18276 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.828290 18375 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:57.900432 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:57.901996 13949 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:05:57.903206 13872 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:57.797485 18073 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:57.788045 17984 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.133s
I20260709 13:05:57.860571 18372 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:57.919754 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:57.925984 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:57.926810 13941 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:57.927627 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:57.929019 13866 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:05:57.931103 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.931130 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:57.932518 13758 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 02dcf286c6bd48e3a6e548def7d786cc with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 13)
I20260709 13:05:57.935305 18344 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:57.936259 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.937120 13874 raft_consensus.cc:2368] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10.
I20260709 13:05:57.939612 18180 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:57.943034 18243 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:57.946288 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 467948us
I20260709 13:05:57.947086 18226 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:57.947996 18114 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:57.952111 18054 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:57.956017 18206 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:57.975647 13945 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:57.881134 18376 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:57.978026 13949 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:57.983083 14023 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:05:57.985311 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:57.986028 13943 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:05:57.986542 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.987566 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:57.989111 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:57.990141 13947 raft_consensus.cc:2393] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:57.997571 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:58.001679 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:57.981031 18116 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.002485 13870 raft_consensus.cc:2368] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11.
I20260709 13:05:58.002600 18116 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:58.003832 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:58.004130 18135 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.004541 13874 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:05:58.004669 18135 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:57.880900 18219 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:05:58.010432 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:58.012518 14020 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:57.898149 18314 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.020406 13945 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
W20260709 13:05:58.026657 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.040014 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:58.036752 13866 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:58.037003 13941 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:58.042035 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:58.042265 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:58.043044 13945 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:58.044410 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:58.037969 13872 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:05:58.042177 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.045516 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:58.040848 13947 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:58.050318 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:58.055136 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:58.056334 14024 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:58.057485 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.058259 14024 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:58.059269 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.059337 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:58.060220 13947 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
W20260709 13:05:58.061201 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 215367us
I20260709 13:05:57.917649 18276 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:58.062019 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:58.080508 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:58.069077 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.081379 14024 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:58.082309 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.074569 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:58.085131 13943 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:05:58.069108 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:58.074596 13945 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:05:58.010036 18376 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:58.091668 13948 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
W20260709 13:05:58.080260 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 73083.4us
W20260709 13:05:58.093330 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.095300 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.096592 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:58.097394 13943 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:58.098191 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.985394 18114 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:05:58.103942 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.106110 13947 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:57.974748 18377 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.131s
W20260709 13:05:58.080704 18043 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 13:05:58.145295 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 197207us
I20260709 13:05:58.188522 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:58.113492 13946 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:58.192497 14024 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:58.152359 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:58.110675 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.078967 18281 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:05:58.198961 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 108099us
I20260709 13:05:58.196807 14021 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:58.091446 18054 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:58.213923 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 133471us
I20260709 13:05:58.107439 18008 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:58.223655 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:05:58.224090 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.226007 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:05:58.228309 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.226814 14021 raft_consensus.cc:2376] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 9. Re-sending same reply.
I20260709 13:05:58.080818 18157 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.135s
W20260709 13:05:58.236969 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 405816us
I20260709 13:05:58.241562 14024 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
W20260709 13:05:58.242525 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 112371us
I20260709 13:05:58.242776 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.253206 14024 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:58.093204 17726 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.127s
I20260709 13:05:58.198767 18387 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:05:58.267377 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 43847.9us
I20260709 13:05:58.263298 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:58.264147 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:58.266919 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52357us
I20260709 13:05:58.205700 18042 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:58.272750 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:58.272694 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:58.279106 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:58.277933 13943 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:05:58.234843 18008 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:58.288918 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.292302 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:58.294107 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:58.235131 18378 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:58.261987 18380 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.219s	user 0.001s	sys 0.002s
I20260709 13:05:58.297849 14021 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:58.237543 18161 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:58.234656 18281 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:58.303078 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:58.332293 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.334314 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:58.300693 18008 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:58.323944 18373 log.cc:927] Time spent T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:05:58.277422 18377 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:05:58.189179 18372 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:58.334865 18112 raft_consensus.cc:3055] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:58.296274 18042 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:58.333966 18295 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:58.335575 18219 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:58.346266 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 229186us
W20260709 13:05:58.377904 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 43648.5us
W20260709 13:05:58.379428 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:58.383617 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:58.384249 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:58.385040 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 112363us
I20260709 13:05:58.352702 18010 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:58.288966 18387 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:58.380810 18374 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:58.351639 14024 raft_consensus.cc:2376] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 9. Re-sending same reply.
I20260709 13:05:58.345067 17726 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:05:58.394953 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:58.394683 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.359611 18278 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:58.358428 18112 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:58.310297 18378 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:58.362848 18135 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:58.390627 18381 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.342s	user 0.000s	sys 0.001s
I20260709 13:05:58.382248 18054 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:58.422574 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.429255 18042 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:58.434600 18409 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:58.500396 18390 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:05:58.500072 18409 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:05:58.501226 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 121978us
I20260709 13:05:58.434965 18276 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:58.436443 18386 log.cc:927] Time spent T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.095s	user 0.001s	sys 0.001s
W20260709 13:05:58.436935 18385 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
W20260709 13:05:58.437783 18397 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:05:58.504191 18042 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:58.504797 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:58.505824 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:58.506589 14561 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
W20260709 13:05:58.506733 18394 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:05:58.392243 18404 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:58.509346 18382 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.387s	user 0.000s	sys 0.001s
I20260709 13:05:58.484462 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:58.488974 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 94101.8us
I20260709 13:05:58.396389 18377 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.397895 18374 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:58.436446 18388 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:05:58.402134 18243 raft_consensus.cc:2764] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:05:58.417424 18391 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:58.421393 18314 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:58.344735 18376 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.457192 14022 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:05:58.520644 14024 raft_consensus.cc:2376] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 10. Re-sending same reply.
I20260709 13:05:58.508776 18409 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:58.527071 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 275708us
I20260709 13:05:58.542855 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.521608 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:58.401458 18281 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:58.543503 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:58.509171 18278 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.545583 18378 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:58.547117 18387 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:58.553930 14020 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:05:58.554261 13872 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:58.567909 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.571084 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:58.571746 13867 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:58.455993 14016 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:58.483637 18405 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:05:58.564327 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.564944 13839 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:58.565073 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 43614.9us
I20260709 13:05:58.574263 14024 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 12: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:05:58.562881 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 240289us
I20260709 13:05:58.573829 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:58.565718 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 146958us
I20260709 13:05:58.580312 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:58.581058 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:58.569931 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:58.583710 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:58.581950 13874 raft_consensus.cc:2410] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:58.594972 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:58.598038 13915 leader_election.cc:400] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10.
I20260709 13:05:58.598143 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:58.598592 13915 leader_election.cc:403] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:58.591467 13838 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:58.520283 18412 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:05:58.603802 18401 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:58.604362 18401 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.604460 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:58.604745 18401 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:58.606402 18406 consensus_queue.cc:1059] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:58.607515 18406 raft_consensus.cc:3055] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:58.529141 18377 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.566466 18135 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:58.550076 13773 catalog_manager.cc:5697] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:58.565554 18404 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:58.612972 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:05:58.614339 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 102921us
I20260709 13:05:58.615989 13915 leader_election.cc:400] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11.
I20260709 13:05:58.616421 13915 leader_election.cc:403] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:58.616829 14020 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:58.619124 13872 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:58.620036 14022 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:58.626722 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:58.627512 13950 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
W20260709 13:05:58.607535 18402 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:05:58.629732 13915 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.797s (SENT) [suppressed 40 similar messages]
I20260709 13:05:58.631055 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:58.632608 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:58.645113 13870 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:05:58.633423 13947 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:58.636770 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.609104 18243 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:58.540279 18112 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.562245 18383 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:05:58.657371 18278 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:58.660804 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:58.601550 18410 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:58.627816 18314 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:05:58.607910 18406 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:05:58.669682 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 81846.8us
I20260709 13:05:58.670791 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:58.652248 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:58.617727 18404 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.674196 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:58.675765 14016 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:05:58.560005 18391 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.670847 18243 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:58.678580 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 56979.9us
I20260709 13:05:58.679145 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:58.695286 13871 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:05:58.680374 13867 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:05:58.668651 14021 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:58.680964 14020 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
W20260709 13:05:58.676898 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47187.9us
I20260709 13:05:58.699301 14020 pending_rounds.cc:85] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 6
I20260709 13:05:58.699659 14020 pending_rounds.cc:107] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:05:58.700037 14020 raft_consensus.cc:2889] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:58.687522 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:58.614832 18135 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:58.700973 13872 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:58.705667 18276 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:58.706292 18415 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:58.706979 18415 raft_consensus.cc:697] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:58.707979 18415 consensus_queue.cc:237] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.709517 18112 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:58.710412 18419 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:58.711038 18419 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.683403 18410 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:05:58.754390 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 49576us
I20260709 13:05:58.713830 18419 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:58.755757 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:58.689432 18220 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:05:58.704197 18243 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:58.566174 18374 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:58.760583 13945 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 12: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:58.688258 18404 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:58.761487 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.704490 18135 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:58.763991 14018 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:05:58.771384 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:58.764578 18407 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11.
I20260709 13:05:58.757066 18410 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:58.778132 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 93766.2us
I20260709 13:05:58.779708 13872 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:05:58.768177 18405 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.794509 18414 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.795359 18377 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:58.795907 18421 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:05:58.796725 18408 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:05:58.797636 18406 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.798617 18389 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.765486 18419 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:58.807312 18391 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:58.846163 18399 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:58.847817 18399 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:58.814468 18405 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:58.852634 18405 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:58.853062 18389 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.842909 18419 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.873850 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:05:58.875626 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 107068us
W20260709 13:05:58.883955 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.893070 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:58.879513 18405 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:58.897635 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:58.820712 18414 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.946673 13868 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:58.936417 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:05:58.936929 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 103559us
I20260709 13:05:58.938426 13873 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:58.958513 13870 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:05:58.938477 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:58.941231 18409 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:58.941375 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:05:58.960242 13874 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:58.930486 18157 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:58.895511 18406 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:58.944718 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:58.965667 13941 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:05:58.933449 18405 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.946578 18415 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.950224 18112 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.859103 18253 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:58.857187 18421 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:58.966361 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 68824us
I20260709 13:05:58.978322 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:58.967284 13838 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:58.969485 13872 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:58.965152 18008 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.003s
I20260709 13:05:58.979606 18008 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:58.981084 14558 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:05:58.985757 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:05:58.973174 18226 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.026s	user 0.015s	sys 0.000s
I20260709 13:05:58.970083 18405 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:58.965070 18409 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:58.994372 18066 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:58.995685 13987 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:58.977344 18112 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:58.999919 18421 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:05:59.001302 18219 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.636s	user 0.009s	sys 0.003s
I20260709 13:05:59.002398 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:59.003291 18278 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:59.003679 18401 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:59.006315 18157 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:59.008285 18008 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:59.014715 18419 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:59.012055 18135 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:59.019868 18384 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.795s	user 0.003s	sys 0.000s
I20260709 13:05:59.028319 13867 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:58.933933 18218 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
W20260709 13:05:59.030262 13840 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.513s (SENT)
W20260709 13:05:59.040336 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44785.1us
I20260709 13:05:59.038313 18066 raft_consensus.cc:2764] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:05:59.033090 18409 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:59.052506 18054 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.005s
I20260709 13:05:59.063561 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:59.057242 13987 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:59.057066 18389 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:59.057839 18419 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:59.060070 18368 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:59.062119 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:59.065557 13945 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:05:59.066012 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:58.975260 18415 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:59.053153 18157 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.051695 18219 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:59.072926 18157 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:59.071365 18419 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:05:59.074553 18419 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:59.071931 18066 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:59.082958 18314 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10.
I20260709 13:05:59.083318 18116 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:05:59.087422 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.071570 18368 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:59.090342 18368 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:59.090915 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2636) took 1126 ms. Trace:
I20260709 13:05:59.091363 13948 rpcz_store.cc:276] 0709 13:05:57.964350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.964519 (+   169us) service_pool.cc:224] Handling call
0709 13:05:59.090690 (+1126171us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400053a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:59.090845 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237770624}
I20260709 13:05:59.091704 18368 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:59.095649 13950 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:59.095834 18415 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:59.100867 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:59.104818 14553 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:59.111497 18135 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:59.102396 13838 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:59.115057 13867 pending_rounds.cc:85] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 7
I20260709 13:05:59.115406 13867 pending_rounds.cc:107] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:05:59.115440 18226 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:59.115741 13867 raft_consensus.cc:2889] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:59.116029 18226 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.116475 18226 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:59.118904 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:59.122344 13941 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:05:59.124814 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:59.072212 18054 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:59.127647 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:59.130913 13869 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:59.096609 18112 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:59.093962 18157 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:59.103909 18421 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:59.135454 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:05:59.142725 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:05:59.121594 18135 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:05:59.134840 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 104619us
W20260709 13:05:59.153590 13840 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.595s (SENT)
I20260709 13:05:59.133500 18428 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:59.166010 13840 leader_election.cc:336] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.632s (SENT)
I20260709 13:05:59.148049 18376 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:59.133622 18244 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:05:59.173624 18439 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:59.174028 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:59.174808 14019 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:05:59.175272 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1995) took 1045 ms. Trace:
I20260709 13:05:59.175550 14019 rpcz_store.cc:276] 0709 13:05:58.129438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.173679 (+1044241us) service_pool.cc:224] Handling call
0709 13:05:59.175213 (+  1534us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:59.178573 18414 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:59.179768 18401 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:59.168366 18428 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.180665 18428 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:59.181663 14524 leader_election.cc:336] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.876s (SENT)
I20260709 13:05:59.174095 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2765) took 1243 ms. Trace:
I20260709 13:05:59.182845 13866 rpcz_store.cc:276] 0709 13:05:57.930519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.930804 (+   285us) service_pool.cc:224] Handling call
0709 13:05:59.173887 (+1243083us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54002d5354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:59.174004 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240779008}
I20260709 13:05:59.191223 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:59.192138 13950 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:59.194069 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58689.4us
W20260709 13:05:59.203670 13840 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.840s (SENT)
I20260709 13:05:59.205785 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:59.208743 14017 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:59.210140 18241 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:59.138084 18157 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:59.211090 13987 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:59.213018 13941 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:59.217406 18206 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:59.152581 18409 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:59.219710 18409 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:59.186148 18243 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:59.225174 18243 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:59.181468 18404 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:59.226297 18401 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:59.227098 13945 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:59.227885 18404 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:59.226014 13943 raft_consensus.cc:2393] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:59.205169 18376 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.238265 18376 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:59.240758 18376 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:59.205368 18439 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:05:59.244334 13942 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:05:59.246481 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 121774us
W20260709 13:05:59.247006 13840 leader_election.cc:336] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.882s (SENT)
I20260709 13:05:59.248123 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:59.185942 18410 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:59.248796 18439 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:59.249224 18439 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.249580 18439 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:59.250778 18410 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:59.251194 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:59.251986 14019 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:05:59.253289 18073 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:59.253732 14560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:05:59.254638 14560 raft_consensus.cc:606] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
W20260709 13:05:59.249507 14524 leader_election.cc:336] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.943s (SENT)
I20260709 13:05:59.256222 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:59.258713 18219 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:59.211755 18370 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.012s	sys 0.018s
W20260709 13:05:59.261273 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 258985us
I20260709 13:05:59.264199 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:59.266938 14020 raft_consensus.cc:2376] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 6. Re-sending same reply.
I20260709 13:05:59.264732 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:59.267964 14016 raft_consensus.cc:2410] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:59.267824 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:59.264919 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:59.269315 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:59.269353 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:59.270116 13941 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:59.271606 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:59.272814 13950 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:05:59.274694 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:59.275321 14021 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:59.278455 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:59.283039 13946 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:05:59.286290 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:59.288334 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:59.288645 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:05:59.288993 13950 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:59.289248 14018 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:59.290836 14522 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:59.294624 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:05:59.295030 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:59.266667 18370 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.012s	sys 0.021s
I20260709 13:05:59.299134 14019 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:59.301250 13950 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:05:59.302505 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:59.303083 13950 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:05:59.303874 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:59.303944 14021 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
W20260709 13:05:59.307601 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 60634.8us
W20260709 13:05:59.308843 13840 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.986s (SENT)
I20260709 13:05:59.300088 13838 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:59.311009 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:59.319798 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:59.324280 18041 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:59.324448 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:05:59.325536 18434 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:59.325814 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:59.326691 18444 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:59.327312 14023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 493) took 2269 ms (client timeout 2144 ms). Trace:
I20260709 13:05:59.327234 18444 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:59.329440 13950 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:05:59.329547 14022 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:59.330137 18409 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:59.330549 18409 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:59.330910 18409 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:59.331156 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 1977) took 1262 ms. Trace:
I20260709 13:05:59.331307 13945 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:59.331549 14022 rpcz_store.cc:276] 0709 13:05:58.068874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.621250 (+552376us) service_pool.cc:224] Handling call
0709 13:05:59.329346 (+708096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:59.330081 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.330948 (+   867us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b5400412650. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:59.331098 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1535361408}
I20260709 13:05:59.334779 13950 pending_rounds.cc:85] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 5
I20260709 13:05:59.335105 13950 pending_rounds.cc:107] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:05:59.335336 13950 raft_consensus.cc:2889] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:59.342228 18432 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:05:59.268085 18219 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:05:59.287714 18407 raft_consensus.cc:2764] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:05:59.290314 18404 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:59.351065 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 43993.5us
W20260709 13:05:59.327888 14023 rpcz_store.cc:269] 0709 13:05:57.058136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.058316 (+   180us) service_pool.cc:224] Handling call
0709 13:05:59.327053 (+2268737us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b5400412654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:05:59.327206 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2069020544}
I20260709 13:05:59.352931 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:05:59.349375 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.321887 13838 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:59.299047 18281 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:59.306284 18008 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:59.355535 13945 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:05:59.356137 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47323.4us
I20260709 13:05:59.310572 18378 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:59.329823 18444 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:59.366410 14558 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:59.370859 13840 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.223s (SENT)
I20260709 13:05:59.371968 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:59.372655 14556 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:59.294409 18243 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:05:59.325016 18041 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:05:59.381479 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.285588 18428 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:59.376648 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:59.390192 18378 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:59.382627 18405 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:59.375136 18387 consensus_queue.cc:1059] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:05:59.415475 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:59.403931 14559 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:05:59.418277 13988 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:59.420071 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2608) took 1927 ms. Trace:
I20260709 13:05:59.420495 14017 rpcz_store.cc:276] 0709 13:05:57.492369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.492553 (+   184us) service_pool.cc:224] Handling call
0709 13:05:57.498038 (+  5485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:59.209321 (+1711283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.419809 (+210488us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b540021cfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:05:59.420004 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3933586176}
I20260709 13:05:59.416230 14558 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:05:59.421972 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:59.400205 18226 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:59.362090 18401 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:59.468642 18401 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:59.470012 18401 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:59.486159 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:05:59.487387 13946 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:59.489724 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.492493 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:59.493192 14558 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:59.407466 14018 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
W20260709 13:05:59.409930 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.414682 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62815.5us
I20260709 13:05:59.426898 13872 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:05:59.383605 18008 raft_consensus.cc:697] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:59.430652 13869 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 13: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:05:59.400231 18041 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:59.440088 18226 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:05:59.451761 14556 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
W20260709 13:05:59.461123 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 44189.1us
I20260709 13:05:59.498342 14559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:05:59.499744 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:59.501221 13941 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:05:59.613350 13947 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:05:59.618391 14551 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:59.618276 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 123623us
I20260709 13:05:59.428944 18370 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:59.610720 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:59.612876 14559 raft_consensus.cc:606] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:05:59.560204 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:05:59.640672 13947 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:59.619951 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:59.711822 14559 raft_consensus.cc:614] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:05:59.712898 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 93016.1us
W20260709 13:05:59.655706 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 156113us
I20260709 13:05:59.715337 18432 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:05:59.670547 18161 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:59.704242 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.716051 18432 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:59.717573 18368 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:59.718699 18368 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:59.719038 18368 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:59.719913 18368 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:59.720925 18368 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:59.720829 18243 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:59.722886 14559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:05:59.723526 14559 raft_consensus.cc:606] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:59.723888 14559 raft_consensus.cc:614] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:05:59.641635 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 101689us
W20260709 13:05:59.623850 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 122861us
I20260709 13:05:59.642210 14553 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:05:59.647496 13947 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:59.643538 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:05:59.620376 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:05:59.727077 13840 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:05:59.731384 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 87956.8us
W20260709 13:05:59.648766 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 145880us
W20260709 13:05:59.731528 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 116493us
I20260709 13:05:59.739214 18066 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:59.739910 18066 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:05:59.716140 18400 raft_consensus.cc:2764] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:05:59.740936 18206 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:59.741025 18066 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:59.741659 18206 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:05:59.742727 18206 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:59.716162 18161 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:59.706662 18445 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
I20260709 13:05:59.752430 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:05:59.781016 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 54013.3us
I20260709 13:05:59.785722 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:59.666679 18377 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:59.750286 18419 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.675s	user 0.023s	sys 0.005s
I20260709 13:05:59.793418 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:05:59.642405 18370 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.616364 18448 consensus_queue.cc:1059] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.238s
I20260709 13:05:59.641156 18314 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.161s
I20260709 13:05:59.797739 18226 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:59.797996 18387 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.804562 18405 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:59.753860 18219 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:05:59.804128 18243 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:59.830031 18401 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:59.858460 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.725292 18008 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:59.872251 14553 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:05:59.873108 14553 raft_consensus.cc:606] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:59.873648 14553 raft_consensus.cc:614] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:59.831207 18370 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:59.875308 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 81972.2us
I20260709 13:05:59.877128 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:05:59.844847 18278 raft_consensus.cc:2764] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:05:59.838641 18314 consensus_queue.cc:1243] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:59.820796 18419 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:59.848392 18377 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.892889 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:59.899537 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:59.901146 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:05:59.856109 18387 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:05:59.894305 18453 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:05:59.672842 18378 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:05:59.855082 18226 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:05:59.903256 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:05:59.903086 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 251101us
I20260709 13:05:59.928736 13839 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:59.929493 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:59.891296 18400 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:59.906550 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 120823us
I20260709 13:05:59.945699 14558 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:05:59.924464 18243 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:59.911273 18389 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:59.961362 13839 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:05:59.961299 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:05:59.924873 18444 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:59.966653 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:59.969295 14558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:05:59.970221 14558 raft_consensus.cc:606] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:05:59.970618 14558 raft_consensus.cc:614] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:59.943545 18161 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:05:59.941937 18407 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:59.943059 18400 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:05:59.926043 18377 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:05:59.992247 18428 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:05:59.996675 18226 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:00.000964 18416 raft_consensus.cc:3055] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:00.001484 18416 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:00.007088 18375 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:00.007972 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:00.013370 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:00.015221 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:00.016227 13950 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:00.019716 18399 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:00.022639 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2110) took 1392 ms. Trace:
I20260709 13:06:00.023047 13943 rpcz_store.cc:276] 0709 13:05:58.630645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.630791 (+   146us) service_pool.cc:224] Handling call
0709 13:06:00.022384 (+1391593us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b5400229054. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:00.022572 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1336158464}
I20260709 13:06:00.025280 13944 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:00.036893 18372 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:00.042492 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:00.045282 13773 catalog_manager.cc:5697] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 6 to 7, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 7 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:00.052153 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 58371.7us
W20260709 13:06:00.082394 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 115842us
I20260709 13:05:59.972220 18468 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:59.962029 18371 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:05:59.983575 18448 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:06:00.102512 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 60142.8us
I20260709 13:06:00.090922 13989 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:00.105126 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:05:59.985759 18400 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:59.973618 18403 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.113s
W20260709 13:06:00.044242 18439 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.794s	user 0.014s	sys 0.000s
W20260709 13:06:00.113668 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:00.114398 13946 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.626s	user 0.009s	sys 0.004s
W20260709 13:06:00.118911 18457 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.430s	user 0.004s	sys 0.000s
W20260709 13:06:00.175879 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:00.120795 13942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2111) took 1489 ms (client timeout 1655 ms). Trace:
W20260709 13:06:00.184924 13942 rpcz_store.cc:269] 0709 13:05:58.631884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.632746 (+   862us) service_pool.cc:224] Handling call
0709 13:06:00.120510 (+1487764us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b54001c6954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:00.120731 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1259625344}
I20260709 13:06:00.120995 13949 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:06:00.124888 13773 catalog_manager.cc:5697] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 5 to 7, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 7 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:00.129046 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:06:00.145030 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 54202.9us
I20260709 13:05:59.978184 18407 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.158412 13946 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:00.079819 18281 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:00.108438 18468 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.107841 18469 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.107173 18428 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:00.161422 18220 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.177912 18112 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:00.210856 18407 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:00.107394 18276 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 13:06:00.209226 18439 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:00.250942 18376 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:00.251466 18376 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.251819 18376 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:00.230443 18219 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 13:06:00.253750 18409 consensus_meta.cc:287] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.922s	user 0.014s	sys 0.000s
I20260709 13:06:00.254251 18409 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:00.256078 18428 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:00.256930 18409 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:00.233371 18469 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:00.271281 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 187748us
I20260709 13:06:00.272593 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:00.226648 18468 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:00.254154 18157 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.224078 18281 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:00.285771 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:00.265035 18220 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:00.298513 18377 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:00.299090 18220 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:00.299309 18377 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:00.289889 18157 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:00.277798 18428 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:00.303743 18329 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:06:00.173848 18416 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:00.332253 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:00.333117 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.334368 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:00.307924 18439 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:00.340027 18377 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:00.342653 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2658) took 1032 ms. Trace:
W20260709 13:06:00.343276 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 57638.5us
I20260709 13:06:00.344754 13840 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:00.343163 13941 rpcz_store.cc:276] 0709 13:05:59.309806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.620449 (+310643us) service_pool.cc:224] Handling call
0709 13:05:59.620764 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:59.621131 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:59.621160 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:59.644715 (+ 23555us) log.cc:844] Serialized 26 byte log entry
0709 13:05:59.880125 (+235410us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:59.880190 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.880300 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:00.342371 (+462071us) raft_consensus.cc:1685] finished
0709 13:06:00.342397 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:00.342574 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20341,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":13760,"thread_start_us":196627,"threads_started":1,"wal-append.queue_time_us":664455}
I20260709 13:06:00.306341 18419 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:00.332038 18387 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:00.378018 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 474784us
I20260709 13:06:00.364948 18377 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:00.366492 18416 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:00.379863 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:00.365758 18428 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:06:00.421447 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:00.425105 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:00.434435 13870 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:00.433149 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:00.436975 13875 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:00.394001 18439 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.440730 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:00.441828 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:00.445525 13868 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:00.448017 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:00.445835 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:00.449260 13867 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:00.446579 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:00.395583 18419 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:00.452965 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:00.453080 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:00.456279 13869 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:00.377883 18470 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:06:00.458480 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:00.463160 13874 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:00.458701 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:00.464180 13868 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:00.459565 13871 raft_consensus.cc:2410] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:06:00.391248 18387 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:06:00.471598 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:00.474668 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:00.476837 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:00.462255 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:00.481616 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:00.437050 18226 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:00.475571 13867 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
W20260709 13:06:00.486480 18475 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:06:00.487246 18253 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:00.493453 18375 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:00.494104 18375 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.493780 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:00.494583 18375 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:00.504333 13868 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:00.506657 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:00.457218 18439 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:00.519376 18206 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:00.513358 18472 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.171s	user 0.002s	sys 0.000s
I20260709 13:06:00.535957 18276 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.536638 18276 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:00.521448 18370 consensus_meta.cc:287] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.631s	user 0.009s	sys 0.005s
I20260709 13:06:00.512590 18409 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:00.519444 18452 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:00.550916 18448 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:00.526136 18428 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:00.545439 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:00.545063 18008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.018s	sys 0.013s
W20260709 13:06:00.562870 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:00.566793 13947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.008s	sys 0.024s
W20260709 13:06:00.568362 13947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.008s	sys 0.025s
I20260709 13:06:00.548019 18477 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.548650 18409 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:00.552160 18281 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:00.565690 18008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.018s	sys 0.013s
I20260709 13:06:00.519034 18226 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:00.570189 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 429114us
I20260709 13:06:00.693982 14556 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:00.552516 18370 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:00.600373 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 395804us
I20260709 13:06:00.576789 18409 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:00.573170 18452 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:00.587000 18374 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:00.565752 18428 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:00.547508 18206 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:00.585350 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 447223us
I20260709 13:06:00.680570 18281 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:00.727543 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 49317.8us
I20260709 13:06:00.739833 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 742) took 1382 ms. Trace:
I20260709 13:06:00.746851 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:00.740332 13945 rpcz_store.cc:276] 0709 13:05:59.357622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.729502 (+371880us) service_pool.cc:224] Handling call
0709 13:05:59.729834 (+   332us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:59.753548 (+ 23714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:59.753593 (+    45us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:59.802210 (+ 48617us) log.cc:844] Serialized 78 byte log entry
0709 13:06:00.077889 (+275679us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:00.077942 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.078029 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:00.739534 (+661505us) raft_consensus.cc:1685] finished
0709 13:06:00.739560 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:00.739760 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44194,"raft.queue_time_us":24544,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":17259,"thread_start_us":227080,"threads_started":1,"wal-append.queue_time_us":507018}
I20260709 13:06:00.586345 18477 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:00.726956 18276 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:00.767916 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:00.771085 13866 raft_consensus.cc:2410] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:06:00.730772 18470 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:00.775625 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:00.771651 18439 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:00.783479 18439 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:00.783514 13866 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:00.784174 18439 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:00.785861 18008 consensus_queue.cc:1059] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:00.721494 18281 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:00.793494 18439 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:00.794790 18439 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:00.795711 18452 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:00.803192 18276 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:00.803676 18281 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:00.803825 18276 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:00.790942 18220 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:00.836086 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:00.810374 18490 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:00.810235 18455 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.951s	user 0.008s	sys 0.007s
I20260709 13:06:00.722548 18448 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:00.833199 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:00.849700 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:00.851418 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:00.852111 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 146225us
I20260709 13:06:00.841464 14018 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:00.812553 18276 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:00.837276 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:00.825497 13943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.010s	sys 0.034s
W20260709 13:06:00.853968 13943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.010s	sys 0.034s
W20260709 13:06:00.825384 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 258044us
W20260709 13:06:00.813793 18479 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:06:00.834462 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:00.849663 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:00.856483 14017 raft_consensus.cc:2393] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:00.810432 18495 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:00.825433 18378 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.022s	sys 0.019s
I20260709 13:06:00.866755 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:00.875874 14023 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:00.866966 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:00.790228 18470 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:00.869661 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:00.821399 18483 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.877970 14016 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:00.878247 18483 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:00.879340 18470 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:00.880723 18483 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:00.881208 18470 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:00.882218 18483 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:00.882882 18483 consensus_queue.cc:1243] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:00.871435 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:00.885116 14021 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:00.873795 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.840768 18220 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:00.891039 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:00.901822 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.927214 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:00.931463 18206 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:00.867619 18344 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.932327 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:00.936677 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 743) took 1578 ms. Trace:
I20260709 13:06:01.028191 13944 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:01.028059 13947 rpcz_store.cc:276] 0709 13:05:59.357886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.729614 (+371728us) service_pool.cc:224] Handling call
0709 13:05:59.729922 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:59.792343 (+ 62421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:59.792387 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:59.793112 (+   725us) log.cc:844] Serialized 26 byte log entry
0709 13:06:00.571943 (+778831us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:00.572008 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.575235 (+  3227us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:00.936397 (+361162us) raft_consensus.cc:1685] finished
0709 13:06:00.936423 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:00.936605 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32750,"raft.queue_time_us":68245,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":2935,"thread_start_us":729637,"threads_started":1,"wal-append.queue_time_us":1019850}
I20260709 13:06:00.937330 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2135) took 1203 ms. Trace:
I20260709 13:06:01.029953 13943 rpcz_store.cc:276] 0709 13:05:59.733811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.023873 (+290062us) service_pool.cc:224] Handling call
0709 13:06:00.024226 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.024606 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:00.024633 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:00.039948 (+ 15315us) log.cc:844] Serialized 26 byte log entry
0709 13:06:00.854422 (+814474us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:00.854472 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.854544 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:00.937097 (+ 82553us) raft_consensus.cc:1685] finished
0709 13:06:00.937122 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:00.937269 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10186,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":814005,"threads_started":1,"wal-append.queue_time_us":892352}
W20260709 13:06:00.941743 18486 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:06:00.943572 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2655) took 1648 ms. Trace:
I20260709 13:06:00.945114 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:01.038307 18477 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:01.039752 13946 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:01.041143 13946 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:06:00.945631 18376 consensus_meta.cc:287] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.693s	user 0.009s	sys 0.002s
I20260709 13:06:01.042366 18376 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:01.043244 13946 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:01.038347 13949 rpcz_store.cc:276] 0709 13:05:59.294935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.461743 (+166808us) service_pool.cc:224] Handling call
0709 13:06:00.120710 (+658967us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.186584 (+ 65874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.943341 (+756757us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b54001c6950. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:00.943504 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1514485632}
I20260709 13:06:00.945925 18375 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:00.948129 18469 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:00.950297 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:00.951800 14024 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:06:00.952126 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:00.953207 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:00.954521 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:00.953955 13773 catalog_manager.cc:5697] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 1 to 6, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 6 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:00.956465 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:00.958564 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 67501.1us
I20260709 13:06:00.962344 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:00.875097 18378 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.023s	sys 0.020s
W20260709 13:06:00.990880 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.892581 13950 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:01.011979 13942 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:06:00.931720 18501 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:00.887744 18487 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:00.881799 18470 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:00.889303 18495 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.037149 18226 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:00.963725 18400 consensus_meta.cc:287] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.756s	user 0.012s	sys 0.002s
I20260709 13:06:01.050798 14022 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:01.051380 14023 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:01.052224 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:01.051038 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:00.853121 18490 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:01.100558 18490 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:01.103365 18470 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:01.049041 18276 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.104756 18276 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:01.106439 18220 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:01.067358 18428 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:01.047206 18499 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:06:01.060142 18487 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:06:01.026535 18368 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.305s	user 0.019s	sys 0.000s
W20260709 13:06:01.045918 18480 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
W20260709 13:06:01.109759 18377 consensus_meta.cc:287] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.716s	user 0.010s	sys 0.004s
I20260709 13:06:01.070122 18370 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:01.015583 18403 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.062280 18406 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.079705 18392 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:01.080303 18412 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.166858 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.079836 18400 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.047430 18375 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:01.048502 18376 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:01.078344 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:01.150272 18377 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:01.063964 18501 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:01.048668 18407 consensus_meta.cc:287] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.774s	user 0.010s	sys 0.004s
I20260709 13:06:01.064270 18401 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.051968 18344 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:01.024302 18314 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.108232 18220 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:01.161547 18519 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.161228 18409 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.166858 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:01.164387 18505 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:06:01.167048 18416 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.002s
W20260709 13:06:01.171625 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 93379.5us
W20260709 13:06:01.113497 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.173712 14021 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:06:01.174136 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 103153us
I20260709 13:06:01.174850 14020 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:06:01.213331 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:01.214234 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.214316 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:01.175235 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:01.213358 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:01.216002 14017 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 6: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:01.216352 14021 raft_consensus.cc:2376] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 10. Re-sending same reply.
I20260709 13:06:01.216884 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.217119 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:01.217201 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.217430 13867 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 12 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:01.217947 13874 raft_consensus.cc:2410] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:06:01.218044 13866 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:01.218250 14020 raft_consensus.cc:2376] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 10. Re-sending same reply.
I20260709 13:06:01.218653 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:01.219180 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:01.176767 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:01.219851 14020 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:01.219986 13949 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:01.220350 13868 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:01.175959 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:01.177500 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:01.176469 13838 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:01.221220 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:01.221992 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 45621.2us
I20260709 13:06:01.222733 13838 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:01.223040 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:01.223585 13840 leader_election.cc:336] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.663s (SENT)
I20260709 13:06:01.120055 18487 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.224131 18112 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:06:01.224434 13840 leader_election.cc:336] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.690s (SENT)
I20260709 13:06:01.224864 18487 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:01.225028 18226 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.225572 18226 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:01.226223 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:01.226856 13868 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:01.224228 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:01.228773 18490 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:01.229398 18492 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:01.224892 18112 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:01.180598 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:01.184696 13942 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:06:01.183485 13947 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:01.230795 18112 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.167222 18392 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:01.232518 18392 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:01.184417 14015 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:01.183544 14023 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:01.180279 18499 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.206452 14018 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:01.223757 14017 raft_consensus.cc:2368] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13.
I20260709 13:06:01.217978 14021 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:01.229842 18492 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.236763 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:01.237390 13868 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:01.230316 18490 consensus_queue.cc:1059] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:01.230434 13944 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:01.168699 18406 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.240962 13873 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:01.246611 14020 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:01.267724 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.219246 13866 raft_consensus.cc:3055] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:01.269428 13866 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:01.184855 18375 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:01.276309 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:01.166930 18403 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:01.235435 18468 consensus_meta.cc:287] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.945s	user 0.009s	sys 0.005s
W20260709 13:06:01.264983 13840 leader_election.cc:336] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.141s (SENT)
I20260709 13:06:01.285744 13840 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:01.288393 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.290294 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:01.291141 14020 raft_consensus.cc:2376] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 6. Re-sending same reply.
I20260709 13:06:01.207363 18505 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:01.256484 18492 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:01.170769 18412 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:01.255869 18226 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:01.233989 18161 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.301327 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:01.299525 13838 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:01.200151 18401 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:01.303222 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 88038.2us
I20260709 13:06:01.224289 18511 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:01.278477 18469 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.205828 18314 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.280503 18377 raft_consensus.cc:3055] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:01.278424 18375 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:01.310530 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:01.310640 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:01.200690 18407 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.327622 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.308470 13868 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:01.275997 18499 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.309283 13866 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:01.335219 13866 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:01.332831 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 445335us
I20260709 13:06:01.312146 18511 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.335533 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:01.347369 13868 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:06:01.348287 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:01.349102 13868 raft_consensus.cc:2410] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:06:01.341254 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:01.350507 13948 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:01.351486 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:01.352730 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:01.353690 18226 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:01.354092 18226 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:01.354700 18226 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:01.362843 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.307687 18416 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:06:01.323915 18375 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:01.211179 18519 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:01.324370 18508 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:01.349843 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:01.341846 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.320741 18377 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:01.398797 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.346228 13941 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:01.295138 18468 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:01.411568 14023 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:01.412803 13840 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.889s (SENT)
I20260709 13:06:01.423534 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:01.409556 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 247921us
I20260709 13:06:01.337989 18490 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:01.311936 18161 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.337018 18495 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:01.365674 18507 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:01.367225 18511 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:01.436057 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.405491 18370 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.342805 18520 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:01.513375 13868 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:01.326889 18469 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.417181 18508 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:01.331419 18525 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:01.471155 13866 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:01.502745 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:01.504927 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.415782 18515 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:01.440270 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.434476 18448 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:01.710408 13874 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:01.561002 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.712375 14021 raft_consensus.cc:2376] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 16. Re-sending same reply.
W20260709 13:06:01.591080 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.722186 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.719297 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 290063us
I20260709 13:06:01.712715 14023 raft_consensus.cc:2368] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13.
I20260709 13:06:01.447165 18391 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.448109 18488 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.245s
I20260709 13:06:01.498217 18403 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:01.496595 18504 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:01.680038 18419 consensus_meta.cc:287] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.222s	user 0.015s	sys 0.000s
I20260709 13:06:01.781795 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.761319 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.512881 18495 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.819823 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:01.500980 18507 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.551422 18400 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:01.713573 18525 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.553543 18508 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:01.867470 18378 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:01.764008 18344 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:01.806285 18008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.019s	sys 0.026s
I20260709 13:06:01.428375 18505 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.546489 18370 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.810216 18409 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:01.825119 18419 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:01.884502 14524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.012s	sys 0.010s
W20260709 13:06:01.900588 14524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.012s	sys 0.010s
I20260709 13:06:01.825392 18507 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:01.871685 18512 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.526s	user 0.001s	sys 0.000s
W20260709 13:06:01.912163 18529 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:06:01.913916 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:01.914937 14553 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:01.866751 18525 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:01.917268 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:01.764340 18391 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.917526 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:01.676694 18377 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.920866 14555 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:06:01.880940 18008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.021s	sys 0.032s
W20260709 13:06:01.895175 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 169764us
I20260709 13:06:01.827811 18495 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.075426 18407 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:01.819173 18504 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.670228 18522 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.468s
I20260709 13:06:01.806632 18515 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:01.938589 14558 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:06:02.027113 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.938040 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:02.121783 14560 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:02.022706 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:01.929095 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.996838 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.029165 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:01.956745 13943 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:01.950286 14023 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:02.125769 14023 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:02.076185 18499 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:02.089198 13875 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.851s	user 0.009s	sys 0.014s
I20260709 13:06:02.139701 13875 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:02.123167 14552 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:01.980290 18401 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:02.056815 18514 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.622s	user 0.000s	sys 0.001s
I20260709 13:06:01.877415 18378 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.028185 14559 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:02.107651 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.022791 18419 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:02.111030 13838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.014s	sys 0.021s
I20260709 13:06:02.120075 14554 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:02.148599 13838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.014s	sys 0.021s
I20260709 13:06:02.083630 18488 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:02.022019 18226 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.608s	user 0.014s	sys 0.000s
I20260709 13:06:02.133909 14553 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:02.122043 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:02.112092 18517 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.684s	user 0.002s	sys 0.000s
I20260709 13:06:01.955633 18537 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:02.146164 14017 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:02.126848 14023 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.116294 18314 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.154376 14021 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:02.152148 18542 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:02.191246 18537 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.191733 18542 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.193797 18537 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.195791 18378 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.196165 18448 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.196717 18448 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.198225 18112 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.198763 18391 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.198729 18112 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:02.200218 18492 consensus_meta.cc:287] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.888s	user 0.008s	sys 0.002s
I20260709 13:06:02.200657 18492 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.200965 18112 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.201547 18112 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.127187 18008 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:02.186414 14023 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:02.184131 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:02.225082 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:02.243729 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 134312us
I20260709 13:06:02.153174 18469 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:02.214233 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.155588 18468 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.145649 18407 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:02.244426 14524 connection.cc:256] RPC call timeout handler was delayed by -1.0817785069999672s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
W20260709 13:06:02.238830 18487 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.013s	user 0.009s	sys 0.004s
I20260709 13:06:02.187567 18403 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.254812 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:02.256712 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.257488 13944 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
W20260709 13:06:02.205535 18521 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.657s	user 0.001s	sys 0.001s
I20260709 13:06:02.175143 18412 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:02.247903 18514 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:06:02.209110 18477 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.274338 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:02.261232 13987 connection.cc:256] RPC call timeout handler was delayed by -1.1733845679999604s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
I20260709 13:06:02.183357 18545 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:02.262815 18344 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.264381 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.282500 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:02.282966 13944 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:06:02.283703 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:02.284061 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.201969 18112 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.176769 18419 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.279498 14559 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:02.290189 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:02.280258 14023 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:06:02.290985 14023 pending_rounds.cc:85] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 6
I20260709 13:06:02.182746 18226 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.258986 13838 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:02.243649 18008 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.193920 18542 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.272127 18504 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:02.288753 14555 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:02.185678 18488 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.291373 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:02.291433 14023 pending_rounds.cc:107] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:06:02.313308 14023 raft_consensus.cc:2889] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:02.314726 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:02.285486 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.274549 18487 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.297173 18548 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:02.289045 13944 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:02.291633 18370 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:02.287066 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:02.316032 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:02.282423 18477 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:02.297672 18375 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.876s	user 0.012s	sys 0.001s
I20260709 13:06:02.313354 14552 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:06:02.271683 18403 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.329854 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2660) took 1218 ms. Trace:
I20260709 13:06:02.294835 18378 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:02.329286 13914 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:02.285562 18281 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:02.305801 18468 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:02.325062 14023 raft_consensus.cc:2376] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14. Re-sending same reply.
I20260709 13:06:02.325147 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:02.322293 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:02.328799 18374 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.609s	user 0.008s	sys 0.014s
I20260709 13:06:02.262490 18407 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.324008 18487 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.331452 14019 rpcz_store.cc:276] 0709 13:06:01.112649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.112830 (+   181us) service_pool.cc:224] Handling call
0709 13:06:01.119696 (+  6866us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:01.127572 (+  7876us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:01.127613 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:01.151092 (+ 23479us) log.cc:844] Serialized 26 byte log entry
0709 13:06:01.292534 (+141442us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:01.292597 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.292669 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:02.329626 (+1036957us) raft_consensus.cc:1685] finished
0709 13:06:02.329653 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.329792 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7439,"raft.queue_time_us":7976,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":32603,"thread_start_us":112990,"threads_started":1,"wal-append.queue_time_us":195649}
W20260709 13:06:02.335995 18532 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.400s	user 0.000s	sys 0.001s
W20260709 13:06:02.342339 18527 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.417s	user 0.000s	sys 0.001s
I20260709 13:06:02.337376 18403 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.338985 18501 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:02.339605 18535 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:06:02.288422 18545 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.341854 18477 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.291101 18344 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.334010 18375 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.342338 13773 catalog_manager.cc:5697] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
W20260709 13:06:02.333674 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 43670.1us
I20260709 13:06:02.316166 18504 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:02.353384 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2652) took 1402 ms. Trace:
I20260709 13:06:02.421898 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2145) took 1493 ms. Trace:
I20260709 13:06:02.422441 13946 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:02.422344 13947 rpcz_store.cc:276] 0709 13:06:00.928125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.050550 (+122425us) service_pool.cc:224] Handling call
0709 13:06:01.050883 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.230981 (+180098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.421662 (+1190681us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54001f4ad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:02.421835 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304338688}
I20260709 13:06:02.354456 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.329770 18492 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.327329 18522 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:02.358490 18460 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Append to log took a long time: real 2.433s	user 0.013s	sys 0.013s
I20260709 13:06:02.358989 18407 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:02.336418 18528 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.437s	user 0.000s	sys 0.001s
I20260709 13:06:02.361346 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:02.364166 18405 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.348435 18540 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:02.364980 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.365955 18157 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.345578 18391 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:02.368216 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2659) took 1289 ms. Trace:
I20260709 13:06:02.371279 18488 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:02.460045 17726 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:06:02.372157 18515 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.333997 18448 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.344799 18468 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.327319 18544 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:06:02.379370 18525 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:02.386920 18549 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 13:06:02.387455 18538 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
W20260709 13:06:02.387882 18541 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
W20260709 13:06:02.393074 18220 consensus_meta.cc:287] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.188s	user 0.008s	sys 0.004s
I20260709 13:06:02.477393 18553 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:02.356405 18511 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.811s	user 0.012s	sys 0.000s
I20260709 13:06:02.344132 13871 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:06:02.372448 18419 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.478873 14552 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:02.402671 13987 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:02.404314 18545 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:02.405229 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:02.481448 18511 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.472512 18468 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:02.399736 18501 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:02.483729 18501 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.484539 18511 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.485755 18501 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:02.495389 13941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2141) took 1604 ms (client timeout 1630 ms). Trace:
W20260709 13:06:02.495898 13941 rpcz_store.cc:269] 0709 13:06:00.890624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.890794 (+   170us) service_pool.cc:224] Handling call
0709 13:06:02.495166 (+1604372us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b5400085a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:02.495321 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":769192320}
I20260709 13:06:02.497313 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.408304 18112 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.372802 18495 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:02.412485 18406 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.413291 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2140) took 1523 ms. Trace:
I20260709 13:06:02.415814 14556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 604) took 2898 ms. Trace:
I20260709 13:06:02.499646 13945 rpcz_store.cc:276] 0709 13:06:00.890232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.890383 (+   151us) service_pool.cc:224] Handling call
0709 13:06:00.890749 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.891146 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:00.891175 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:00.891725 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:06:02.260273 (+1368548us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b18003d8220. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:06:02.313644 (+ 53371us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:02.313687 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.313763 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:02.412902 (+ 99139us) raft_consensus.cc:1685] finished
0709 13:06:02.412928 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.413202 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6031,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":20931,"spinlock_wait_cycles":93133056,"thread_start_us":53147,"threads_started":1,"wal-append.queue_time_us":1430955}
I20260709 13:06:02.501857 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.502524 13945 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 6.
I20260709 13:06:02.502800 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.414484 18507 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.418520 18008 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.423887 14023 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 12: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:02.421260 14016 rpcz_store.cc:276] 0709 13:06:00.951355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.952099 (+   744us) service_pool.cc:224] Handling call
0709 13:06:00.952461 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.952966 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:00.953341 (+   375us) log.cc:844] Serialized 14 byte log entry
0709 13:06:01.178154 (+224813us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:01.179176 (+  1022us) log.cc:844] Serialized 26 byte log entry
0709 13:06:01.179361 (+   185us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:01.179603 (+   242us) log.cc:844] Serialized 14 byte log entry
0709 13:06:01.190527 (+ 10924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.190657 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:02.352164 (+1161507us) raft_consensus.cc:1685] finished
0709 13:06:02.352224 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.353278 (+  1054us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44216,"prepare.queue_time_us":1380,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":5171,"raft.queue_time_us":54855,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":19619,"spinlock_wait_cycles":12062080,"thread_start_us":190674,"threads_started":1,"wal-append.queue_time_us":322875}
I20260709 13:06:02.504448 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.505110 13945 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:06:02.465935 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 106197us
I20260709 13:06:02.466899 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.465232 14019 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:06:02.468107 14022 rpcz_store.cc:276] 0709 13:06:01.079012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.121000 (+ 41988us) service_pool.cc:224] Handling call
0709 13:06:01.142438 (+ 21438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:01.181960 (+ 39522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:01.182010 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:01.198722 (+ 16712us) log.cc:844] Serialized 26 byte log entry
0709 13:06:01.333069 (+134347us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:01.333126 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.333244 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:02.367954 (+1034710us) raft_consensus.cc:1685] finished
0709 13:06:02.367979 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.368144 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12495,"raft.queue_time_us":18867,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":9042,"thread_start_us":129189,"threads_started":1,"wal-append.queue_time_us":502557}
W20260709 13:06:02.471346 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 124971us
I20260709 13:06:02.475059 18344 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.475543 18375 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.476917 18487 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.509363 18495 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:06:02.477981 18553 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.480443 18525 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:02.480824 18392 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.248s	user 0.013s	sys 0.001s
I20260709 13:06:02.355516 18281 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:06:02.470171 18391 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:06:02.489194 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 86611.9us
I20260709 13:06:02.528241 18391 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:02.489918 18504 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:02.468007 18405 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.529810 13869 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
W20260709 13:06:02.493826 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 1.06153e+06us
I20260709 13:06:02.500545 14556 rpcz_store.cc:276] 0709 13:05:59.517523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.614764 (+ 97241us) service_pool.cc:224] Handling call
0709 13:05:59.615078 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:00.694580 (+1079502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.415466 (+1720886us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54005302d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:02.415733 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2480510720}
I20260709 13:06:02.373598 18508 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.535089 18159 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:02.543083 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:02.506556 13947 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:02.510612 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:02.545356 13945 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:02.511981 14524 leader_election.cc:336] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.673s (SENT)
I20260709 13:06:02.514370 14019 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:02.478953 13944 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:02.519941 14016 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:02.521734 18507 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.506760 18432 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:02.522387 18112 consensus_queue.cc:1059] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:06:02.525619 18392 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.526430 18553 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.556015 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.556895 13947 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:02.558452 18491 raft_consensus.cc:993] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:06:02.559152 18491 raft_consensus.cc:1081] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:06:02.559577 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:02.528646 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:02.508620 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.560715 13941 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:06:02.561975 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:02.562747 13941 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:02.537714 18508 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.508735 13988 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:02.539582 13875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2783) took 2112 ms (client timeout 2053 ms). Trace:
I20260709 13:06:02.549166 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:02.566488 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 57828us
I20260709 13:06:02.524363 18501 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:02.514523 18066 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:06:02.557796 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:02.566223 13875 rpcz_store.cc:269] 0709 13:06:00.427019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.440595 (+ 13576us) service_pool.cc:224] Handling call
0709 13:06:02.539358 (+2098763us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b540029e354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:02.539493 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1815551232}
I20260709 13:06:02.567986 14522 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:02.549875 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.579262 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:02.571802 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:02.578096 13947 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:02.570179 18553 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:02.581838 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2163) took 1253 ms. Trace:
I20260709 13:06:02.581925 18553 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.582089 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:02.553562 18157 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:02.575860 18276 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.583379 18487 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:02.583474 18504 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:02.583758 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2715) took 1362 ms. Trace:
I20260709 13:06:02.579654 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:02.584273 13868 rpcz_store.cc:276] 0709 13:06:01.221415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.349616 (+128201us) service_pool.cc:224] Handling call
0709 13:06:02.583541 (+1233925us) spinlock_profiling.cc:243] Waited 95.6 ms on lock 0x7b54002e4ad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:02.583696 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219785984}
I20260709 13:06:02.582295 13947 rpcz_store.cc:276] 0709 13:06:01.328390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.557528 (+1229138us) service_pool.cc:224] Handling call
0709 13:06:02.581773 (+ 24245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:02.585791 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:02.589449 13868 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:06:02.591848 13868 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:06:02.593946 13868 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:02.596244 13868 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:02.599303 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:02.582508 18553 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:02.597807 18432 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:02.602298 18417 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:02.602116 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.602596 18344 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:02.605945 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:02.606091 18370 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:02.587136 18501 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:02.608688 13869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2696) took 2169 ms (client timeout 2091 ms). Trace:
I20260709 13:06:02.611045 13840 leader_election.cc:400] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13.
W20260709 13:06:02.621654 13869 rpcz_store.cc:269] 0709 13:06:00.439785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.457903 (+ 18118us) service_pool.cc:224] Handling call
0709 13:06:02.608477 (+2150574us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b54000a9454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:02.608606 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4719731456}
I20260709 13:06:02.617666 18417 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.624292 18378 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:02.624881 18417 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:02.625149 18501 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.625450 18378 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:06:02.626478 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.619313 13947 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:02.619329 13987 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:02.629468 13945 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:02.625844 18378 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:02.609409 18159 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:02.636401 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:02.639494 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.612026 18432 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:02.648080 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:02.627552 18544 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:02.656934 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:02.650801 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.609161 18276 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:06:02.628720 18376 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:02.660908 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:02.661959 14016 raft_consensus.cc:2410] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:06:02.662904 14019 pending_rounds.cc:85] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 6
I20260709 13:06:02.663226 14019 pending_rounds.cc:107] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:06:02.663470 14019 raft_consensus.cc:2889] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:02.617856 18470 consensus_queue.cc:1059] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:06:02.664819 18554 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:06:02.672708 18276 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:02.673262 18276 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:06:02.674348 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.674270 18276 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.641830 18344 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:02.676373 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.687879 13947 raft_consensus.cc:2410] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:06:02.645881 18066 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:02.650445 18370 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:02.650642 18432 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:02.663893 18376 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.655623 18544 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:02.703706 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.665215 18540 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:02.688774 18220 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:02.708434 13941 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:02.709144 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:02.712602 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.713796 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.716023 13941 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:06:02.686834 18344 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:02.698009 18066 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:02.666486 18042 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:06:02.702358 18370 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.705561 18376 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.703688 18276 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:02.666168 18468 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.715319 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:02.723277 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.706840 18544 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:02.698303 18488 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:06:02.727149 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:02.697680 18522 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:06:02.724375 13947 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:02.737562 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:02.738505 13941 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:02.725076 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 47086.8us
I20260709 13:06:02.703411 17726 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:06:02.707769 18540 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.720566 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.720106 18066 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.665328 18378 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.737010 18448 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:02.727617 18370 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:02.748839 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.736843 18525 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.727026 18501 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:02.744273 18540 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:02.787184 13947 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:06:02.764772 18392 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.751345 18559 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.765944 18448 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:02.779829 18378 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:02.813726 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.774663 18525 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:02.808853 18557 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:06:02.785899 18501 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.808271 18487 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.821208 18487 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.823448 18487 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:02.826115 13941 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:02.826306 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.827234 13947 raft_consensus.cc:2410] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:06:02.827172 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.828075 13941 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:02.828929 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.830063 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:02.836158 14024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 1971) took 4822 ms (client timeout 4505 ms). Trace:
I20260709 13:06:02.836436 14022 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:06:02.836648 14024 rpcz_store.cc:269] 0709 13:05:58.013798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.581480 (+567682us) service_pool.cc:224] Handling call
0709 13:06:02.835873 (+4254393us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b540004b354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:02.836049 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5395958016}
I20260709 13:06:02.840289 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:02.827495 13838 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:02.781765 18344 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.844655 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.824565 17726 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:02.846252 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:02.847070 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.849735 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:02.849952 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.850445 14024 raft_consensus.cc:2376] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13. Re-sending same reply.
I20260709 13:06:02.851068 18562 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:02.851369 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.851975 14024 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:06:02.854607 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:02.855264 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.855919 13950 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:02.856383 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2159) took 1580 ms. Trace:
I20260709 13:06:02.856304 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.856843 13950 rpcz_store.cc:276] 0709 13:06:01.276142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.855012 (+1578870us) service_pool.cc:224] Handling call
0709 13:06:02.856314 (+  1302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:02.857251 18470 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:02.857688 18470 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.858074 18470 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:02.799950 18559 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.859807 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.857813 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:02.862291 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.883814 14024 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 6.
I20260709 13:06:02.884843 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.884989 13840 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:02.862936 18161 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:02.863798 13941 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:02.867849 18492 raft_consensus.cc:3055] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:02.865900 18505 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:02.889868 13838 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:02.859488 17726 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.870805 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2021) took 1052 ms. Trace:
I20260709 13:06:02.891465 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.891634 14021 rpcz_store.cc:276] 0709 13:06:01.818726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.819480 (+   754us) service_pool.cc:224] Handling call
0709 13:06:02.870617 (+1051137us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54003bb0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:02.870741 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766905216}
W20260709 13:06:02.872560 13946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2152) took 1695 ms (client timeout 2044 ms). Trace:
I20260709 13:06:02.844777 18042 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:02.880960 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.892719 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.881767 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2146) took 1953 ms. Trace:
W20260709 13:06:02.892899 13946 rpcz_store.cc:269] 0709 13:06:01.176683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.177216 (+   533us) service_pool.cc:224] Handling call
0709 13:06:02.872301 (+1695085us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54003309d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:02.872459 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2734658816}
I20260709 13:06:02.893411 13943 rpcz_store.cc:276] 0709 13:06:00.928448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.057669 (+129221us) service_pool.cc:224] Handling call
0709 13:06:01.058110 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.125325 (+1067215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.869124 (+743799us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b5400086e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:02.881688 (+ 12564us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2065865472}
I20260709 13:06:02.894152 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.894554 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:02.880460 18405 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.888784 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:02.885702 13950 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:02.856855 18344 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:02.894976 13946 raft_consensus.cc:2410] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:06:02.892729 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.887010 18159 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:02.897790 18509 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:02.894542 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.920398 13943 raft_consensus.cc:2410] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:06:02.895562 13838 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:02.918749 13941 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:02.908857 14015 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:02.919345 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:02.922096 13946 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:02.886165 13947 raft_consensus.cc:2410] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:06:02.910531 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:02.911233 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:02.925982 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.926779 13943 raft_consensus.cc:2410] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:06:02.888499 18490 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:02.935182 13950 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:06:02.936619 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:02.937506 13943 raft_consensus.cc:2410] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:06:02.889070 18226 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.888397 18468 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.905706 18371 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:02.890843 18492 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:02.945542 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2153) took 1768 ms. Trace:
I20260709 13:06:02.956619 13943 rpcz_store.cc:276] 0709 13:06:01.177133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.944604 (+1767471us) service_pool.cc:224] Handling call
0709 13:06:02.944963 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.945184 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:02.945212 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:02.945260 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:02.945298 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.945358 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.945478 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:02.946282 13946 raft_consensus.cc:2410] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:06:02.924311 18042 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:02.947518 18525 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.948055 18501 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:02.946094 18157 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:02.966388 18157 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.966773 18157 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:02.967445 17726 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:02.967551 18417 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.954798 18371 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.969257 18371 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:02.972357 18370 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.974359 13868 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:06:02.976079 18042 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.922392 18553 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.948498 18391 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.939690 18490 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.959110 18405 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:02.920298 18159 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:02.860070 18448 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:02.939358 18562 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.964349 13947 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:02.963701 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:02.984200 13941 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
W20260709 13:06:02.987588 13838 leader_election.cc:336] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Timed out: RequestConsensusVote RPC to 127.12.109.66:43953 timed out after 1.655s (SENT)
I20260709 13:06:02.997140 13838 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:02.955964 18112 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:06:02.989800 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2156) took 1755 ms. Trace:
I20260709 13:06:02.953105 18432 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:02.996739 14015 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:02.996716 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2155) took 1818 ms. Trace:
I20260709 13:06:02.953888 18220 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:02.999104 14020 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:02.969925 18417 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:03.002629 13946 rpcz_store.cc:276] 0709 13:06:01.234630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.988652 (+1754022us) service_pool.cc:224] Handling call
0709 13:06:02.989174 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.989387 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:02.989413 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:02.989484 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:02.989540 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.989607 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.989735 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.005359 13947 rpcz_store.cc:276] 0709 13:06:01.178672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.961109 (+1782437us) service_pool.cc:224] Handling call
0709 13:06:02.961662 (+   553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.996390 (+ 34728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.996646 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:02.992688 18490 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:03.010164 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:03.011749 14015 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:03.012653 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:03.013274 14015 raft_consensus.cc:2376] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 16. Re-sending same reply.
I20260709 13:06:03.014281 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:03.016883 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:03.017570 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.018805 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 752) took 1679 ms. Trace:
I20260709 13:06:03.019629 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:03.019243 13947 rpcz_store.cc:276] 0709 13:06:01.339696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.007517 (+1667821us) service_pool.cc:224] Handling call
0709 13:06:03.017915 (+ 10398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.018249 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.018277 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:03.018499 (+   222us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:03.018543 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.018616 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.018742 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.020558 14015 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:03.021510 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:03.023043 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:03.001045 18370 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:03.011030 18157 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.031862 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:03.047672 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:03.047950 13944 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:03.048713 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2662) took 1025 ms. Trace:
I20260709 13:06:03.048897 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:03.049060 13944 rpcz_store.cc:276] 0709 13:06:02.023719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.047377 (+1023658us) service_pool.cc:224] Handling call
0709 13:06:03.047738 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.048455 (+   717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.048652 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.024363 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:03.050717 14015 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:03.038990 18507 raft_consensus.cc:3055] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:03.051365 18507 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:03.052655 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2664) took 1028 ms. Trace:
I20260709 13:06:03.053092 13949 rpcz_store.cc:276] 0709 13:06:02.024234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.051579 (+1027345us) service_pool.cc:224] Handling call
0709 13:06:03.051907 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.052284 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:03.052317 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:03.052362 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:03.052403 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.052473 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.052595 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.054329 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:03.054915 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2665) took 1030 ms. Trace:
I20260709 13:06:03.055480 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:03.056658 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:03.056902 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:03.057117 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:03.058200 13944 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:03.058534 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:03.058945 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2663) took 1035 ms. Trace:
I20260709 13:06:03.050817 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2709) took 1834 ms. Trace:
I20260709 13:06:03.052289 18507 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.057219 14021 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:03.040325 13914 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:03.049757 14018 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:03.060103 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:03.059620 13874 rpcz_store.cc:276] 0709 13:06:01.216366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.525438 (+309072us) service_pool.cc:224] Handling call
0709 13:06:01.560901 (+ 35463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.711892 (+150991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.050525 (+1338633us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540026c5d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:03.050750 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268777216}
I20260709 13:06:03.058493 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:03.076521 14560 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:03.076691 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:03.077836 18157 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:03.078083 18041 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.078408 18525 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:03.078904 18525 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:03.078668 18041 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.079340 18525 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:03.079702 18392 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:03.080283 18392 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:03.081084 18041 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:03.035128 18066 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.061463 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:03.083017 18540 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:03.083441 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2708) took 1867 ms. Trace:
I20260709 13:06:03.084275 14551 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:03.085280 18540 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:03.059715 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:03.087726 14553 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:03.055279 13949 rpcz_store.cc:276] 0709 13:06:02.024502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.054120 (+1029618us) service_pool.cc:224] Handling call
0709 13:06:03.054398 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.054609 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.054638 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:03.054682 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:03.054721 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.054771 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.054866 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.089255 13949 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:03.059442 13944 rpcz_store.cc:276] 0709 13:06:02.023980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.057666 (+1033686us) service_pool.cc:224] Handling call
0709 13:06:03.057978 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.058684 (+   706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.058878 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.090919 13944 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:03.091565 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2668) took 1066 ms. Trace:
I20260709 13:06:03.091895 13944 rpcz_store.cc:276] 0709 13:06:02.025285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.090419 (+1065134us) service_pool.cc:224] Handling call
0709 13:06:03.090750 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.091291 (+   541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.091491 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.089906 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2667) took 1064 ms. Trace:
I20260709 13:06:03.094362 13949 rpcz_store.cc:276] 0709 13:06:02.025031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.088682 (+1063651us) service_pool.cc:224] Handling call
0709 13:06:03.089055 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.089651 (+   596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.089844 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.095434 13942 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 12. Current term is 13. Ops: []
W20260709 13:06:03.098044 18481 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 2.300s	user 0.008s	sys 0.000s
I20260709 13:06:03.098747 14554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 608) took 1183 ms. Trace:
I20260709 13:06:03.099135 14554 rpcz_store.cc:276] 0709 13:06:01.915346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.955384 (+ 40038us) service_pool.cc:224] Handling call
0709 13:06:02.119829 (+164445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.151934 (+ 32105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.098536 (+946602us) spinlock_profiling.cc:243] Waited 58.8 ms on lock 0x7b54003359d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:03.098663 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135324032}
I20260709 13:06:03.101846 14018 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:06:03.103219 14018 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:03.083869 13867 rpcz_store.cc:276] 0709 13:06:01.216105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.219472 (+  3367us) service_pool.cc:224] Handling call
0709 13:06:01.219796 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:01.309747 (+ 89951us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:01.309781 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:01.310998 (+  1217us) log.cc:844] Serialized 26 byte log entry
0709 13:06:01.434549 (+123551us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:01.434595 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.082990 (+1648395us) spinlock_profiling.cc:243] Waited 81.4 ms on lock 0x7b54000a6c54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:03.083061 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.083117 (+    56us) raft_consensus.cc:1685] finished
0709 13:06:03.083132 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.083376 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":997,"raft.queue_time_us":4972,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":12794,"spinlock_wait_cycles":187328256,"thread_start_us":92416,"threads_started":1,"wal-append.queue_time_us":505968,"wal-append.run_cpu_time_us":2223,"wal-append.run_wall_time_us":1084887}
I20260709 13:06:03.105039 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:03.105643 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.106160 14021 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:06:03.085278 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2699) took 1871 ms. Trace:
I20260709 13:06:03.107565 14021 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:06:03.108425 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.109543 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:03.071139 14015 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:03.006245 18372 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:03.074995 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:03.113214 14561 raft_consensus.cc:2410] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:06:03.106871 13872 rpcz_store.cc:276] 0709 13:06:01.213461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.215182 (+  1721us) service_pool.cc:224] Handling call
0709 13:06:01.215522 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:01.332760 (+117238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:01.332793 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:01.340247 (+  7454us) log.cc:844] Serialized 26 byte log entry
0709 13:06:01.680655 (+340408us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:01.680722 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.084840 (+1404118us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54000a8054. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:03.084903 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.084956 (+    53us) raft_consensus.cc:1685] finished
0709 13:06:03.084969 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.085207 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2188,"raft.queue_time_us":2753,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":7327,"spinlock_wait_cycles":263404416,"thread_start_us":251477,"threads_started":1,"wal-append.queue_time_us":473823,"wal-append.run_cpu_time_us":2309,"wal-append.run_wall_time_us":1023978}
I20260709 13:06:03.115983 13947 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:03.115293 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:03.122876 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:03.124419 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2661) took 1101 ms. Trace:
I20260709 13:06:03.125092 13947 rpcz_store.cc:276] 0709 13:06:02.023408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.020555 (+997147us) service_pool.cc:224] Handling call
0709 13:06:03.112004 (+ 91449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.124033 (+ 12029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.124337 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.126475 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:03.136790 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:03.132453 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:03.001644 18562 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:03.143669 18161 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:03.145841 18492 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.146405 18470 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.147256 18492 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:03.148789 18470 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:03.151917 18562 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:03.145362 18371 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.161197 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 636) took 1053 ms. Trace:
I20260709 13:06:03.161864 13871 rpcz_store.cc:276] 0709 13:06:02.107492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.145787 (+ 38295us) service_pool.cc:224] Handling call
0709 13:06:03.160944 (+1015157us) spinlock_profiling.cc:243] Waited 99.2 ms on lock 0x7b54000d9b54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:03.161106 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228240768}
I20260709 13:06:03.077456 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:03.179102 13839 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:03.199011 13947 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:03.127954 18504 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:03.135224 18329 raft_consensus.cc:2764] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:06:03.212239 14555 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:06:03.205663 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:03.128211 18417 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:03.157411 17726 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:03.230967 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:03.231666 13867 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:03.166741 18112 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:03.237986 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:03.238654 14020 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:03.241570 18432 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:03.242329 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.242570 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:03.206146 18372 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:03.227496 18507 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:03.274555 18372 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.275053 18372 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:03.278858 18371 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:03.279580 18371 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.071264 18159 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:03.246357 13873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2784) took 2818 ms (client timeout 1581 ms). Trace:
I20260709 13:06:03.280103 18371 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:03.280550 13873 rpcz_store.cc:269] 0709 13:06:00.427990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.437212 (+  9222us) service_pool.cc:224] Handling call
0709 13:06:03.246136 (+2808924us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b54002e54d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:03.246275 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1739676416}
I20260709 13:06:03.244120 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:03.209542 18504 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.232667 18417 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:03.286830 18417 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:03.236403 18376 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.289274 18376 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:03.291216 14022 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:03.241679 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:03.232720 18498 consensus_queue.cc:1059] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:06:03.234283 18470 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:03.252456 13867 raft_consensus.cc:2410] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:06:03.254819 14020 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:03.296777 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:03.297755 14020 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:03.166957 18399 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:06:03.243949 18112 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:03.132454 18544 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:03.214627 18370 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:06:03.330494 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:03.331266 14019 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:06:03.225512 18161 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:06:03.267314 18392 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:03.270582 18391 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:03.205883 18499 consensus_queue.cc:1059] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:06:03.341939 18370 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:03.342187 18492 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.342392 18370 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:03.342502 18499 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:03.227213 18559 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:03.246731 13866 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: [6.5-10.7]
I20260709 13:06:03.291945 14016 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:03.290297 18504 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:03.227721 18519 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:03.322605 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 755) took 1178 ms. Trace:
I20260709 13:06:03.351372 13949 rpcz_store.cc:276] 0709 13:06:02.144163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.095111 (+950948us) service_pool.cc:224] Handling call
0709 13:06:03.095401 (+   290us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:03.095794 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.095819 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:03.096532 (+   713us) log.cc:844] Serialized 52 byte log entry
0709 13:06:03.292542 (+196010us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:03.292593 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.292671 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.322241 (+ 29570us) raft_consensus.cc:1685] finished
0709 13:06:03.322273 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.322491 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38972,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":8340,"thread_start_us":195551,"threads_started":1,"wal-append.queue_time_us":222565}
I20260709 13:06:03.328359 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:03.353989 13949 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:03.354269 14020 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:03.356020 14020 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:06:03.294382 18520 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.551s	user 0.014s	sys 0.014s
I20260709 13:06:03.357681 14020 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:03.359809 18520 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.640s	user 0.014s	sys 0.015s
I20260709 13:06:03.360208 18520 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:03.361286 18520 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.342935 18499 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:03.338708 14022 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:03.341291 18371 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:03.345953 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:03.308341 18112 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:03.342821 18492 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:03.306150 18470 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.283090 18066 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:03.369997 14022 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:03.400393 14024 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:06:03.401121 14019 raft_consensus.cc:2410] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:06:03.403419 13947 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:03.404230 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:03.424046 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:03.426479 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:03.418502 18531 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.302s	user 0.008s	sys 0.008s
I20260709 13:06:03.431545 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:03.365885 18405 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:03.344830 18525 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:03.435237 13840 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:03.429425 18276 consensus_meta.cc:287] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.002s
I20260709 13:06:03.366972 18499 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.343400 18370 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:03.433739 18505 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.004s
I20260709 13:06:03.430019 18468 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:03.429718 18371 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:03.434685 18531 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.323s	user 0.008s	sys 0.009s
I20260709 13:06:03.420087 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2161) took 2123 ms. Trace:
I20260709 13:06:03.455847 14023 raft_consensus.cc:2368] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 12.
I20260709 13:06:03.452175 13840 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:03.457294 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:03.420177 18399 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 13:06:03.447546 18378 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.631s	user 0.012s	sys 0.000s
I20260709 13:06:03.435359 18417 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:03.436455 18042 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:03.448529 18476 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:03.459223 13946 rpcz_store.cc:276] 0709 13:06:01.296897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.007274 (+1710377us) service_pool.cc:224] Handling call
0709 13:06:03.007637 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.007866 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:03.296389 (+288523us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.419556 (+123167us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:03.419640 (+    84us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:03.419722 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.419828 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.420006 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23817,"prepare.queue_time_us":304576,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":2309,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":10953,"thread_start_us":389450,"threads_started":2,"wal-append.queue_time_us":148528}
I20260709 13:06:03.468786 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 753) took 1347 ms. Trace:
I20260709 13:06:03.471014 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:03.479676 18476 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.480109 13946 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:03.480309 18476 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:03.481493 18377 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:03.452577 18041 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:06:03.489068 18525 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:03.493892 13988 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:03.494252 18476 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:03.495070 18399 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:03.500682 18399 raft_consensus.cc:697] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:03.501734 18399 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:03.502348 18226 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:03.502553 18544 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:03.502888 18226 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:03.495813 13988 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:03.503563 18470 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:03.504370 13988 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:03.504070 18226 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.505522 18226 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:03.506318 18276 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:03.507160 18492 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:03.507654 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.478839 18468 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:03.455463 18405 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:06:03.470089 18370 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:03.479277 13944 rpcz_store.cc:276] 0709 13:06:02.121228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.092670 (+971442us) service_pool.cc:224] Handling call
0709 13:06:03.092964 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.093175 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:03.312184 (+219009us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.468373 (+156189us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:03.468436 (+    63us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:03.468492 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.468570 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.468717 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":219526,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":360,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":12352,"thread_start_us":374018,"threads_started":2}
I20260709 13:06:03.514842 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:03.440881 18490 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:03.515437 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:03.525499 13867 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:03.517534 13946 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:06:03.470739 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 758) took 1178 ms. Trace:
I20260709 13:06:03.516973 13944 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:03.468526 18499 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:03.528183 13945 rpcz_store.cc:276] 0709 13:06:02.292405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.100936 (+808531us) service_pool.cc:224] Handling call
0709 13:06:03.101216 (+   280us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:03.248731 (+147515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.248792 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:03.255061 (+  6269us) log.cc:844] Serialized 52 byte log entry
0709 13:06:03.323450 (+ 68389us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:03.323517 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.323630 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.470530 (+146900us) raft_consensus.cc:1685] finished
0709 13:06:03.470553 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.470681 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44705,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":64,"thread_start_us":210043,"threads_started":2,"wal-append.queue_time_us":67386}
I20260709 13:06:03.506273 18498 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:03.546217 13873 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:03.549108 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:03.550765 13947 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:06:03.537039 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 83842.3us
I20260709 13:06:03.556844 13872 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:06:03.533779 13946 pending_rounds.cc:85] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 5
I20260709 13:06:03.552759 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.560443 13946 pending_rounds.cc:107] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:06:03.560997 13946 raft_consensus.cc:2889] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:03.524377 18559 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.566380 13870 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:03.570463 14556 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:03.633862 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:03.592340 18553 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:03.599121 18498 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:06:03.643339 18585 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:06:03.664599 13867 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:06:03.665917 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:03.669013 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.694330 13874 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:03.638660 18042 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:03.695631 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.697244 13867 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:03.698582 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:03.699661 13869 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:03.698678 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.636328 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:03.700712 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:03.701650 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 65582.4us
I20260709 13:06:03.703095 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:03.620743 18559 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:03.704110 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.715826 13867 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:03.637701 18492 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:06:03.727650 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.717252 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.738894 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.738989 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.740964 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.741303 13874 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:03.739224 13947 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:03.751482 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2717) took 2529 ms. Trace:
I20260709 13:06:03.751929 13866 rpcz_store.cc:276] 0709 13:06:01.221928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.727296 (+505368us) service_pool.cc:224] Handling call
0709 13:06:03.246346 (+1519050us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:03.347463 (+101117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.751279 (+403816us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54002e54d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:03.751421 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3282390784}
I20260709 13:06:03.752439 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:03.747272 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:03.726321 18428 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:03.704281 18416 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:03.807821 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 55473.2us
I20260709 13:06:03.778826 13869 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:03.736814 18042 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:03.812665 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:03.816123 13915 connection.cc:256] RPC call timeout handler was delayed by -1.9636441129999866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
I20260709 13:06:03.817605 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:03.819007 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 65032.6us
I20260709 13:06:03.811345 18041 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:03.824066 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.775127 18470 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:03.826103 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2160) took 2536 ms. Trace:
I20260709 13:06:03.826570 13874 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:03.828081 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:03.840516 13869 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:03.750818 18417 consensus_queue.cc:1059] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:06:03.830717 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:03.831213 13838 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:03.831596 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:03.838559 13941 rpcz_store.cc:276] 0709 13:06:01.289496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.999657 (+1710161us) service_pool.cc:224] Handling call
0709 13:06:02.999983 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.004320 (+  4337us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:03.207122 (+202802us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.443276 (+236154us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.534301 (+ 91025us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.534521 (+   220us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:03.534562 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.534644 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.825776 (+291132us) raft_consensus.cc:1685] finished
0709 13:06:03.825809 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.826005 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5566,"prepare.queue_time_us":307811,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":6746,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":11229,"thread_start_us":263299,"threads_started":2,"wal-append.queue_time_us":195575}
W20260709 13:06:03.824406 13915 connection.cc:256] RPC call timeout handler was delayed by -1.9060441130000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
W20260709 13:06:03.851449 13915 connection.cc:256] RPC call timeout handler was delayed by -1.9060441130000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
W20260709 13:06:03.851758 13915 connection.cc:256] RPC call timeout handler was delayed by -1.8979441129999941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 13:06:03.852039 13915 connection.cc:256] RPC call timeout handler was delayed by -1.833144113000003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 13:06:03.852330 13915 connection.cc:256] RPC call timeout handler was delayed by -1.7629441130000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
W20260709 13:06:03.852602 13915 connection.cc:256] RPC call timeout handler was delayed by -1.7161441130000412s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
W20260709 13:06:03.852877 13915 connection.cc:256] RPC call timeout handler was delayed by -1.7116441130000339s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
W20260709 13:06:03.853155 13915 connection.cc:256] RPC call timeout handler was delayed by -1.6765441129999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
W20260709 13:06:03.853447 13915 connection.cc:256] RPC call timeout handler was delayed by -1.6477441130000443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 13:06:03.853724 13915 connection.cc:256] RPC call timeout handler was delayed by -1.4812441130000025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
W20260709 13:06:03.853991 13915 connection.cc:256] RPC call timeout handler was delayed by -1.4812441130000025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
W20260709 13:06:03.854305 13915 connection.cc:256] RPC call timeout handler was delayed by -1.3687441130000479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
W20260709 13:06:03.854571 13915 connection.cc:256] RPC call timeout handler was delayed by -1.3579441130000305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
W20260709 13:06:03.854842 13915 connection.cc:256] RPC call timeout handler was delayed by -1.3309441129999868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
W20260709 13:06:03.855123 13915 connection.cc:256] RPC call timeout handler was delayed by -1.2463441130000774s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23140000000000002s for a response
W20260709 13:06:03.855404 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0763660169999412s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
W20260709 13:06:03.855679 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0079660169999443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 13:06:03.855950 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0079660169999443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 13:06:03.856235 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0061660169999413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
I20260709 13:06:03.856904 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:03.859088 13875 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:03.829012 18476 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:03.861284 18372 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.001s
W20260709 13:06:03.838619 18586 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.367s	user 0.001s	sys 0.000s
I20260709 13:06:03.863966 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.860699 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.828588 18428 raft_consensus.cc:697] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:03.866786 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:03.830312 13947 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:58096
I20260709 13:06:03.879446 13947 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.879850 13947 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:03.885766 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:03.858397 18470 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:03.864428 18416 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.824903 18468 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:06:03.913066 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.864537 18490 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:03.874990 18492 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:06:03.911401 18587 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 13:06:03.895529 13945 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:03.913575 18041 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:06:03.909732 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 52917.4us
I20260709 13:06:03.945076 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:03.913007 18531 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:03.843732 18498 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:03.972100 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:03.975135 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:04.001183 14552 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:03.977326 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:03.976130 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:03.948972 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:03.946004 13947 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:04.016877 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 193965us
I20260709 13:06:04.029866 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:04.031014 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:04.031298 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:04.032140 13875 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:04.032260 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:04.032858 13874 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:04.033468 18428 consensus_queue.cc:237] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.040879 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:03.920804 18416 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.042701 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:04.043294 13945 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:04.044273 18416 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:04.006357 18041 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:04.045986 18041 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:04.048880 18559 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:04.049201 18370 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.003s
I20260709 13:06:04.050606 13874 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:06:04.050213 18470 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.051173 18548 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:04.051620 18548 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:04.052544 18548 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.053661 18548 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:04.010772 18492 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:04.055140 18492 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.057238 18488 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:04.057750 18488 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:04.058092 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:04.058960 13949 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:04.006692 18405 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:06:04.055455 13947 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:04.062749 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:04.068758 13947 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:04.069133 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2024) took 2187 ms. Trace:
I20260709 13:06:04.073218 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:04.079903 13875 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:04.065155 14556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59630 (request call id 619) took 1007 ms. Trace:
I20260709 13:06:04.082038 14556 rpcz_store.cc:276] 0709 13:06:03.057986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.058240 (+   254us) service_pool.cc:224] Handling call
0709 13:06:04.064908 (+1006668us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54004b0fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:04.065065 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934052608}
I20260709 13:06:03.869481 18372 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:03.998966 18579 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
W20260709 13:06:03.999303 18591 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
I20260709 13:06:04.091872 13945 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:04.069545 14017 rpcz_store.cc:276] 0709 13:06:01.881412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.489803 (+608391us) service_pool.cc:224] Handling call
0709 13:06:02.490236 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:02.530129 (+ 39893us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:02.530162 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:02.536431 (+  6269us) log.cc:844] Serialized 26 byte log entry
0709 13:06:02.654380 (+117949us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:02.779259 (+124879us) log.cc:844] Serialized 14 byte log entry
0709 13:06:02.779534 (+   275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.061242 (+1281708us) spinlock_profiling.cc:243] Waited 39.4 ms on lock 0x7b54002eac54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:04.061293 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.061342 (+    49us) raft_consensus.cc:1685] finished
0709 13:06:04.061440 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.069071 (+  7631us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2623,"prepare.queue_time_us":88364,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":825,"raft.queue_time_us":7924,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":15086,"spinlock_wait_cycles":90731904,"thread_start_us":173424,"threads_started":2,"wal-append.queue_time_us":193471,"wal-append.run_cpu_time_us":2746,"wal-append.run_wall_time_us":1080422}
I20260709 13:06:04.091521 18066 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:04.097899 18595 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:06:04.098701 18594 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
W20260709 13:06:04.099520 18596 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:06:04.102236 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:04.102684 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:04.191094 13947 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:04.193029 13947 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:04.103590 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:04.063714 18499 consensus_meta.cc:287] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.004s
I20260709 13:06:03.910830 18042 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.113683 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 520) took 1383 ms. Trace:
I20260709 13:06:04.058637 18488 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:04.111047 18581 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.374s	user 0.000s	sys 0.006s
W20260709 13:06:04.108745 18590 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.415s	user 0.006s	sys 0.000s
I20260709 13:06:04.231274 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:04.127135 18592 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.313s	user 0.001s	sys 0.001s
I20260709 13:06:04.188771 13944 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
W20260709 13:06:04.131588 18584 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
I20260709 13:06:04.124104 18276 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.217207 14021 rpcz_store.cc:276] 0709 13:06:02.730369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.108480 (+378111us) service_pool.cc:224] Handling call
0709 13:06:03.108841 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.109281 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.109321 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.109887 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.418170 (+308283us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:03.436930 (+ 18760us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.437176 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.437262 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.113440 (+676178us) raft_consensus.cc:1685] finished
0709 13:06:04.113466 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.113610 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8394,"prepare.queue_time_us":18466,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":3907,"raft.queue_time_us":7777,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":17375,"thread_start_us":257874,"threads_started":1,"wal-append.queue_time_us":497676}
I20260709 13:06:04.254527 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:04.220811 13945 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:04.212545 18400 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:06:04.181931 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 233033us
I20260709 13:06:04.235644 13943 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:04.236382 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 642) took 1659 ms. Trace:
W20260709 13:06:04.236524 14023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2666) took 1597 ms (client timeout 1586 ms). Trace:
I20260709 13:06:04.237634 14018 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
W20260709 13:06:04.188234 18504 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.818s	user 0.012s	sys 0.000s
I20260709 13:06:04.259980 14018 pending_rounds.cc:85] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 6
I20260709 13:06:04.260295 14018 pending_rounds.cc:107] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:06:04.260564 14018 raft_consensus.cc:2889] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:04.261277 14023 rpcz_store.cc:269] 0709 13:06:02.638911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.639149 (+   238us) service_pool.cc:224] Handling call
0709 13:06:04.236293 (+1597144us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54000ab754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:04.236449 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1875457920}
I20260709 13:06:04.263274 13868 rpcz_store.cc:276] 0709 13:06:02.576615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.597164 (+ 20549us) service_pool.cc:224] Handling call
0709 13:06:02.601369 (+  4205us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:02.974850 (+373481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.236075 (+1261225us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b54001aa254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:04.236307 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":857044224}
W20260709 13:06:04.231712 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:04.221889 18112 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.788s	user 0.011s	sys 0.000s
I20260709 13:06:04.265067 13872 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:04.256969 18376 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:04.269361 18504 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:04.272600 18226 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.767s	user 0.004s	sys 0.008s
I20260709 13:06:04.276991 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:04.279168 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:04.349249 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 522) took 1125 ms. Trace:
I20260709 13:06:04.282366 18499 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.283902 18157 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:04.286911 18161 raft_consensus.cc:3055] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:06:04.288949 13948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2162) took 2962 ms (client timeout 1655 ms). Trace:
I20260709 13:06:04.179975 18066 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.254254 18378 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:04.290673 18583 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.422s	user 0.000s	sys 0.001s
I20260709 13:06:04.290925 18520 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.291877 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:04.366827 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 75072.5us
I20260709 13:06:04.318019 18400 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:04.266429 18490 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.293366 18508 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:04.295998 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2684) took 1594 ms. Trace:
I20260709 13:06:04.380041 13950 rpcz_store.cc:276] 0709 13:06:02.702100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.244970 (+542870us) service_pool.cc:224] Handling call
0709 13:06:03.245278 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.245508 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.245533 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.248845 (+  3312us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.439390 (+190545us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:03.439452 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.439522 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.295746 (+856224us) raft_consensus.cc:1685] finished
0709 13:06:04.295771 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.295927 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":190161,"threads_started":1,"wal-append.queue_time_us":276110}
I20260709 13:06:04.352309 18253 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:04.301357 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:04.389991 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:04.390597 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:04.390976 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.304225 13868 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:06:04.395864 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:04.339534 13869 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:04.349855 13875 raft_consensus.cc:2410] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:06:04.267431 18417 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.353470 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:04.401167 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:04.401695 13872 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
W20260709 13:06:04.353211 13948 rpcz_store.cc:269] 0709 13:06:01.326163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.351234 (+ 25071us) service_pool.cc:224] Handling call
0709 13:06:04.288572 (+2937338us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b54001c6954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:04.288796 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4794143488}
W20260709 13:06:04.363308 13989 connection.cc:256] RPC call timeout handler was delayed by -1.2136491869999873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20910000000000004s for a response
I20260709 13:06:04.349756 14020 rpcz_store.cc:276] 0709 13:06:03.224000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.358567 (+134567us) service_pool.cc:224] Handling call
0709 13:06:03.358881 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.359327 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.359361 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.366314 (+  6953us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.454007 (+ 87693us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:03.454084 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.454246 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.349024 (+894778us) raft_consensus.cc:1685] finished
0709 13:06:04.349048 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.349181 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":87078,"threads_started":1,"wal-append.queue_time_us":225238}
I20260709 13:06:04.338704 18376 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:04.436162 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:04.364727 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 388668us
I20260709 13:06:04.437296 13875 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 7: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:04.261845 18276 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.376870 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2164) took 2193 ms. Trace:
I20260709 13:06:04.437942 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:04.446581 13942 rpcz_store.cc:276] 0709 13:06:02.183763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.097442 (+913679us) service_pool.cc:224] Handling call
0709 13:06:03.097776 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.317708 (+219932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.317747 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.318408 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.553136 (+234728us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:03.553181 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.553283 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.376629 (+823346us) raft_consensus.cc:1685] finished
0709 13:06:04.376651 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.376796 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":47930,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":114,"thread_start_us":441407,"threads_started":2,"wal-append.queue_time_us":418180}
I20260709 13:06:04.377502 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:04.449155 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 71701.9us
I20260709 13:06:04.450585 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:04.245160 13867 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:06:04.306252 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 411102us
I20260709 13:06:04.405448 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:06:04.424288 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.432564 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:04.454327 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:04.414527 13989 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:04.465453 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:04.466320 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:04.466918 13867 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:04.352681 18157 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:04.352864 18161 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:06:04.483194 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 45325.6us
I20260709 13:06:04.485008 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:04.486765 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:04.487991 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:04.480216 13838 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:04.468955 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:04.492005 14552 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:04.492467 14552 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:04.488799 14554 raft_consensus.cc:1813] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Handling vote request from an unknown peer 3779649036364744823fc035dac892c7
I20260709 13:06:04.493513 14552 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:04.495736 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:04.496053 14552 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:04.496176 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:04.496616 13869 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:04.381501 18409 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:04.341629 18112 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:04.487013 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:04.503330 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:04.504108 13942 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:04.504606 13872 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:04.503393 14554 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:06:04.516112 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.362567 18499 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.471944 18601 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:04.536031 13869 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
W20260709 13:06:04.507292 18377 consensus_meta.cc:287] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.025s	user 0.012s	sys 0.003s
I20260709 13:06:04.546274 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:04.524410 18607 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:04.540548 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:04.558741 13942 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:04.542160 18409 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:04.389099 18490 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:04.543121 18066 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:04.543241 18598 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:06:04.388878 18525 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 13:06:04.543835 18372 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:04.538981 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:04.370642 18378 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:04.526309 18595 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:06:04.379652 18520 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.497547 18470 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:04.560508 13871 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:04.562511 13872 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:04.517078 18370 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:04.543164 18599 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 13:06:04.465034 18253 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.542245 14551 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:04.575467 14558 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:04.576596 13867 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:04.481488 18417 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:04.578693 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 128208us
I20260709 13:06:04.523074 18376 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.538990 18329 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.483835 18531 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:04.559386 18468 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.568562 18559 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:04.545365 18544 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:06:04.547605 18545 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:04.732527 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:04.735433 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 647) took 1220 ms. Trace:
W20260709 13:06:04.663448 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 249029us
I20260709 13:06:04.736104 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:04.735857 13873 rpcz_store.cc:276] 0709 13:06:03.515069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.516088 (+  1019us) service_pool.cc:224] Handling call
0709 13:06:03.523875 (+  7787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.546939 (+ 23064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.735166 (+1188227us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b54002e7cd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:04.735369 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50651904}
I20260709 13:06:04.737160 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:04.740154 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:04.744318 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:04.741677 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:04.745565 14021 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:04.742970 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:04.748301 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:04.749401 14024 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:04.636041 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2055) took 1423 ms. Trace:
I20260709 13:06:04.755650 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:04.578691 18161 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.598801 18609 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:04.644348 18112 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:04.579344 18400 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.645605 18499 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:04.647837 18276 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:04.647075 18378 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:04.766788 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:04.796434 13988 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.140s (SENT) [suppressed 28 similar messages]
W20260709 13:06:04.645505 18602 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
W20260709 13:06:04.646278 18604 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:06:04.722729 18598 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:04.736979 18545 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:04.807343 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51787.3us
I20260709 13:06:04.752784 18511 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.805783 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:04.793646 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 270137us
I20260709 13:06:04.643870 18504 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:04.754086 14016 rpcz_store.cc:276] 0709 13:06:03.212820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.348287 (+135467us) service_pool.cc:224] Handling call
0709 13:06:03.350113 (+  1826us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:03.357397 (+  7284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.357432 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:03.400648 (+ 43216us) log.cc:844] Serialized 52 byte log entry
0709 13:06:03.694455 (+293807us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:03.742844 (+ 48389us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.743217 (+   373us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.743389 (+   172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.743466 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.100603 (+357137us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b18003c1540. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:04.286212 (+185609us) spinlock_profiling.cc:243] Waited 8.33 ms on lock 0x7b18003c1540. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:04.635651 (+349439us) raft_consensus.cc:1685] finished
0709 13:06:04.635691 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.635941 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6654,"prepare.queue_time_us":143820,"prepare.run_cpu_time_us":6277,"prepare.run_wall_time_us":35771,"raft.queue_time_us":8107,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":34839,"spinlock_wait_cycles":159338624,"thread_start_us":260100,"threads_started":1,"wal-append.queue_time_us":453521}
I20260709 13:06:04.611591 18370 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.859604 14019 raft_consensus.cc:2410] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:06:04.634960 18066 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:04.762287 18490 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:04.872453 14021 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:04.807040 18376 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:04.766371 18607 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.817005 18559 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:04.896881 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:04.901003 14551 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:04.901413 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:04.759930 18468 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:04.875451 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 360065us
I20260709 13:06:04.877018 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:04.851266 18409 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:04.880745 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 144704us
I20260709 13:06:04.888844 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:04.909890 14021 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:04.911120 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:04.912565 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:04.862068 18276 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:04.889598 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:04.756013 18329 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.926759 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:04.861678 18499 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:06:04.904227 18607 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:04.933707 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:04.934455 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:04.931885 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:04.927310 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:04.848917 18618 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:06:04.876843 18545 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:04.938398 14558 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:04.938110 18253 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:04.901521 18619 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.951321 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:04.878646 18511 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.956872 14552 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
W20260709 13:06:04.943307 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 146930us
I20260709 13:06:04.944237 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:04.882439 18573 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:04.895823 18066 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.941748 18505 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:04.963562 13947 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:04.962656 14558 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:04.876760 18598 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:04.938387 18559 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:04.968500 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2036) took 2271 ms. Trace:
W20260709 13:06:04.965653 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.946529 18378 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:04.968945 14018 rpcz_store.cc:276] 0709 13:06:02.697359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.103793 (+406434us) service_pool.cc:224] Handling call
0709 13:06:04.237039 (+1133246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.261051 (+ 24012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.968252 (+707201us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54000ab750. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:04.968435 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2580372224}
I20260709 13:06:04.970932 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:04.971306 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 96088.8us
I20260709 13:06:04.974733 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:04.974304 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 44423.7us
I20260709 13:06:04.976094 13947 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:06:04.974463 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:04.979760 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:04.980098 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:04.980952 14019 raft_consensus.cc:2393] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:04.981029 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:04.902320 18531 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:04.981746 14018 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:04.983268 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:04.966710 18505 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:04.942498 18603 consensus_queue.cc:1059] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 13:06:04.978235 18605 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 13:06:04.998664 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:04.999869 18631 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:04.972398 18616 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:04.975694 18520 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:05.008371 18505 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.009635 18505 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:04.959048 18619 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.000283 18253 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:06:04.972811 18559 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:05.018927 18613 log.cc:927] Time spent T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:06:05.051405 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:05.032307 18617 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 13:06:05.036330 18611 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:06:05.014458 18378 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:05.035914 18042 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.019629 18508 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:05.023160 18612 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:06:05.063954 13987 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:05.024122 18499 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:05.077394 18545 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.095126 18559 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:05.096356 18511 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:05.039685 18620 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:06:05.097730 18329 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:05.069753 13870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2792) took 1864 ms (client timeout 1587 ms). Trace:
W20260709 13:06:05.103513 13870 rpcz_store.cc:269] 0709 13:06:03.205165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.205313 (+   148us) service_pool.cc:224] Handling call
0709 13:06:05.069525 (+1864212us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54000e9a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:05.069664 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588474240}
I20260709 13:06:05.070118 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.105541 14019 raft_consensus.cc:2393] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:05.071841 18593 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.123s	user 0.008s	sys 0.011s
I20260709 13:06:05.075670 18157 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.108539 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.109079 18157 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:05.109972 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2789) took 1927 ms. Trace:
I20260709 13:06:05.110152 18559 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:05.110572 18559 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.110404 13871 rpcz_store.cc:276] 0709 13:06:03.203941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.204145 (+   204us) service_pool.cc:224] Handling call
0709 13:06:03.224198 (+ 20053us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.604373 (+1380175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.109708 (+505335us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54003ab6d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:05.109903 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3072672128}
I20260709 13:06:05.111033 18559 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:05.112913 18621 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:06:04.973227 18409 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.115957 18619 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:05.031965 18066 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:05.033541 18598 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:05.117440 18598 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:05.117800 18598 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:05.118726 18598 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.120184 18598 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:05.082165 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:05.131067 14022 raft_consensus.cc:2368] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:06:05.132241 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.084784 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.087821 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.134857 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.078258 18370 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:05.091275 18042 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.098938 14556 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
W20260709 13:06:05.069023 18548 consensus_meta.cc:287] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.983s	user 0.013s	sys 0.000s
I20260709 13:06:05.101658 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:05.107359 18545 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:05.100530 18508 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.089386 18607 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:05.147574 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46043.3us
I20260709 13:06:05.102679 18161 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:05.148612 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.149324 14022 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 7: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:05.157809 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2693) took 1747 ms. Trace:
I20260709 13:06:05.158622 14553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 789) took 1185 ms. Trace:
I20260709 13:06:05.159049 14553 rpcz_store.cc:276] 0709 13:06:03.973361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.973536 (+   175us) service_pool.cc:224] Handling call
0709 13:06:03.973902 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.974464 (+   562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.974496 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.007802 (+ 33306us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.580201 (+572399us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:04.580250 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.580320 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.158381 (+578061us) raft_consensus.cc:1685] finished
0709 13:06:05.158405 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.158556 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1329,"raft.run_cpu_time_us":3824,"raft.run_wall_time_us":30011,"thread_start_us":473197,"threads_started":1,"wal-append.queue_time_us":679472}
I20260709 13:06:05.160171 18042 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:05.158249 13946 rpcz_store.cc:276] 0709 13:06:03.410656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.530026 (+119370us) service_pool.cc:224] Handling call
0709 13:06:03.533493 (+  3467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.561795 (+ 28302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.561850 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.282010 (+720160us) spinlock_profiling.cc:243] Waited 68.2 ms on lock 0x7b2c000e5d98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:06:04.282515 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.587790 (+305275us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:04.587861 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.587976 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.156069 (+568093us) raft_consensus.cc:1685] finished
0709 13:06:05.156092 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.157736 (+  1644us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156923008,"thread_start_us":304685,"threads_started":1,"wal-append.queue_time_us":480888}
I20260709 13:06:05.163187 18508 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:05.156746 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:05.152724 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.171475 14017 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:05.171242 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:05.171712 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.172354 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.172948 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.173341 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.173462 14023 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:05.172201 14019 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
W20260709 13:06:05.175882 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 137845us
I20260709 13:06:05.177896 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:05.178774 14017 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:05.179399 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.145988 18371 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.153740 18378 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.181025 18253 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:05.147486 18329 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:05.177156 18370 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:06:05.181586 18617 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:06:05.184284 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 523) took 1960 ms. Trace:
W20260709 13:06:05.191418 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.208568 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:05.198207 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.190374 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:05.207365 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.224069 14020 raft_consensus.cc:2410] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:06:05.174691 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.204345 18159 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:05.213146 14015 rpcz_store.cc:276] 0709 13:06:03.224302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.367672 (+143370us) service_pool.cc:224] Handling call
0709 13:06:03.368082 (+   410us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:03.420957 (+ 52875us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.420994 (+    37us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:03.429658 (+  8664us) log.cc:844] Serialized 78 byte log entry
0709 13:06:03.651395 (+221737us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:03.697240 (+ 45845us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.703219 (+  5979us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.746122 (+ 42903us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.746335 (+   213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.746415 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.060939 (+314524us) spinlock_profiling.cc:243] Waited 9.67 ms on lock 0x7b180012e920. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:05.071190 (+1010251us) spinlock_profiling.cc:243] Waited 9.54 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:05.184027 (+112837us) raft_consensus.cc:1685] finished
0709 13:06:05.184052 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.184214 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44747,"prepare.queue_time_us":46756,"raft.queue_time_us":35606,"raft.run_cpu_time_us":607,"raft.run_wall_time_us":75806,"spinlock_wait_cycles":58096000,"thread_start_us":192382,"threads_started":1,"wal-append.queue_time_us":422244}
W20260709 13:06:05.185338 18597 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.891s	user 0.000s	sys 0.004s
I20260709 13:06:05.230639 14020 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:06:05.240504 14020 pending_rounds.cc:85] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 6
I20260709 13:06:05.241012 14020 pending_rounds.cc:107] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:06:05.241426 14020 raft_consensus.cc:2889] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:05.181650 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:05.189041 18448 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.234510 14021 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:05.235631 14015 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:06:05.236765 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:05.245975 14019 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:05.226073 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 237422us
I20260709 13:06:05.248209 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:05.219224 18253 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:05.225477 18508 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:05.225726 18370 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:06:05.261169 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.261294 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52872us
I20260709 13:06:05.210695 18329 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.272331 18329 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:05.202908 18371 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.277130 18371 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:05.278182 18377 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:05.257565 18448 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.280439 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:05.280722 18329 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:05.281636 18619 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:05.282044 18448 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:05.282042 18619 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.283281 18253 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.283039 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:05.285039 18499 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:05.286193 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:05.287127 18499 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:05.286773 18370 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.288319 13989 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:05.289885 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:05.284013 18619 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:05.292480 18329 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:05.293874 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:05.282351 18508 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:05.294662 18578 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.656s	user 0.009s	sys 0.009s
I20260709 13:06:05.296455 18508 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:05.298434 18329 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:05.299772 18559 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.300181 13989 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:05.301258 14024 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:05.282935 18157 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:06:05.262557 18630 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:06:05.303771 18329 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:05.306835 18468 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:05.308446 18499 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:05.282660 13840 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:05.332582 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.334108 14023 raft_consensus.cc:2410] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:06:05.330166 18468 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:05.342738 14016 raft_consensus.cc:2410] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:06:05.330753 18559 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:05.346235 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 345086us
I20260709 13:06:05.350132 18619 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.352520 18619 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:05.352843 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:05.354995 18492 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.365904 18492 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.366261 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.368005 14021 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:06:05.359740 18601 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:05.361907 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.363533 13773 catalog_manager.cc:5697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:05.368743 18492 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:05.357399 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:05.375056 13866 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
W20260709 13:06:05.370787 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 106949us
I20260709 13:06:05.372402 14021 pending_rounds.cc:85] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 4
I20260709 13:06:05.372465 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:05.377004 14021 pending_rounds.cc:107] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:06:05.377404 14021 raft_consensus.cc:2889] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:05.377171 13838 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:05.378916 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.379528 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:05.380851 14021 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:05.381438 13875 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:05.336512 14023 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:05.384770 14017 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:05.385337 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:05.385996 13870 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:05.384939 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:05.387069 14017 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:05.388420 14017 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:05.388753 13839 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:05.389475 14017 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:05.389902 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:05.391441 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:05.391855 18376 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:05.392251 18553 raft_consensus.cc:2764] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:06:05.429878 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:05.392349 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:05.392540 18159 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:05.429972 18376 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.431538 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:05.393956 18492 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:05.394799 18157 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:05.433360 18376 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:05.395648 18622 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:05.435945 18622 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:05.396199 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 419864us
I20260709 13:06:05.440497 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:05.442859 18548 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:05.444180 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:05.446175 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:05.397080 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 422708us
I20260709 13:06:05.397866 18632 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:05.398360 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:05.398921 18329 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.400175 18042 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:05.448904 18042 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:05.448719 18329 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.450022 18042 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.397903 14022 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:06:05.373291 18631 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:05.403249 13946 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:05.403720 14024 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:06:05.403785 14019 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:05.454818 14024 pending_rounds.cc:85] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 6
I20260709 13:06:05.455096 14024 pending_rounds.cc:107] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:06:05.455349 14024 raft_consensus.cc:2889] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:05.455564 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.456780 14553 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:05.457471 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:05.458343 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:05.460947 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.404937 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.405371 13942 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:05.463891 14021 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:05.406471 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.465654 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.409250 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:05.468915 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.410584 13941 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:05.467383 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.413072 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:05.472347 13870 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:06:05.413579 18276 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:05.437182 18622 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.415292 18633 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:05.416462 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.485666 13948 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:05.417691 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.394837 18623 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:06:05.384037 18601 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:05.423286 18606 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.816s	user 0.004s	sys 0.000s
I20260709 13:06:05.391791 18643 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.424052 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:05.426302 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:05.427189 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:05.430081 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.448125 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:05.502437 14016 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:06:05.439860 13914 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:05.453876 18329 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:05.454315 14020 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:06:05.471338 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.412024 13838 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:05.462073 14555 raft_consensus.cc:2376] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 13. Re-sending same reply.
I20260709 13:06:05.414017 13945 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:05.482645 14561 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:05.481037 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:05.484467 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.487069 13947 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:06:05.402009 18616 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:05.492069 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:05.482972 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:05.514482 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 74799.3us
I20260709 13:06:05.514354 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:05.515939 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 67927.2us
I20260709 13:06:05.517095 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:05.509464 14024 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:06:05.522593 13914 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:05.525326 13914 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:05.526319 13875 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:05.477885 18276 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:05.538275 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:05.539692 13914 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:05.539829 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:05.529295 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.541080 14024 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 13: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:06:05.533476 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.544490 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 132594us
I20260709 13:06:05.553506 14024 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:05.556093 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:05.569105 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:05.571410 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:05.525796 18531 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:05.524539 18220 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:05.537127 18631 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:05.554636 18409 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:05.537138 18520 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:05.549883 18625 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:05.549602 18400 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:05.559701 18623 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:05.551795 18405 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:05.623909 18405 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.624404 18405 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:05.602644 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:05.501341 18633 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.629129 18633 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:05.572467 18525 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:05.576207 18624 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:05.531735 18616 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.632369 18616 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:05.612071 18629 consensus_queue.cc:1059] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:06:05.635407 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:05.615438 18634 raft_consensus.cc:3055] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:05.636408 18634 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:05.637495 18634 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.638144 14561 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:05.639197 18634 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:05.553846 18573 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:06:05.513994 18643 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.547724 18648 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 13:06:05.611006 18066 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:05.601280 18504 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:06:05.660328 18520 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:05.665756 18616 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:05.664278 18525 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:05.662424 18623 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:05.684262 18652 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:06:05.686677 18643 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:05.688850 18417 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:05.690165 18632 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:05.690642 18632 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:05.691638 18632 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.693161 18632 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:05.675664 18520 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:05.680346 18616 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.712342 18520 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:05.714684 18625 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:05.718086 18620 raft_consensus.cc:3055] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:06:05.762295 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 192971us
I20260709 13:06:05.764668 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:05.762737 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 194141us
I20260709 13:06:05.767765 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:05.614549 13950 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:05.770485 13873 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:05.771884 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2200) took 1007 ms. Trace:
I20260709 13:06:05.772433 13950 rpcz_store.cc:276] 0709 13:06:04.764303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.764427 (+   124us) service_pool.cc:224] Handling call
0709 13:06:05.613820 (+849393us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b5800202e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:06:05.614162 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.771203 (+157041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.771597 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5578112}
I20260709 13:06:05.779093 13943 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:05.658028 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 55517.4us
W20260709 13:06:05.788067 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 143966us
I20260709 13:06:05.789697 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:05.790706 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:05.791869 13870 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:05.793543 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.794891 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:05.796166 13869 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:06:05.800127 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:05.801134 14022 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:05.610617 18276 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.806478 18276 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:05.807133 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:05.808380 14558 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:05.806756 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:05.810583 18041 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.810806 14561 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:06:05.820655 18041 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.668298 18066 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:05.823884 18505 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:05.826169 18505 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.831305 18505 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:05.715168 18616 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:05.719816 18623 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.827277 13773 catalog_manager.cc:5697] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 4, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 4 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:05.830503 18544 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.719591 18620 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:05.842173 13873 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:05.844380 18620 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:05.845130 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 81118.6us
I20260709 13:06:05.847335 18417 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:06:05.868418 18544 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.871819 13838 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:05.872056 18525 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.875478 18372 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:05.876571 18372 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:05.878543 18648 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:05.879292 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.880412 18041 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:05.850435 18161 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:06:05.883998 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.845932 18371 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:05.885206 13942 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:05.886566 18544 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:05.887804 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.888886 13950 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:05.891487 18400 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:06:05.895016 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 127583us
I20260709 13:06:05.854274 18620 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:05.898478 13869 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:05.889936 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.909628 14022 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:05.899715 18615 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:05.917753 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:05.919070 13942 raft_consensus.cc:2376] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 14. Re-sending same reply.
I20260709 13:06:05.928687 18559 raft_consensus.cc:3055] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:05.929282 18559 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:05.904735 18371 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:05.930795 18371 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.947523 18629 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:05.949291 18629 raft_consensus.cc:3055] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:05.949721 18629 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:05.932958 18559 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:05.951299 18559 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:05.956305 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 153053us
I20260709 13:06:05.825695 18066 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.961194 13870 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:05.973115 18041 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.974179 18041 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:05.978737 18066 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:05.984258 18601 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:05.991438 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.993342 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.995208 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:05.996179 13941 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:06:05.997251 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:05.999240 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.000653 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.002192 13949 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:06.003767 13941 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:05.936393 18417 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:06:06.018393 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:06.019788 13869 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:06:06.014024 13948 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:06:06.020826 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.033079 13943 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:06:06.031154 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:06.047813 18520 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:06.024570 18371 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:06.050521 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:06.023203 18615 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:06.063030 18161 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:06.064275 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2075) took 1265 ms. Trace:
I20260709 13:06:06.065114 14015 rpcz_store.cc:276] 0709 13:06:04.765330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.244796 (+479466us) service_pool.cc:224] Handling call
0709 13:06:05.245225 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.249226 (+  4001us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:05.484882 (+235656us) log.cc:844] Serialized 14 byte log entry
0709 13:06:05.912209 (+427327us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:05.942701 (+ 30492us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:05.980330 (+ 37629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.021354 (+ 41024us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.031068 (+  9714us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20996,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":11348,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":221,"thread_start_us":550275,"threads_started":2,"wal-append.queue_time_us":425868}
I20260709 13:06:06.070888 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:06.072592 13944 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:06.071749 18492 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:06.064389 18329 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:06.077159 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.080425 18615 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:06.080643 18157 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:06:06.102993 18665 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
W20260709 13:06:06.094142 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.096138 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:06.097169 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.099148 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.102249 18520 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:06.085719 18405 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:06.104513 13773 catalog_manager.cc:5697] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 3 to 6, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 6 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:06.105517 18409 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:06:06.090950 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 132277us
I20260709 13:06:06.111752 18157 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:06.120453 14561 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:06.123090 18488 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:06.124759 18544 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:06.139235 13914 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:06.127348 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:06.128434 14558 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:06:06.129206 13773 catalog_manager.cc:5697] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:06.116106 13941 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:06.148051 13944 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:06:06.160602 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:06.147332 18409 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:06.164012 18409 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:06.164526 18409 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:06.171277 18157 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:06.174188 18157 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:06.178401 13945 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:06:06.179133 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.187384 18405 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:06.188369 13944 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:06.169921 18161 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.225164 18161 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:06.167260 18041 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:06.198808 13871 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:06.153901 18629 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:06.190367 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:06.223901 14558 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:06:06.223285 18505 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:06.231097 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:06.250340 18620 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.250800 18409 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:06.251775 13773 catalog_manager.cc:5697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 7 to 8, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 8 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:06.254226 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.273808 18329 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:06.229215 18161 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:06.332126 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 352912us
I20260709 13:06:06.345924 18329 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:06.368258 18370 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.369321 18370 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:06.371668 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.372174 18370 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:06.372700 18505 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:06.373203 18620 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:06.373730 18405 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.374460 18405 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:06.374848 18409 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:06.376006 18620 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:06.376277 13773 catalog_manager.cc:5697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 2 to 7, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 7 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:06.380482 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:06.373431 18329 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:06.396199 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 428518us
I20260709 13:06:06.389603 13773 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 5 to 7, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 7 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:06.376350 18672 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.313s	user 0.001s	sys 0.003s
I20260709 13:06:06.377173 18629 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:06.399155 13772 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57626 (request call id 247) took 28214 ms (client timeout 15000 ms). Trace:
I20260709 13:06:06.376819 18405 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:06.437299 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:06.442432 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 252194us
I20260709 13:06:06.443564 18409 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:06.444363 18632 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:06.444473 18409 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:06.456151 13838 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:06.377722 18377 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:06:06.443799 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:06.470505 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.477918 18112 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.478797 18112 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:06.469988 18405 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:06.451327 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.497582 14561 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:06.460922 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:06.482659 18041 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:06.494087 13866 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:06:06.502472 13914 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:06.514688 14524 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:06.519788 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 102896us
I20260709 13:06:06.496414 18371 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.530093 14558 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:06:06.540505 18669 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.435s	user 0.001s	sys 0.001s
I20260709 13:06:06.542721 18660 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:06.543051 18371 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:06.545696 18371 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:06.546521 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.524252 18159 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:06:06.543463 18660 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:06.525864 18488 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:06:06.563490 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:06.567047 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 109117us
I20260709 13:06:06.561928 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:06.570813 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:06.571321 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.572077 13941 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:06.576709 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:06.578435 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.579255 13950 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 7.
I20260709 13:06:06.563165 18488 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.582784 18488 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:06.570524 18329 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:06.570941 18112 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:06.579972 13838 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:06.575491 18371 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:06.601951 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:06.600730 18371 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.587682 18041 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:06.417963 13772 rpcz_store.cc:269] 0709 13:05:38.055469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.098659 (+ 43190us) service_pool.cc:224] Handling call
0709 13:06:06.269624 (+28170965us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:55.320225 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:55.360280 (+ 40055us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:55.360352 (+    72us) write_op.cc:435] Acquired schema lock
0709 13:05:55.360394 (+    42us) tablet.cc:662] Decoding operations
0709 13:05:55.556074 (+195680us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:55.556217 (+   143us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:55.556255 (+    38us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:05:55.557925 (+  1670us) tablet.cc:701] Row locks acquired
0709 13:05:55.557986 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:05:55.578753 (+ 20767us) write_op.cc:270] Start()
0709 13:05:55.598344 (+ 19591us) write_op.cc:276] Timestamp: P: 1783602355573014 usec, L: 0
0709 13:05:55.598398 (+    54us) op_driver.cc:348] REPLICATION: starting
0709 13:05:55.599478 (+  1080us) log.cc:844] Serialized 6745 byte log entry
0709 13:05:56.149320 (+549842us) op_driver.cc:464] REPLICATION: finished
0709 13:05:56.548740 (+399420us) write_op.cc:301] APPLY: starting
0709 13:05:56.548885 (+   145us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:56.796875 (+247990us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:56.797021 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:57.325274 (+528253us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:57.326019 (+   745us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:05:57.378933 (+ 52914us) write_op.cc:312] APPLY: finished
0709 13:05:57.574759 (+195826us) log.cc:844] Serialized 156 byte log entry
0709 13:05:57.928339 (+353580us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:57.943745 (+ 15406us) write_op.cc:454] Released schema lock
0709 13:05:57.943832 (+    87us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:57.944374 (+   542us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:57.944409 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":266812,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":399033,"apply.run_cpu_time_us":91859,"apply.run_wall_time_us":1395623,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75085,"num_ops":17,"prepare.queue_time_us":145933,"prepare.run_cpu_time_us":30244,"prepare.run_wall_time_us":470311,"raft.queue_time_us":219073,"raft.run_cpu_time_us":44653,"raft.run_wall_time_us":385707,"replication_time_us":478845,"thread_start_us":935582,"threads_started":5,"wal-append.queue_time_us":603110,"wal-append.run_cpu_time_us":28974,"wal-append.run_wall_time_us":2655827}]]}
I20260709 13:06:06.618027 13941 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:06.604360 18559 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:06:06.607256 18616 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.689s	user 0.014s	sys 0.004s
W20260709 13:06:06.620322 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 51295.1us
W20260709 13:06:06.621716 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 181610us
I20260709 13:06:06.616758 14522 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:06.610236 18371 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:06.608459 18409 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:06.638499 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.622740 18559 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.659618 18112 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:06.660421 18112 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:06.660548 18680 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:06:06.660966 18112 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:06.663514 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 103790us
W20260709 13:06:06.680276 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 140386us
I20260709 13:06:06.663941 18559 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:06.684952 13942 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:06:06.685284 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:06.687220 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 98057.8us
I20260709 13:06:06.630892 18616 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:06.669023 18620 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:06.712121 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:06.700223 18682 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.092s	user 0.002s	sys 0.000s
W20260709 13:06:06.709158 18676 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.205s	user 0.004s	sys 0.000s
I20260709 13:06:06.720770 18159 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:06.666733 18660 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:06.734284 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:06.735703 18520 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:06:06.738473 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 115159us
I20260709 13:06:06.739074 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:06.739617 13838 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:06.740522 18159 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:06.740998 18159 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.741621 18159 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:06.740754 18066 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:06.740751 13838 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:06.701474 18632 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:06:06.752889 18066 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:06.761961 18681 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:06:06.755592 18520 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:06.773945 18041 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:06.784686 18377 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:06.785238 18405 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:06:06.787743 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.770767 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:06.800559 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 134868us
I20260709 13:06:06.771993 18066 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:06.791208 18041 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.806201 18544 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:06:06.814594 18041 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:06.819581 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.820393 13942 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:06:06.823171 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.824028 13948 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:06.825711 13915 leader_election.cc:400] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 12.
I20260709 13:06:06.827108 18492 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:06.835234 18616 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:06.839608 18678 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.223s	user 0.004s	sys 0.000s
I20260709 13:06:06.859409 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:06.860764 18683 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:06:06.861819 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.876291 13915 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:06.888689 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:06.892381 18616 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:06.892896 18616 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:06.893312 18616 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:06.894196 18688 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 13:06:06.889258 18687 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:06:06.915324 18329 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:06.916188 18329 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:06.923377 18329 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:06.931702 18525 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:06.941365 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 52821.9us
I20260709 13:06:06.942886 18505 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.943578 18505 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:06.950198 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 120699us
I20260709 13:06:06.950440 18559 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:06.921288 18157 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.746s	user 0.009s	sys 0.010s
I20260709 13:06:06.951795 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:06.954795 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.958449 13989 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:06.939399 18112 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:06.977311 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 236141us
W20260709 13:06:06.982606 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 287813us
I20260709 13:06:06.984458 13941 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:06.985515 13950 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:06.987179 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:06.987870 13942 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:06.969148 18632 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:06.996241 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:06.997750 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 46213.2us
I20260709 13:06:06.967417 18041 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.001062 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:07.003260 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:07.015336 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:07.023674 14556 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:07.020789 18559 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.031538 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:07.032820 13947 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:07.036160 18559 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:07.036697 18559 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.037070 18559 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:07.045863 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:07.034344 18505 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:06.992084 18488 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.049304 18632 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:07.143322 18159 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.149008 18159 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:07.151659 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 356672us
W20260709 13:06:07.164613 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 120867us
I20260709 13:06:07.150709 18159 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:06:07.166708 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 160897us
I20260709 13:06:07.172573 13838 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:07.166631 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 173821us
W20260709 13:06:07.177570 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 195077us
I20260709 13:06:07.140501 18066 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.195333 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:07.189982 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 200863us
I20260709 13:06:07.156219 18371 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.178174 18112 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:07.226574 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 465848us
I20260709 13:06:07.231721 18371 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.221933 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:07.240666 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:07.243079 18041 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.246148 18066 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.187480 18693 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:07.246757 18520 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:07.192203 18632 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:07.296676 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 305441us
I20260709 13:06:07.303341 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:07.296275 18669 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 13:06:07.311679 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.295393 18112 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:07.254225 18159 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:07.315161 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 142713us
I20260709 13:06:07.303992 18660 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:07.326324 18377 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:07.327241 13838 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:07.319080 18544 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:07.333473 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 145936us
I20260709 13:06:07.318879 18700 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:07.338217 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.310196 18632 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.327284 18693 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.318570 18520 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:07.322311 18112 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:07.360575 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:07.354477 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 51252.1us
W20260709 13:06:07.384554 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 57423.6us
I20260709 13:06:07.385811 13838 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:07.388335 18708 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:07.388813 18708 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.389144 18708 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:07.390787 18157 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:07.391232 18157 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.391577 18157 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:07.395254 18711 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.395970 18711 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:07.400687 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 160126us
I20260709 13:06:07.368228 18488 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.381731 18112 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:07.366753 18544 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.415902 18701 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.416658 18701 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.417382 18544 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.377360 18370 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:07.418435 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:07.426385 18370 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.427398 18492 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:07.427860 18492 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:07.428267 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:07.429129 18370 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:07.428843 18492 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:07.432266 18689 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
W20260709 13:06:07.415093 18616 consensus_meta.cc:287] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.521s	user 0.012s	sys 0.002s
I20260709 13:06:07.452126 18525 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:07.455760 18693 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.456408 18520 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:07.457362 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:07.458405 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 60965.3us
I20260709 13:06:07.464035 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:07.473523 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 86856.7us
I20260709 13:06:07.468029 13947 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
W20260709 13:06:07.476902 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 103344us
I20260709 13:06:07.479889 14556 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:07.448771 18492 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:07.481905 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:07.502853 18709 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.477865 18706 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.472740 18409 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:07.504253 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:07.513736 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:07.469753 18616 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.491716 18505 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:07.475317 18559 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.503955 18660 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:07.537232 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:07.572820 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.542197 18710 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.541811 18720 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:07.603734 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 63494.2us
I20260709 13:06:07.503750 18709 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.562665 18629 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.624825 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:07.627650 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:07.602715 18620 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:07.630988 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:07.633350 13948 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:07.631294 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 117670us
I20260709 13:06:07.634757 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:07.632262 13947 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:06:07.637600 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:07.638448 13941 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:07.644256 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:07.645095 13947 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:07.645896 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:07.621300 18159 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.618548 18660 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.612232 18701 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:07.563730 18505 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.622376 18409 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:07.539602 18706 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:07.659880 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 209838us
I20260709 13:06:07.660588 18559 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.660904 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:07.661357 18559 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:07.641636 18632 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.650836 18701 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.673951 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:07.675053 13946 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
W20260709 13:06:07.676309 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 139260us
I20260709 13:06:07.650713 18660 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.662873 18711 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:07.677356 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:07.651583 18629 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.683427 18713 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.683933 18698 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:07.672173 18615 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:07.684350 18730 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:06:07.686190 18716 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:07.686523 18722 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:06:07.644762 18720 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.689131 18709 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.637539 18710 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:07.689894 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 204918us
I20260709 13:06:07.692025 18706 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:07.696082 18724 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:06:07.697314 18157 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.742594 13941 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:07.743405 13948 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:06:07.702523 18729 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:07.755080 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 85596.7us
I20260709 13:06:07.672435 18701 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.767845 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:07.668219 18409 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.734030 18716 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.803391 13947 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:07.805843 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:07.765056 18559 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:07.806842 13866 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 7.
I20260709 13:06:07.767093 18370 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:07.814389 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46675.6us
I20260709 13:06:07.770131 18722 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:07.786975 18706 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:07.815421 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 154663us
I20260709 13:06:07.786526 18616 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.745872 18615 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.833693 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:07.813143 18329 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:07.826280 13840 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:07.837888 13871 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:06:07.838438 18377 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:07.823208 18370 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.840411 18660 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.838668 18329 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.841763 18370 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:07.801105 18716 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:07.844378 18631 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:07.845016 18631 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.813745 18559 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.846511 18559 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:07.846581 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:07.847371 13868 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:07.828398 18720 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.829074 18620 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.831043 18629 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.831472 18544 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:07.831668 18710 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.832621 18505 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:07.764653 18693 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:06:07.759104 18713 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.834261 18722 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.893123 18713 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.835429 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:07.895941 18112 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.857761 18620 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.898154 18741 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:07.898947 18741 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.899647 18620 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.858162 18740 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:06:07.900087 18409 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:07.902045 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:07.860749 18631 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:07.862705 18731 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:06:07.904240 18708 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.864674 18157 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.906821 18713 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:07.865000 18615 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:07.914417 18329 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.914759 18157 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:07.916186 18695 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:07.916744 18157 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:07.829784 18492 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:06:07.877912 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.918367 18492 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:07.882560 18544 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.919420 18492 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.920462 18492 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:07.893779 18728 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:07.835068 18701 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.922418 18728 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.922915 18409 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:07.893659 18722 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:07.903786 13987 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:07.921847 18544 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.924958 13987 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:07.922981 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 87658.9us
I20260709 13:06:07.925954 18728 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.926165 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:07.927232 18728 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:07.927873 13915 leader_election.cc:400] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:06:07.928222 13915 leader_election.cc:403] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:06:07.928547 18708 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.786579 18729 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.928836 18660 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:06:07.931305 18729 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.931974 18701 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:07.931444 18713 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:07.933352 18713 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:07.935292 18713 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.876720 13868 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:06:07.953965 18708 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:07.955062 18708 consensus_queue.cc:1243] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:07.955384 18112 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:07.954399 18741 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:07.963014 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:07.961934 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:07.964030 18729 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:07.965531 18729 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:07.966212 18729 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:07.967664 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:07.964836 18741 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:07.965900 18660 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:07.969071 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:07.969981 18701 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:07.971683 18329 consensus_queue.cc:1059] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:07.969959 18716 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.972702 18701 raft_consensus.cc:3055] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:07.973086 18701 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:07.973695 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:07.974324 18701 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:07.975299 18728 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:07.956302 18559 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.976584 18701 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:07.977988 18559 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.978338 18708 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:07.978989 18708 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:07.979568 18708 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:07.982790 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:07.983826 18722 consensus_queue.cc:1059] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:07.985281 18157 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:07.985675 18157 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:07.983068 18716 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:07.982693 18112 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:07.986644 18157 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:07.997742 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:08.002278 18693 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:08.003120 18693 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:08.025728 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:08.035476 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:08.037441 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:08.056614 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:08.041638 14556 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:06:08.064407 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:08.062402 18559 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:06:08.064143 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 50979.8us
W20260709 13:06:08.124454 18745 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:06:08.126801 18722 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.038295 18329 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:08.124109 18742 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.032702 18693 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:08.145776 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 268322us
I20260709 13:06:08.144673 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:08.155356 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.156381 13948 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:06:08.155215 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 165298us
I20260709 13:06:08.149081 13868 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:06:08.149338 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.151871 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:08.149941 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.131907 18112 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:08.165653 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:08.167865 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:08.142477 18329 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.142730 18559 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:08.179296 13868 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:08.169365 14015 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:08.180393 13838 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:08.179096 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:08.189109 14022 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:08.147912 18722 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:08.193009 18729 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:08.195078 18722 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:08.195608 18735 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:08.197399 13871 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:06:08.198957 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.199831 13872 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:08.201180 13869 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:08.163550 18695 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:08.202270 18695 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.202680 18695 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:08.202819 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.203815 13873 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:08.205180 13874 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:08.186340 18112 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:08.193069 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:08.206749 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.207213 18741 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:08.208148 18112 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:08.183185 18329 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:08.208868 18157 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:08.209945 18409 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:08.210146 18729 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:08.210978 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.211351 13838 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:08.212212 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:08.212352 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:08.212777 13941 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:08.213737 18744 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:08.211311 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:08.214273 18492 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:08.214905 18740 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:08.215641 18740 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:08.216730 18740 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:08.217455 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:08.214368 18744 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:08.219079 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:08.219758 14553 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:08.219396 18744 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:08.208187 14021 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:08.214962 18492 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:08.222846 18492 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:08.223667 13946 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:08.223906 18708 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:08.224793 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:08.225152 13870 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:08.225379 13946 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:08.214798 18559 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:08.218556 14554 raft_consensus.cc:2410] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:06:08.253451 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:08.245692 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.259897 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:08.260995 13948 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:08.264616 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.259804 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.270895 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.271704 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.270908 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:08.272655 13948 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:08.278503 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:08.282334 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:08.283766 13871 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:08.284608 13874 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:08.284668 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:08.285398 14022 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:06:08.285573 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:08.295377 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:08.296136 14561 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:08.290766 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.305822 13946 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:08.293035 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:08.304832 13872 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:08.310417 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.311225 13948 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:06:08.309140 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:08.314807 14524 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:08.304603 13988 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:08.268183 18716 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.317590 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.318917 13946 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:06:08.323338 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:08.319159 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:08.325251 13871 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:08.328341 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.330499 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:08.337344 13941 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 8.
I20260709 13:06:08.338573 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.341503 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.342545 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.264664 18720 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:08.281230 18728 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:08.168740 18742 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:08.379159 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.386428 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:08.386514 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.378871 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:08.348707 13949 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 9: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:08.387260 13870 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:08.385000 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:08.387672 14022 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:08.350314 13874 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:08.384195 13941 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:06:08.383893 13872 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:08.290295 18708 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:08.320570 18693 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:08.378597 18544 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:08.431648 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:08.450461 13870 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:08.450831 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46983.5us
I20260709 13:06:08.394841 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.317772 18525 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:08.407747 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.319469 18741 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.380481 18488 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:08.412767 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.320911 18748 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:08.399458 18734 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.432570 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:08.326854 18713 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:08.413648 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.452538 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:08.431377 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.451788 13871 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:08.460371 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.463434 14017 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:08.477895 13949 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:08.466641 18741 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:08.391139 18720 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:08.466835 14019 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:08.466576 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:08.467408 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:08.501902 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.503695 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:08.460469 13948 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:08.466349 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.469830 13944 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:08.470142 18544 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:08.470534 13838 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:08.516661 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46225.9us
I20260709 13:06:08.477427 13946 raft_consensus.cc:2410] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:06:08.486616 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:08.465543 14024 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:08.331772 18716 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:08.518754 14020 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
W20260709 13:06:08.512012 18756 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:06:08.475196 18748 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.513083 13867 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:06:08.513230 18734 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:08.513702 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:08.512671 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.538180 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:08.512467 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:08.515327 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 82886.4us
I20260709 13:06:08.539176 13987 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:08.518431 14016 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:08.430966 18693 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:08.518584 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:08.517412 13838 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:08.521727 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.544766 13948 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:08.520846 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:08.526214 13870 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:08.546401 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.475004 18488 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.512557 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.547669 13949 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:08.484750 18713 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.535126 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:08.553309 14021 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 12: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:08.534487 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:08.514019 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:08.431407 18728 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:08.554517 14019 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:08.556564 13869 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:08.522974 18492 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:08.558823 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:08.563494 13944 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:08.564443 14019 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:08.560137 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:08.565442 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48123.2us
I20260709 13:06:08.565991 13872 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:08.558372 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:08.534868 18751 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:08.566531 14022 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:08.685177 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.749022 13870 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:08.692039 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.549752 18693 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:08.688763 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:08.549944 18488 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:08.539824 18748 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:08.553118 18631 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:08.691697 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.696703 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:08.559746 18713 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:08.704648 14017 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
W20260709 13:06:08.708388 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.597757 18112 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:08.597477 18559 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:08.762701 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.763021 18559 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.763453 18559 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:08.764104 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:08.765303 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.771306 18112 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:08.772647 18728 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:08.772375 18112 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:08.775142 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.728600 14024 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:06:08.728950 18762 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:06:08.730701 14020 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 13:06:08.782682 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 501928us
I20260709 13:06:08.703321 18741 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:08.571313 13949 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:08.522576 18712 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:08.605170 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:08.750401 13873 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:08.756345 14022 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:08.740592 18544 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:08.762247 14019 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:06:08.765863 13870 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:06:08.781275 13872 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:08.758671 13867 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 7.
I20260709 13:06:08.803328 13873 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:08.754783 18751 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:08.794709 18695 consensus_meta.cc:287] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.580s	user 0.016s	sys 0.005s
I20260709 13:06:08.805132 13948 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:08.816401 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:08.806149 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:08.847404 13950 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.010s	sys 0.034s
I20260709 13:06:08.791635 18701 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.854262 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:08.850231 13950 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.011s	sys 0.034s
I20260709 13:06:08.793934 18712 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.862555 13948 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:06:08.868978 18329 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.660s	user 0.011s	sys 0.000s
I20260709 13:06:08.874504 13869 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:08.875773 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.815819 18492 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:08.822077 18751 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:08.877588 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.878904 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:08.871714 18762 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:06:08.869537 18329 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:08.882344 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.881660 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.884361 13873 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:08.882886 13867 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:08.873035 18712 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:08.861058 18742 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:08.887770 13870 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:08.894830 18559 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:08.896203 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:08.899237 13872 raft_consensus.cc:2376] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12. Re-sending same reply.
I20260709 13:06:08.899559 18716 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:08.900548 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.847554 18695 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:08.910266 13866 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.649s	user 0.013s	sys 0.000s
I20260709 13:06:08.910846 13866 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:08.911996 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.850291 18728 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:06:08.871076 18701 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:08.917629 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:08.919528 13869 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:06:08.922747 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 383630us
I20260709 13:06:08.934566 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:08.949580 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:08.952230 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:08.963105 18695 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:08.965458 18741 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:08.979105 14023 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:08.987187 18559 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:08.920984 18720 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:08.950402 18505 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:06:08.965176 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:08.968855 18157 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.750s	user 0.010s	sys 0.003s
W20260709 13:06:09.006834 18757 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.037s	sys 0.007s
I20260709 13:06:09.007117 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2236) took 1384 ms. Trace:
I20260709 13:06:08.969087 18734 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:08.975399 18741 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.006497 18695 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:06:08.967962 18716 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:09.027577 13943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.028s	sys 0.037s
I20260709 13:06:09.027746 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:08.996169 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:09.048292 18559 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:09.042924 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 77843us
W20260709 13:06:09.006106 13941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.015s	sys 0.071s
W20260709 13:06:09.056874 13941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.015s	sys 0.071s
I20260709 13:06:09.052035 13870 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:09.056600 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:09.060226 18734 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:09.060603 18734 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.060952 18734 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:09.061381 18525 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.001s
I20260709 13:06:09.063990 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:09.066260 18720 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:09.066637 18720 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:09.067022 18720 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:09.049893 18741 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:09.070531 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:09.071488 14021 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:09.077085 18734 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:09.078435 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:09.044322 18757 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.038s	sys 0.007s
I20260709 13:06:09.080551 13872 raft_consensus.cc:2410] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:06:09.044515 13942 rpcz_store.cc:276] 0709 13:06:07.620346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.627043 (+  6697us) service_pool.cc:224] Handling call
0709 13:06:07.651918 (+ 24875us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:08.144328 (+492410us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:08.170842 (+ 26514us) log.cc:844] Serialized 14 byte log entry
0709 13:06:08.259202 (+ 88360us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:08.269809 (+ 10607us) log.cc:844] Serialized 26 byte log entry
0709 13:06:08.278043 (+  8234us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:08.278167 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.278319 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:09.001833 (+723514us) raft_consensus.cc:1685] finished
0709 13:06:09.001856 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.002024 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37651,"prepare.queue_time_us":35059,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":19478,"raft.run_cpu_time_us":468,"raft.run_wall_time_us":62605,"thread_start_us":546465,"threads_started":2,"wal-append.queue_time_us":130298}
W20260709 13:06:09.051652 13943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.029s	sys 0.037s
I20260709 13:06:09.078435 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:09.061838 18525 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:09.084882 14023 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:08.994864 18492 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:09.095167 18488 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:09.099278 18769 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:06:09.094585 18693 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.104635 18693 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:09.105333 18693 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:09.118782 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:09.119758 13945 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:06:09.121791 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:09.086690 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.093250 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:09.094540 18701 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:09.129451 14024 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:06:09.131816 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:09.133117 13947 raft_consensus.cc:2410] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:06:09.139320 13945 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:06:09.105934 18768 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:06:09.166111 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:09.168416 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 432945us
I20260709 13:06:09.170550 13945 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:09.142962 18329 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:09.141781 18631 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:09.105931 18693 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.165380 18371 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:09.145380 18742 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:09.183445 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:09.159291 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2783229320000373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20950000000000002s for a response
I20260709 13:06:09.165689 18744 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:09.185022 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:09.186651 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:09.113988 18751 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.200524 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:09.210942 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 434595us
I20260709 13:06:09.192361 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:09.201738 18775 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:06:09.220965 13839 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:09.219049 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 466939us
I20260709 13:06:09.230464 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:09.232265 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:09.209151 18695 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:09.230089 18631 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:09.237375 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:09.237596 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:09.238272 13869 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:09.238719 18734 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:09.238906 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:09.239194 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:09.239368 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:09.239813 14022 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:09.239946 13869 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:09.240327 14020 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:06:09.240761 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:09.234975 18698 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:09.241627 18698 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.250080 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:09.242151 18698 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:09.242235 18713 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.252482 14022 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:09.253564 14018 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:09.245419 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:09.243139 18405 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:09.241937 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:09.256374 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:09.256882 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:09.259449 14022 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:09.249104 18712 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.244973 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:09.260218 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:09.266266 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:09.270231 14024 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:09.272748 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:09.267732 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 416768us
I20260709 13:06:09.277820 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:09.196424 18741 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:09.194599 18371 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.299408 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:09.313474 14022 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:09.310410 13944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.015s	sys 0.026s
W20260709 13:06:09.315482 13944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.015s	sys 0.027s
I20260709 13:06:09.193560 18744 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:09.366626 14023 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.366998 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:09.213557 18742 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.373466 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:09.296687 18708 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:09.202940 18748 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.232167 18720 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:09.376915 14020 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:09.378911 13869 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 8 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:09.372771 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 424468us
I20260709 13:06:09.375092 14554 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 7.
I20260709 13:06:09.405469 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:09.442191 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:09.482664 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:09.486328 14017 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:09.487802 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:09.462803 18773 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:06:09.497581 14023 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:09.508922 14017 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:09.514667 14554 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:06:09.535257 18751 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:09.535837 14554 raft_consensus.cc:606] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:06:09.536415 14554 raft_consensus.cc:614] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:09.515429 18729 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:09.514720 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:09.542834 13946 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.053s	sys 0.080s
W20260709 13:06:09.543164 13946 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.054s	sys 0.080s
I20260709 13:06:09.510989 18728 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:06:09.524405 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:09.525815 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:09.480795 18782 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:06:09.528158 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2708) took 1054 ms. Trace:
I20260709 13:06:09.547665 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:09.548367 13948 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:09.530292 18716 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:09.537060 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:09.548045 14019 rpcz_store.cc:276] 0709 13:06:08.473923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.722152 (+248229us) service_pool.cc:224] Handling call
0709 13:06:08.748799 (+ 26647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.794169 (+ 45370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.527861 (+733692us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b5400325ad0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:09.528055 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51451264}
I20260709 13:06:09.540956 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:09.541424 14022 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:09.549577 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:09.543921 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2251) took 1371 ms. Trace:
I20260709 13:06:09.553213 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:09.554162 13942 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:09.558780 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 48292.4us
I20260709 13:06:09.559628 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:09.558998 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:09.561393 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2243) took 1906 ms. Trace:
I20260709 13:06:09.566648 13949 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:09.567698 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 334285us
I20260709 13:06:09.563900 13943 rpcz_store.cc:276] 0709 13:06:07.655268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.659872 (+  4604us) service_pool.cc:224] Handling call
0709 13:06:07.660239 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.660456 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:08.186342 (+525886us) log.cc:844] Serialized 14 byte log entry
0709 13:06:09.558334 (+1371992us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:09.558397 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:09.558451 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.561198 (+  2747us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.561333 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8086,"prepare.queue_time_us":162047,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"thread_start_us":1832715,"threads_started":3,"wal-append.queue_time_us":1048824}
I20260709 13:06:09.542685 18729 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:09.589190 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:09.576817 14017 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:06:09.580138 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:09.582751 13941 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.010s	sys 0.026s
W20260709 13:06:09.591506 13941 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.010s	sys 0.027s
I20260709 13:06:09.592540 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2250) took 1420 ms. Trace:
I20260709 13:06:09.591536 13945 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:06:09.582437 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:09.598457 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:09.599383 13949 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:09.600797 14019 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:06:09.602469 14022 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:06:09.604146 14021 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:09.593163 13941 rpcz_store.cc:276] 0709 13:06:08.172557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.390432 (+217875us) service_pool.cc:224] Handling call
0709 13:06:08.432125 (+ 41693us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.432409 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:08.468943 (+ 36534us) log.cc:844] Serialized 14 byte log entry
0709 13:06:09.592033 (+1123090us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:09.592123 (+    90us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:09.592212 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.592307 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.592455 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":43485,"prepare.run_cpu_time_us":166,"prepare.run_wall_time_us":166,"raft.queue_time_us":502309,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":8222,"thread_start_us":1122582,"threads_started":2,"wal-append.queue_time_us":614140}
W20260709 13:06:09.603933 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 327389us
I20260709 13:06:09.552242 18713 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:09.561546 18632 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:09.610757 14019 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
W20260709 13:06:09.616959 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.620618 14554 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:06:09.621554 14554 raft_consensus.cc:606] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:06:09.622001 14554 raft_consensus.cc:614] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:09.555297 18329 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:06:09.568185 18791 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:09.635855 14019 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:06:09.642287 13948 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:09.644193 18716 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:09.645277 14021 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:09.596218 18693 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:09.646389 18741 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:09.647306 18371 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:09.664470 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:09.681936 14019 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:06:09.668897 14024 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:09.672369 13949 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:09.673902 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2249) took 1501 ms. Trace:
I20260709 13:06:09.679152 13945 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:06:09.675004 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2253) took 1501 ms. Trace:
I20260709 13:06:09.691344 13950 rpcz_store.cc:276] 0709 13:06:08.172224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.226228 (+ 54004us) service_pool.cc:224] Handling call
0709 13:06:08.226557 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.226793 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:08.277972 (+ 51179us) log.cc:844] Serialized 14 byte log entry
0709 13:06:09.315912 (+1037940us) log.cc:844] Serialized 14 byte log entry
0709 13:06:09.673383 (+357471us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:09.673446 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:09.673511 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.673619 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.673815 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34801,"prepare.queue_time_us":89218,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":2755,"raft.queue_time_us":1029184,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":1388169,"threads_started":3,"wal-append.queue_time_us":778344}
I20260709 13:06:09.697201 13944 rpcz_store.cc:276] 0709 13:06:08.173529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.691019 (+517490us) service_pool.cc:224] Handling call
0709 13:06:08.725456 (+ 34437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.725749 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:08.755473 (+ 29724us) log.cc:844] Serialized 14 byte log entry
0709 13:06:09.674538 (+919065us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:09.674606 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:09.674690 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.674785 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.674931 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":52319,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"thread_start_us":918546,"threads_started":2,"wal-append.queue_time_us":759344}
I20260709 13:06:09.690804 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:09.652771 18505 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:09.666263 18716 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.713212 13945 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:09.711787 14019 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:09.660235 18791 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:09.676124 18712 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:09.721369 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 161813us
I20260709 13:06:09.727656 13950 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:09.730347 14522 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:09.726150 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 167249us
I20260709 13:06:09.731734 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2699) took 1243 ms. Trace:
I20260709 13:06:09.716423 18716 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:09.736027 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:09.664330 18788 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:06:09.713057 18505 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.713824 18693 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:09.721299 18791 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:09.718451 18695 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:09.732331 14015 rpcz_store.cc:276] 0709 13:06:08.471439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.484375 (+ 12936us) service_pool.cc:224] Handling call
0709 13:06:08.538276 (+ 53901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.538517 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:09.054530 (+516013us) log.cc:844] Serialized 14 byte log entry
0709 13:06:09.714861 (+660331us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:09.714915 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:09.714968 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.715030 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.715132 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17459,"prepare.queue_time_us":834444,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"raft.queue_time_us":29000,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":37871,"thread_start_us":1044973,"threads_started":3,"wal-append.queue_time_us":535751}
I20260709 13:06:09.732227 18782 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:09.732089 18781 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:09.744030 18788 consensus_queue.cc:1243] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:09.736855 18631 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:09.737783 13950 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.744302 18505 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:09.737376 18720 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:09.738848 18748 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:09.730301 18698 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.777650 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:09.761797 18764 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 1.059s	user 0.008s	sys 0.003s
I20260709 13:06:09.736706 18741 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:06:09.757473 18782 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:09.738560 18492 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:06:09.749410 18712 consensus_queue.cc:1059] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:06:09.781366 14015 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.775877 18806 consensus_queue.cc:1059] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:06:09.759330 18711 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 13:06:09.780232 18783 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
W20260709 13:06:09.790526 18796 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
W20260709 13:06:09.790776 18799 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:06:09.776796 18631 raft_consensus.cc:697] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:09.766404 18781 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.750466 18693 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.730504 18729 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.763783 18695 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:09.770982 18797 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:09.812883 13942 raft_consensus.cc:2410] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:06:09.814559 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 205164us
I20260709 13:06:09.816160 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:09.869040 13949 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:09.873260 13942 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:09.812951 18728 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.806198 18807 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:09.875744 18740 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:09.876251 18806 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:09.879832 18811 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:06:09.886207 18695 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:09.553414 13946 rpcz_store.cc:276] 0709 13:06:08.172898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.537010 (+364112us) service_pool.cc:224] Handling call
0709 13:06:08.557325 (+ 20315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.557627 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:08.689841 (+132214us) log.cc:844] Serialized 14 byte log entry
0709 13:06:09.543449 (+853608us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:09.543512 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:09.543595 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.543690 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.543844 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24948,"prepare.queue_time_us":134892,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":521763,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"thread_start_us":853036,"threads_started":2,"wal-append.queue_time_us":743172}
I20260709 13:06:09.884289 18806 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:09.889910 18698 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:09.890206 18806 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.893937 18806 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:09.897794 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 81716.3us
I20260709 13:06:09.898936 18811 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:09.899470 18811 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:09.899811 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:09.900275 18811 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:09.901794 18810 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:09.903062 18631 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.904021 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:09.904341 14018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 534) took 1644 ms (client timeout 2104 ms). Trace:
I20260709 13:06:09.904655 14019 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:06:09.904819 14018 rpcz_store.cc:269] 0709 13:06:08.259628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.259767 (+   139us) service_pool.cc:224] Handling call
0709 13:06:09.904120 (+1644353us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b5400325fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:09.904254 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2134993664}
I20260709 13:06:09.906744 14021 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:09.907159 18693 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:09.907338 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2773) took 1369 ms. Trace:
I20260709 13:06:09.901909 18811 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:09.907732 13874 rpcz_store.cc:276] 0709 13:06:08.537607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.767228 (+229621us) service_pool.cc:224] Handling call
0709 13:06:08.767603 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:08.767858 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:08.767889 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:08.768414 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:06:09.167808 (+399394us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:09.192644 (+ 24836us) log.cc:844] Serialized 14 byte log entry
0709 13:06:09.199833 (+  7189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.199943 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:09.907097 (+707154us) raft_consensus.cc:1685] finished
0709 13:06:09.907118 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.907249 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6622,"prepare.queue_time_us":18277,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":6890,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"thread_start_us":398954,"threads_started":1,"wal-append.queue_time_us":726517}
I20260709 13:06:09.910843 18329 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:09.911317 18329 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:09.912408 18329 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:09.913517 18329 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:09.884487 18728 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.914551 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:09.903143 13989 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:09.918277 13840 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:09.921428 13950 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:09.916419 18810 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:06:09.888566 18802 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:06:09.885377 18713 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:09.927018 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:09.915669 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:09.928755 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:09.929427 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:09.933210 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:09.924819 18740 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:09.932771 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:09.882372 18797 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.950450 18818 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:09.950932 18713 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:09.932611 18712 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:09.954546 18808 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:06:09.956046 18793 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.956278 18405 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:09.923002 18693 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:09.958151 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:09.932989 18711 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:09.961127 13840 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:09.961175 18788 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:06:09.961784 18795 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.075s	user 0.001s	sys 0.001s
I20260709 13:06:09.884711 18807 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:09.989344 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:09.969869 18632 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:09.992167 18632 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:09.993944 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:09.994678 13874 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.926635 18809 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:09.982867 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:09.999003 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:10.001505 18632 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.002252 18794 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:10.002401 18800 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:10.002851 18794 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.002962 18800 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:10.003341 18794 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:10.003479 18800 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:10.004145 18632 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.004850 18632 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:10.005249 18712 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:10.005715 18712 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.005815 18632 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:10.006155 18712 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:10.006909 18632 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.009284 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:10.011298 13873 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:10.012950 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:10.013731 18157 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:10.014181 18157 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:10.014389 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:10.015121 13867 raft_consensus.cc:2368] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12.
I20260709 13:06:10.015017 18157 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.015255 18693 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.016283 18157 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:10.016402 13872 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:06:10.017617 18693 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:10.017900 13870 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:06:10.018214 18810 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.019076 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:10.020479 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:09.948504 18698 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:10.021595 18698 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.021966 18698 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:10.022432 18797 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:10.026837 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:10.027724 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 66693.7us
I20260709 13:06:10.027966 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:09.962328 18801 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:09.990840 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:10.031157 13870 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [6.5-11.7]
I20260709 13:06:09.924985 18749 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:06:10.028564 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:09.932960 18741 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:10.030541 18405 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:09.886236 18782 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.046482 14019 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:06:10.042343 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:10.043943 13871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2770) took 1635 ms (client timeout 1992 ms). Trace:
W20260709 13:06:10.047412 13871 rpcz_store.cc:269] 0709 13:06:08.408387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.501046 (+ 92659us) service_pool.cc:224] Handling call
0709 13:06:10.043703 (+1542657us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000a6c54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:10.043855 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2428433664}
W20260709 13:06:10.050194 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 59432.6us
I20260709 13:06:10.050999 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:09.987862 18505 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.051735 18782 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:10.055608 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:10.057230 18720 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:10.057775 18720 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:10.057976 18742 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:10.058725 18720 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.062731 18708 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:10.063318 18708 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:10.066198 13949 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:10.066443 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:10.067257 18728 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:10.068325 18801 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:10.068532 13941 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:10.070479 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:10.072007 18713 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:10.072606 18713 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:10.069941 13941 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:06:10.075407 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:10.075728 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:10.076552 14558 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
W20260709 13:06:10.079767 18798 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.414s	user 0.002s	sys 0.000s
I20260709 13:06:10.079811 18405 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:10.080608 18807 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:10.082479 18492 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:10.087525 18742 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:10.048278 18741 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:10.090917 18741 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.095820 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:09.987218 18793 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.078725 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:10.051537 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:10.111631 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.046983 18740 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:10.123376 14015 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
W20260709 13:06:10.123952 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 72472.4us
I20260709 13:06:10.077260 18808 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:10.141880 13874 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:10.107108 18544 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:10.153934 13871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:06:10.158584 13871 raft_consensus.cc:606] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 8 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:06:10.132768 18492 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:06:10.164844 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 89203.3us
I20260709 13:06:10.168123 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:10.170504 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:10.172547 13989 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:10.182989 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:10.183985 13874 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:10.143067 18405 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:06:10.201570 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:10.156963 18740 consensus_queue.cc:1059] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:06:10.176036 18734 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:10.177680 18744 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.158864 18112 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:06:10.186707 18559 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:10.214970 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:10.219497 13867 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:10.166468 18800 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.165817 18544 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.221415 14555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:06:10.223704 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:10.226974 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:10.230762 14555 raft_consensus.cc:606] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:10.156457 18713 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.203150 18781 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.207903 18632 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.240615 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 68122.8us
W20260709 13:06:10.244626 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.265484 13989 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:10.193610 18698 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.265722 13874 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 8.
I20260709 13:06:10.219477 18741 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:10.268514 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:10.270553 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.229132 18112 consensus_queue.cc:1243] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:10.329052 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:10.329792 13947 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:06:10.330439 13947 pending_rounds.cc:85] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 9
I20260709 13:06:10.331488 13989 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:10.228528 18797 raft_consensus.cc:476] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Not starting pre-election -- already a leader
I20260709 13:06:10.232201 18792 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:10.223099 18777 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:10.221022 18488 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.343973 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:10.330732 13947 pending_rounds.cc:107] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:06:10.347219 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:10.347477 13947 raft_consensus.cc:2889] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:10.347419 13945 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:10.370398 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:10.373850 13942 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:10.328926 18827 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.339488 18734 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:10.271891 18782 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.386968 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.348433 18797 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:10.390473 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 59078.4us
I20260709 13:06:10.229883 18716 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.396564 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:10.403113 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 176245us
I20260709 13:06:10.339075 18788 consensus_queue.cc:1059] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:06:10.352456 18741 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:10.422911 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:10.327348 18818 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:06:10.383955 18734 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:10.424463 14049 heartbeater.cc:646] Failed to heartbeat to 127.12.109.126:43315 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.12.109.126:43315 timed out after 15.000s (SENT)
I20260709 13:06:10.424726 18712 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.463404 18712 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:10.464833 18722 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:10.465298 18722 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:10.426261 18559 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:10.469336 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.426203 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:10.470793 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 44735.2us
I20260709 13:06:10.471678 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:10.472376 18800 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.472944 13989 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:10.430681 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:10.474535 18833 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 13:06:10.430835 18829 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:06:10.431736 18824 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
W20260709 13:06:10.432571 18828 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:06:10.433964 18698 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.435138 13942 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:06:10.273440 18632 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.346833 18525 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:06:10.458791 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:10.396338 18832 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:10.388815 18827 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.460566 18716 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:10.483771 18329 consensus_meta.cc:287] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.570s	user 0.010s	sys 0.003s
I20260709 13:06:10.394769 18793 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.349305 18792 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.404412 18782 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:10.478437 18791 consensus_meta.cc:287] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.720s	user 0.006s	sys 0.006s
W20260709 13:06:10.489830 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 360938us
W20260709 13:06:10.517697 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 59043.7us
I20260709 13:06:10.538962 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:10.484453 18788 raft_consensus.cc:3055] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:10.485422 18741 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:10.515482 18838 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:10.516366 18722 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.516147 18559 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:10.531951 18712 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:10.539796 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:10.484763 18809 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:10.457171 18734 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:10.461388 18826 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 13:06:10.557839 18695 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:06:10.541635 18791 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.561900 18791 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.564479 18801 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:10.541854 18793 raft_consensus.cc:2764] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:06:10.557972 18809 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.568488 18809 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:10.569257 18787 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:10.570533 18787 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:10.570900 18787 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:10.571717 18787 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.573160 18787 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:10.546922 18741 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.549391 18792 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.537063 18544 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.587466 18734 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.588586 18632 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:10.543147 18698 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:06:10.590579 18734 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:10.590756 18698 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:10.593390 18157 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:10.593780 18157 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:10.594606 18157 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.595894 18827 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.598819 18722 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:10.558724 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:10.655264 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 96645.3us
I20260709 13:06:10.657332 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:10.658249 18749 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:10.660112 18749 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:10.660506 18749 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:10.660897 18749 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:10.663842 18794 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.660s	user 0.008s	sys 0.005s
I20260709 13:06:10.664397 18794 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.665621 13946 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:06:10.666217 18792 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:10.667773 18792 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:10.669143 18794 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.669085 18792 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:10.678669 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:10.681921 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:10.682574 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:10.682744 18749 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.683506 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:10.684221 13871 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:10.684916 18749 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.483101 18797 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:10.685479 13872 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:10.688935 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:10.690346 13875 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:10.690575 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.691948 13872 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:10.692957 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:10.693560 13870 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:10.694499 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:10.695473 13949 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:10.482484 18818 consensus_queue.cc:1243] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:10.540342 18788 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:10.582470 18729 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:10.583595 18505 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:10.584506 18742 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:10.586381 18741 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:10.586658 18822 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:10.599540 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:10.705855 14561 raft_consensus.cc:1813] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Handling vote request from an unknown peer 3779649036364744823fc035dac892c7
I20260709 13:06:10.706351 14561 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 17: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:10.600172 14555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:06:10.707325 14555 raft_consensus.cc:606] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:10.707679 14555 raft_consensus.cc:614] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:10.551854 18782 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.595940 18157 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:10.709692 18797 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:10.611013 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:10.614287 14553 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:10.618104 18791 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:10.619046 18488 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:10.619844 18814 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.733s	user 0.002s	sys 0.000s
I20260709 13:06:10.578580 18525 consensus_queue.cc:1243] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:10.827853 13946 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:10.621553 18840 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:06:10.622447 18835 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:10.823132 18371 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.630321 18708 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 13:06:10.631577 13947 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:10.635064 13942 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:10.638317 18492 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:10.639145 18740 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:10.858542 18492 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:10.858716 18740 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:10.639801 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:10.859937 18740 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.859802 18492 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.861193 18492 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:10.646713 18712 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:10.624845 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:10.864727 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 283832us
I20260709 13:06:10.703418 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:10.703552 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:10.703876 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:10.866489 13871 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:10.704182 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:10.710994 18788 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.696914 18695 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.610076 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:10.713177 13949 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:10.713415 13870 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:06:10.714002 14555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:06:10.717846 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:10.722736 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 199722us
W20260709 13:06:10.723315 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.726847 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:10.869097 13868 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
W20260709 13:06:10.718112 18808 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.571s	user 0.012s	sys 0.000s
W20260709 13:06:10.748836 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.716275 18742 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:10.769867 13941 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:10.801154 13943 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
W20260709 13:06:10.805109 13950 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.025s	sys 0.054s
W20260709 13:06:10.865119 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 240332us
W20260709 13:06:10.873862 13950 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.025s	sys 0.054s
I20260709 13:06:10.765230 18811 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:06:10.807991 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 114333us
I20260709 13:06:10.809269 18751 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:10.880867 14017 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:06:10.881261 14017 pending_rounds.cc:85] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 5
I20260709 13:06:10.624796 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:10.881510 14017 pending_rounds.cc:107] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:06:10.865875 13873 raft_consensus.cc:1813] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Handling vote request from an unknown peer 3779649036364744823fc035dac892c7
I20260709 13:06:10.881888 14017 raft_consensus.cc:2889] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:10.870157 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 143442us
I20260709 13:06:10.883257 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:10.884497 14019 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:06:10.884629 13915 leader_election.cc:400] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12.
I20260709 13:06:10.881942 14551 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:10.886181 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:10.873586 14556 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:06:10.805540 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 315918us
I20260709 13:06:10.888222 18157 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:10.888715 18157 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.889222 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:10.890348 13873 raft_consensus.cc:2368] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18.
I20260709 13:06:10.846274 18835 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:10.876832 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:10.864431 18776 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.021s	sys 0.014s
I20260709 13:06:10.908542 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:10.874838 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:10.880447 14015 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:10.910209 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:10.910915 14015 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:10.891744 18157 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:10.876207 13949 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:10.893049 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.895195 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:10.877627 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:10.896888 13947 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:10.862849 18525 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.896924 14017 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:10.922031 18525 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.922890 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:10.923780 14017 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:06:10.924430 18525 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.924561 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:10.925360 14017 raft_consensus.cc:2393] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:10.878206 18794 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:10.881598 18808 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:10.896752 18544 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:06:10.930331 18852 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.892825 18810 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:06:10.930907 18852 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.931699 18810 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:10.932060 18810 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:10.933215 18852 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.932978 18810 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.863587 18371 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.881376 18781 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:10.895720 18751 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.934470 18781 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:10.935480 18371 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:10.935557 18781 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.905373 18835 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:10.827829 18488 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.940567 18488 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:10.880475 18712 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:10.909016 18776 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.021s	sys 0.014s
I20260709 13:06:10.880044 18695 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.944084 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2118) took 1642 ms. Trace:
I20260709 13:06:10.905877 14016 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:10.945192 14555 raft_consensus.cc:606] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:06:10.871023 18708 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.945631 14555 raft_consensus.cc:614] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:10.946142 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:10.946897 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.947271 18708 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:10.947999 13869 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:10.895256 18740 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:10.948794 13866 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:06:10.906724 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:10.916401 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:10.949788 14021 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 8: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:10.882606 18832 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:10.950609 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:10.951939 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:10.891738 18801 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:10.953536 18801 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:10.918031 18820 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:10.895306 18818 consensus_queue.cc:1059] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:06:10.944504 14023 rpcz_store.cc:276] 0709 13:06:09.301772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.590746 (+288974us) service_pool.cc:224] Handling call
0709 13:06:09.628575 (+ 37829us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:09.655166 (+ 26591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:09.655205 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:09.655761 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:06:09.749471 (+ 93710us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:09.775713 (+ 26242us) log.cc:844] Serialized 14 byte log entry
0709 13:06:09.776089 (+   376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.776228 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:09.894185 (+117957us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b18003735a0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:10.012712 (+118527us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b18003735a0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c730758f 00007fb2c0346571 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd
0709 13:06:10.618999 (+606287us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:10.943834 (+324835us) raft_consensus.cc:1685] finished
0709 13:06:10.943859 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.944014 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30793,"prepare.queue_time_us":25859,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":12483,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":199,"spinlock_wait_cycles":14631808,"thread_start_us":89023,"threads_started":1,"wal-append.queue_time_us":159564}
I20260709 13:06:10.912009 18806 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:06:10.878520 18811 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:10.919728 18749 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:10.936163 18751 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:10.937876 18729 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:10.891072 18112 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.946820 14016 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:10.950673 13875 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:10.915881 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:10.950690 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:10.917052 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:10.917795 18742 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:10.957046 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:11.003196 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:11.017765 14015 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
W20260709 13:06:11.018690 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 102877us
I20260709 13:06:11.019460 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:10.945096 18695 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:11.021767 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:10.915102 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:11.023352 13873 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:10.964887 14020 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:10.965338 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:10.971582 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 59066.5us
I20260709 13:06:11.024657 14023 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:06:11.025000 14023 pending_rounds.cc:85] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 5
I20260709 13:06:10.973685 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:11.025213 14023 pending_rounds.cc:107] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:06:11.025463 14023 raft_consensus.cc:2889] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:10.913189 13989 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.992s (SENT) [suppressed 10 similar messages]
I20260709 13:06:10.984501 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:10.988384 13867 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:10.951963 18786 raft_consensus.cc:993] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: : Instructing follower 3779649036364744823fc035dac892c7 to start an election
I20260709 13:06:11.003223 13871 raft_consensus.cc:2376] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11. Re-sending same reply.
I20260709 13:06:10.930928 18788 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:11.023582 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:11.041083 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:11.041558 18157 raft_consensus.cc:1081] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 8 LEADER]: Signalling peer 3779649036364744823fc035dac892c7 to start an election
I20260709 13:06:11.030030 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:11.047911 13869 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 8: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:11.048506 13868 raft_consensus.cc:2368] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18.
I20260709 13:06:11.049386 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:11.035494 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:11.050614 14022 raft_consensus.cc:1813] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Handling vote request from an unknown peer 3779649036364744823fc035dac892c7
I20260709 13:06:11.051052 14022 raft_consensus.cc:2368] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18.
I20260709 13:06:11.027144 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:06:10.972978 18820 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:11.054294 14020 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:06:11.052248 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:11.052565 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:11.000675 18795 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:06:11.077996 14021 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:06:11.080509 13947 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:06:11.081884 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:11.085124 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:10.985683 18782 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:06:11.027877 18810 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:11.091346 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 106946us
I20260709 13:06:11.106477 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:11.110747 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:11.092900 14551 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:06:11.089775 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 93950.7us
I20260709 13:06:11.088686 14017 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:11.114804 14017 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:06:11.030387 18791 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:11.021842 18729 consensus_queue.cc:1243] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:11.120903 13871 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:11.080952 18820 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:11.128208 14020 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:11.081686 18853 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:11.147943 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:11.149873 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:11.152585 13871 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:11.080310 18751 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:06:11.029155 18112 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:11.097083 18841 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:11.154331 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:11.109225 18695 consensus_queue.cc:1059] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:06:11.135317 18850 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.145088 18844 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.191372 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2262) took 1609 ms. Trace:
I20260709 13:06:11.193810 13943 rpcz_store.cc:276] 0709 13:06:09.574003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.629834 (+ 55831us) service_pool.cc:224] Handling call
0709 13:06:11.183249 (+1553415us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001c6454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:11.183394 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2574371584}
I20260709 13:06:11.191913 18488 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:11.230647 13866 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
W20260709 13:06:11.268002 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 113763us
I20260709 13:06:11.268841 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:11.164008 18822 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:11.173209 18793 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:11.281543 13944 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:11.269742 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:11.282650 14024 raft_consensus.cc:1217] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Deduplicated request from leader. Original: 5.4->[6.5-11.7]   Dedup: 8.6->[11.7-11.7]
I20260709 13:06:11.283239 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:11.163429 18853 raft_consensus.cc:697] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:11.288254 14024 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [6.5-11.7]
I20260709 13:06:11.291420 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:11.296087 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:11.267217 18751 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.297765 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:11.304337 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:11.200093 18488 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:11.307786 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:11.280066 18787 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:11.313339 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:11.318399 13941 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 12: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:11.318239 13873 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
W20260709 13:06:11.319375 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 406173us
I20260709 13:06:11.288727 18806 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.291018 18793 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:11.325399 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:11.300374 18818 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:11.328656 13873 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:06:11.295717 18112 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:11.326781 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 306238us
I20260709 13:06:11.298544 18712 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:11.301680 18840 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:11.347548 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:11.309417 18785 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:11.365533 18525 raft_consensus.cc:3055] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:11.402838 18831 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.405020 18803 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:11.413524 18852 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:11.418310 18811 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:11.423604 18848 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.504s	user 0.004s	sys 0.004s
W20260709 13:06:11.435834 18845 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.518s	user 0.003s	sys 0.000s
I20260709 13:06:11.445215 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:06:11.455709 18846 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.535s	user 0.004s	sys 0.004s
I20260709 13:06:11.456992 13867 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:11.318930 18794 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:11.316639 18787 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:11.502132 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.502429 18787 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:11.472273 13943 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a"
dest_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:35884
W20260709 13:06:11.502911 18858 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:06:11.331243 18371 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.154s
I20260709 13:06:11.200793 18850 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:11.469863 13838 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:11.305812 18695 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:06:11.316953 18744 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.166s
I20260709 13:06:11.462872 18801 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:11.474843 18803 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:11.474619 18525 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:11.472506 18818 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:11.328316 18806 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:11.318468 18751 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:11.513942 18711 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:11.548882 18847 log.cc:927] Time spent T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.440s	user 0.001s	sys 0.001s
I20260709 13:06:11.486701 18831 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:11.522670 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52933.8us
W20260709 13:06:11.494452 18842 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260709 13:06:11.465186 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 117801us
I20260709 13:06:11.482240 18852 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:11.486555 18544 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:06:11.512493 18834 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:06:11.294399 18822 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:11.273722 18844 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:11.541466 18749 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.482996 18488 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:11.534745 18811 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:11.567386 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:11.544816 18785 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:11.593480 13875 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:11.512599 18793 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:11.586935 18791 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:11.596729 18544 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:11.597496 18488 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:11.597061 18791 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:11.602319 18751 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:11.602604 18791 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:11.595778 18803 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:11.605885 18803 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:11.606165 18835 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.000s
I20260709 13:06:11.606741 18835 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:11.608034 13945 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:11.609225 18835 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:11.609777 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2286) took 1004 ms. Trace:
I20260709 13:06:11.603245 18751 raft_consensus.cc:3055] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:11.567464 18818 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:11.585644 18525 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:11.610172 13945 rpcz_store.cc:276] 0709 13:06:10.604968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.605126 (+   158us) service_pool.cc:224] Handling call
0709 13:06:10.605486 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.608419 (+1002933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.609491 (+  1072us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400092754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:11.609716 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2305178880}
I20260709 13:06:11.615027 18525 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:11.621265 18831 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:11.558771 18711 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:11.626770 18801 raft_consensus.cc:3055] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:11.626334 18791 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:11.627182 18801 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:11.628198 18801 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:11.587361 18869 consensus_queue.cc:1175] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" responder_term: 11 status { last_received { term: 11 index: 10 } last_committed_idx: 9 last_received_current_leader { term: 11 index: 10 } }
I20260709 13:06:11.610384 18793 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:11.582911 18785 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.632951 18785 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:11.633505 18788 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.002s
W20260709 13:06:11.636801 18740 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.688s	user 0.011s	sys 0.000s
I20260709 13:06:11.640630 18844 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:11.641829 18859 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:06:11.647399 13875 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:11.504649 18853 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:11.569218 18852 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:11.674985 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 364920us
I20260709 13:06:11.610625 18751 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:11.630296 18711 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:11.677935 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:11.580909 18749 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:11.693967 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:11.695041 13870 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
W20260709 13:06:11.675093 18834 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:06:11.697921 13942 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:06:11.701440 13942 pending_rounds.cc:85] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 6
I20260709 13:06:11.701825 13942 pending_rounds.cc:107] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:06:11.702191 13942 raft_consensus.cc:2889] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:11.710325 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:11.711383 13948 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:11.725929 13866 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:06:11.733479 13871 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.580s	user 0.014s	sys 0.000s
I20260709 13:06:11.734174 13871 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:11.735064 13873 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:06:11.690828 18405 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:06:11.753223 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 174141us
W20260709 13:06:11.735625 18860 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:06:11.749154 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 479433us
W20260709 13:06:11.810081 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 155629us
I20260709 13:06:11.810935 13915 leader_election.cc:400] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18.
I20260709 13:06:11.811401 13915 leader_election.cc:403] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:11.690793 18831 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:06:11.761891 18840 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:11.747035 18788 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:11.768338 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:11.818073 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:11.777540 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:11.767977 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:11.818990 13945 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:11.819450 13948 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:11.826077 13943 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:11.826645 13943 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:11.757647 18791 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:11.836050 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:11.820791 13868 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.024s	sys 0.023s
W20260709 13:06:11.830386 18492 consensus_meta.cc:287] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.969s	user 0.006s	sys 0.010s
W20260709 13:06:11.839097 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:11.835964 18849 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.909s	user 0.002s	sys 0.003s
W20260709 13:06:11.834901 18810 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.716s	user 0.009s	sys 0.005s
I20260709 13:06:11.758363 18405 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:11.771314 18740 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:06:11.851374 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:11.861943 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:11.863032 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:11.848421 13872 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
W20260709 13:06:11.854189 14015 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.012s	sys 0.013s
I20260709 13:06:11.840044 18850 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:11.899794 14016 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:06:11.840613 13868 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.024s	sys 0.023s
W20260709 13:06:11.887960 14015 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.013s	sys 0.013s
I20260709 13:06:11.844748 18874 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:06:11.828460 18788 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:11.885979 18835 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.905025 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2844) took 1228 ms. Trace:
I20260709 13:06:11.890478 18729 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:11.890655 18782 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:11.890002 18880 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:11.914886 18749 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:11.899988 13943 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:11.918628 18885 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:06:11.922209 18840 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:11.923408 18822 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:11.923854 18881 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:06:11.926218 18720 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:11.926751 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 540940us
I20260709 13:06:11.989970 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2806) took 1097 ms. Trace:
I20260709 13:06:11.953493 18782 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.931102 18806 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:11.934830 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 256964us
I20260709 13:06:11.935382 18810 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.936147 18808 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:11.937326 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:11.940506 18801 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:11.941546 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:12.041854 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 104666us
W20260709 13:06:11.942678 13872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2852) took 1245 ms (client timeout 1525 ms). Trace:
W20260709 13:06:11.847030 18781 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.003s	sys 0.035s
I20260709 13:06:12.046397 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:11.950234 13874 rpcz_store.cc:276] 0709 13:06:10.676748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.679616 (+  2868us) service_pool.cc:224] Handling call
0709 13:06:10.680061 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.680490 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:10.680519 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.681063 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 13:06:10.859272 (+178209us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:10.859324 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.859422 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:11.592751 (+733329us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:11.904676 (+311925us) raft_consensus.cc:1685] finished
0709 13:06:11.904704 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.904952 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6880,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":106,"spinlock_wait_cycles":64491776,"thread_start_us":177727,"threads_started":1,"wal-append.queue_time_us":244888}
I20260709 13:06:11.952324 18544 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:11.909752 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2778) took 1033 ms. Trace:
W20260709 13:06:11.946887 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 169458us
I20260709 13:06:11.848737 18112 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
W20260709 13:06:11.958384 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.959424 18788 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:11.960600 18820 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.809s	user 0.007s	sys 0.003s
W20260709 13:06:12.094403 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 230976us
I20260709 13:06:12.095183 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:12.095395 13875 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:11.961766 18729 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:11.808673 18751 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:11.848649 18876 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.116s
I20260709 13:06:11.909562 18787 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:11.984421 18405 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:11.962630 18880 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.104099 18880 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:11.946360 18882 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:11.991456 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:11.927529 18741 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:11.990510 13868 rpcz_store.cc:276] 0709 13:06:10.892548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.077684 (+185136us) service_pool.cc:224] Handling call
0709 13:06:11.078631 (+   947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.078845 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:11.901325 (+822480us) log.cc:844] Serialized 14 byte log entry
0709 13:06:11.989278 (+ 87953us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:11.989410 (+   132us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:11.989541 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.989731 (+   190us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.989896 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14856,"prepare.queue_time_us":233504,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":858826,"threads_started":2}
W20260709 13:06:12.036084 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:12.042680 13872 rpcz_store.cc:269] 0709 13:06:10.697484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.697602 (+   118us) service_pool.cc:224] Handling call
0709 13:06:11.942435 (+1244833us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b54000ea954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:11.942601 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2255603968}
I20260709 13:06:12.049902 13915 leader_election.cc:400] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18.
W20260709 13:06:12.110913 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.956167 18835 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.110517 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:11.994815 18722 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:06:12.086194 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.093014 13942 rpcz_store.cc:276] 0709 13:06:10.876590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.952815 (+ 76225us) service_pool.cc:224] Handling call
0709 13:06:10.953110 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.728285 (+775175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.909488 (+181203us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b54000f5354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:11.909684 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1702616448}
I20260709 13:06:12.050390 18806 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:12.106577 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 115229us
I20260709 13:06:12.109007 18716 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:06:12.111665 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 61890.8us
I20260709 13:06:11.998598 18720 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.113338 18882 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:12.138936 18882 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.139729 18720 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:12.140353 13944 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:06:12.140761 13944 pending_rounds.cc:85] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 6
I20260709 13:06:12.140997 13944 pending_rounds.cc:107] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:06:12.141234 13944 raft_consensus.cc:2889] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:12.116736 18787 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:12.115839 18803 consensus_meta.cc:287] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.508s	user 0.013s	sys 0.000s
I20260709 13:06:12.100524 18777 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.997998 18782 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:12.050791 18810 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.093603 18808 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:12.168082 18808 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.168907 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:12.122560 14552 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:12.122949 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:12.170044 13914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.004s	sys 0.007s
I20260709 13:06:12.174381 13875 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:12.170853 18713 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:12.046459 18781 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.003s	sys 0.036s
I20260709 13:06:12.130040 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:12.175307 18713 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.176143 13945 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:12.098537 18820 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:12.177484 14551 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 14. Current term is 15. Ops: []
W20260709 13:06:12.178686 18857 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.861s	user 0.003s	sys 0.005s
I20260709 13:06:12.131214 18729 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.179577 18820 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:12.133817 18884 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:06:12.135008 18712 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.181011 18712 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.181636 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2857) took 1461 ms. Trace:
I20260709 13:06:12.182430 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2140) took 1295 ms. Trace:
I20260709 13:06:12.182555 18713 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:12.136212 18835 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:12.137437 18785 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.000s
I20260709 13:06:12.118350 18741 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:12.146214 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:12.147053 13989 leader_election.cc:400] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18.
I20260709 13:06:12.193674 13989 leader_election.cc:403] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:12.152549 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 45582.1us
I20260709 13:06:12.158751 18720 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.202699 18810 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:12.191351 18803 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:12.203141 18720 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.165302 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:12.165002 18777 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.204846 18880 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.205960 18505 raft_consensus.cc:3055] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:12.206292 18777 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:12.206419 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:12.144868 18716 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:12.206714 18785 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:12.207660 18716 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:12.209275 18785 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:12.210199 18741 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:12.212415 18741 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:12.133853 18811 consensus_meta.cc:287] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.003s
W20260709 13:06:12.174540 13914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.006s	sys 0.007s
W20260709 13:06:12.214789 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 852620us
I20260709 13:06:12.182920 14019 rpcz_store.cc:276] 0709 13:06:10.887504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.887603 (+    99us) service_pool.cc:224] Handling call
0709 13:06:10.887895 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.888238 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:10.888273 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.907401 (+ 19128us) log.cc:844] Serialized 26 byte log entry
0709 13:06:11.095207 (+187806us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:11.533217 (+438010us) log.cc:844] Serialized 14 byte log entry
0709 13:06:11.946514 (+413297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.946634 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:12.177927 (+231293us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:12.182139 (+  4212us) raft_consensus.cc:1685] finished
0709 13:06:12.182230 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.182360 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":427758,"prepare.queue_time_us":96210,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":404521,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":50324,"spinlock_wait_cycles":12081920,"thread_start_us":531725,"threads_started":2,"wal-append.queue_time_us":252743}
I20260709 13:06:12.216084 14555 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:12.171139 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:12.144109 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:12.183620 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2146) took 1290 ms. Trace:
I20260709 13:06:12.124119 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:12.174778 13868 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:06:12.185055 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62229us
I20260709 13:06:12.197994 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:12.205736 18720 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:12.168108 18734 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:12.204393 14558 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:12.114342 14556 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:12.201468 18794 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:06:12.203907 18855 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.910s	user 0.003s	sys 0.000s
I20260709 13:06:12.145994 18787 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:12.221668 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:12.217170 14561 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:12.230971 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2285) took 2117 ms. Trace:
I20260709 13:06:12.225953 18803 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:06:12.239171 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:12.239393 18803 raft_consensus.cc:3055] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:12.239921 13942 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:12.228143 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:12.182088 13869 rpcz_store.cc:276] 0709 13:06:10.720831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.086294 (+365463us) service_pool.cc:224] Handling call
0709 13:06:11.086656 (+   362us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:11.157325 (+ 70669us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:11.588758 (+431433us) log.cc:844] Serialized 14 byte log entry
0709 13:06:11.809152 (+220394us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:11.816399 (+  7247us) log.cc:844] Serialized 52 byte log entry
0709 13:06:11.816581 (+   182us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:11.823745 (+  7164us) log.cc:844] Serialized 14 byte log entry
0709 13:06:11.840780 (+ 17035us) log.cc:844] Serialized 14 byte log entry
0709 13:06:11.871298 (+ 30518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.871522 (+   224us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:11.916819 (+ 45297us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b180035f1a0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:12.181379 (+264560us) raft_consensus.cc:1685] finished
0709 13:06:12.181407 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.181570 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":129549,"prepare.queue_time_us":323807,"prepare.run_cpu_time_us":176,"prepare.run_wall_time_us":3405,"raft.queue_time_us":214132,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":28422,"spinlock_wait_cycles":15121536,"thread_start_us":531560,"threads_started":3,"wal-append.queue_time_us":527865}
W20260709 13:06:12.221220 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.216970 14015 rpcz_store.cc:276] 0709 13:06:10.892879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.019792 (+126913us) service_pool.cc:224] Handling call
0709 13:06:11.020145 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:11.045451 (+ 25306us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:11.284294 (+238843us) log.cc:844] Serialized 14 byte log entry
0709 13:06:11.945670 (+661376us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:11.953112 (+  7442us) log.cc:844] Serialized 26 byte log entry
0709 13:06:11.954802 (+  1690us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:11.954863 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.955072 (+   209us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:12.183390 (+228318us) raft_consensus.cc:1685] finished
0709 13:06:12.183416 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.183558 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106732,"prepare.queue_time_us":123564,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":78217,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":22857,"thread_start_us":767089,"threads_started":2,"wal-append.queue_time_us":668466}
I20260709 13:06:12.240978 13875 raft_consensus.cc:2368] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 16. Current term is 17.
I20260709 13:06:12.226239 13873 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:12.237540 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:12.229506 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:12.242070 13941 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:12.227803 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:12.243955 13874 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:12.239825 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2727) took 1241 ms. Trace:
I20260709 13:06:12.238785 13950 rpcz_store.cc:276] 0709 13:06:10.113700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.113881 (+   181us) service_pool.cc:224] Handling call
0709 13:06:10.114285 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.607538 (+493253us) spinlock_profiling.cc:243] Waited 8.74 ms on lock 0x7b480012c3d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:06:10.874534 (+266996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:10.874567 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.921347 (+ 46780us) log.cc:844] Serialized 26 byte log entry
0709 13:06:11.091186 (+169839us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:11.091258 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.091390 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:12.203231 (+1111841us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:12.230705 (+ 27474us) raft_consensus.cc:1685] finished
0709 13:06:12.230730 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.230884 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11505,"raft.queue_time_us":15619,"raft.run_cpu_time_us":3379,"raft.run_wall_time_us":476320,"spinlock_wait_cycles":32784640,"thread_start_us":903919,"threads_started":2,"wal-append.queue_time_us":262928}
W20260709 13:06:12.247385 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:12.248345 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 101453us
I20260709 13:06:12.249306 13989 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:12.231149 18712 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:12.230280 18876 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:06:12.251994 18787 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.227157 18898 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:12.253255 18898 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:12.245536 14016 rpcz_store.cc:276] 0709 13:06:10.998949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.002014 (+  3065us) service_pool.cc:224] Handling call
0709 13:06:12.239619 (+1237605us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b540023f854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:12.239762 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1877053824}
I20260709 13:06:12.255750 14553 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:12.242803 13838 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:12.183470 18751 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:12.261440 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.268755 13838 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:12.283851 14556 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
W20260709 13:06:12.284921 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 43789.7us
I20260709 13:06:12.288200 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:12.289448 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:06:12.233331 18878 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:06:12.285732 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:12.254597 18787 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:12.290755 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:12.294781 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:12.234167 18794 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:12.308475 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:12.309238 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:12.239825 18803 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:12.296905 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:12.206370 18505 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:12.306957 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:12.317884 14551 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:12.310390 14561 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:12.296234 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:12.313011 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:12.315449 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:12.257189 18777 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18.
I20260709 13:06:12.284862 18785 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:12.324344 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:12.295938 18786 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:12.325763 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:12.299328 18844 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:12.296221 18741 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.226298 18734 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:12.299862 18818 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.004s	sys 0.010s
I20260709 13:06:12.332333 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:12.321967 13988 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:12.333874 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:12.295609 18712 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:12.298025 18793 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.340592 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:12.323900 13942 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:12.360957 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:12.322649 18720 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:12.297204 18559 consensus_queue.cc:1059] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:06:12.377202 14015 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:12.377539 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:12.365480 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:12.324995 18832 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:12.327499 18785 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.379520 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:12.372809 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:12.327320 18787 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:12.339254 18786 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.378400 14016 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:12.379384 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:12.329936 18810 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:06:12.378388 14019 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
W20260709 13:06:12.382814 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 54700.9us
I20260709 13:06:12.385998 13941 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:12.564198 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:12.574404 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:12.576229 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:12.486930 14015 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:12.577663 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:12.583001 13869 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:12.584686 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:12.365126 18749 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.578233 14018 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:12.383605 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:12.384994 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51191.1us
I20260709 13:06:12.579285 14020 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
W20260709 13:06:12.383984 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 62124us
I20260709 13:06:12.649673 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:12.648773 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:12.674508 13942 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
W20260709 13:06:12.629289 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.648301 13874 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [6.5-11.7]
I20260709 13:06:12.643683 13873 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:06:12.648339 13875 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 12 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:06:12.654377 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:12.625944 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 53825.2us
I20260709 13:06:12.590193 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:12.608472 14015 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:12.752274 14016 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:12.380259 18818 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.004s	sys 0.010s
W20260709 13:06:12.600798 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 217797us
I20260709 13:06:12.385469 13950 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:12.467218 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:12.384251 13838 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:12.360229 18831 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.577993 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:12.772253 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45485.1us
I20260709 13:06:12.754927 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:12.773731 14524 leader_election.cc:400] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 16. Current term is 17.
I20260709 13:06:12.774070 14524 leader_election.cc:403] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:12.774155 13875 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:12.462129 18405 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:12.589167 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.766503 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:12.531860 18878 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:06:12.580955 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:12.770558 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:12.776139 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:12.555341 18880 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:12.461621 18901 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:12.526798 18729 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.494639 18712 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:12.585371 18785 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:12.783314 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 401421us
I20260709 13:06:12.760922 18896 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.792241 13943 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:12.476001 18882 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:12.780402 14018 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:06:12.794435 14015 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:12.831562 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:12.573649 18797 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.532688 18898 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:12.800730 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:12.802436 18874 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.834s	user 0.004s	sys 0.009s
I20260709 13:06:12.586814 18786 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:12.554589 18713 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.223s
I20260709 13:06:12.764894 18525 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.457885 18793 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.819223 13875 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:12.911379 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:35884 (request call id 2303) took 1601 ms. Trace:
I20260709 13:06:12.787310 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:12.914727 13943 rpcz_store.cc:276] 0709 13:06:11.229237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.341790 (+112553us) service_pool.cc:224] Handling call
0709 13:06:12.790500 (+1448710us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54000f4954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c342aa3d 00007fb2c83facd0 00007fb2c197f254 00007fb2c197f1e2 00007fb2c197f111 00007fb2c197f08c 00007fb2c197e342 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:12.829997 (+ 39497us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":262,"spinlock_wait_cycles":742155136}
I20260709 13:06:12.382540 18741 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.801870 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:12.845747 14020 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:12.849773 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:12.852898 18157 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:12.780790 18832 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.642144 18787 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:12.793040 18810 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:12.909133 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:12.908601 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:12.384251 18803 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:13.028172 14523 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.12.109.126:43315, user_credentials={real_user=slave}} blocked reactor thread for 73483.9us
I20260709 13:06:12.781664 18544 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:06:13.023285 14018 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
W20260709 13:06:12.855141 18906 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.393s	user 0.001s	sys 0.000s
W20260709 13:06:12.966964 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 486055us
W20260709 13:06:12.915791 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 128586us
I20260709 13:06:12.801421 18911 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:06:13.023730 14019 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:12.863056 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2148) took 1969 ms. Trace:
I20260709 13:06:13.025394 18157 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:12.780917 18844 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:06:12.781239 18831 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.063210 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:13.064234 14018 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:12.913872 18712 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.762630 18749 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:12.793244 18405 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.036533 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:13.072609 14016 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:12.814989 18729 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.054795 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:13.041465 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:13.068147 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:12.966445 18898 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:12.860939 18874 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18.
I20260709 13:06:13.043881 18734 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:12.965353 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:12.898846 18791 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.201s
I20260709 13:06:12.852254 18797 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.029227 18559 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:13.130489 18559 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.039595 18832 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.133140 18559 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:13.037958 18890 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.133559 14024 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:13.133890 18890 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:13.139402 14024 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:12.855954 18882 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:13.100880 14019 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:06:13.065392 18910 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.536s	user 0.001s	sys 0.001s
I20260709 13:06:13.106935 14016 raft_consensus.cc:1217] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Deduplicated request from leader. Original: 7.6->[8.7-10.8]   Dedup: 8.7->[10.8-10.8]
I20260709 13:06:13.106120 14020 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:06:12.856453 18894 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:12.849607 18896 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.110706 13875 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:13.110777 14017 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:13.107518 13869 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
W20260709 13:06:13.111663 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 70289.9us
I20260709 13:06:13.048864 14023 rpcz_store.cc:276] 0709 13:06:10.893516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.026393 (+132877us) service_pool.cc:224] Handling call
0709 13:06:11.066530 (+ 40137us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:11.121900 (+ 55370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:11.121933 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:11.441734 (+319801us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b2c000e8788. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:06:11.442655 (+   921us) log.cc:844] Serialized 52 byte log entry
0709 13:06:11.854991 (+412336us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:11.947509 (+ 92518us) log.cc:844] Serialized 14 byte log entry
0709 13:06:11.947909 (+   400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.948029 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:12.232154 (+284125us) spinlock_profiling.cc:243] Waited 5.65 ms on lock 0x7b180018c460. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:12.862668 (+630514us) raft_consensus.cc:1685] finished
0709 13:06:12.862705 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.862938 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54808,"prepare.queue_time_us":95072,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":55358,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":6257,"spinlock_wait_cycles":15996032,"thread_start_us":395461,"threads_started":1,"wal-append.queue_time_us":655928}
I20260709 13:06:13.113323 18917 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:06:13.113600 18912 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.541s	user 0.000s	sys 0.002s
I20260709 13:06:13.114640 18916 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.461s
I20260709 13:06:13.115639 18874 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:13.119065 18492 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.119378 18793 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:13.122627 18915 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.354s	user 0.001s	sys 0.000s
I20260709 13:06:13.127523 18787 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:13.035508 18810 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:13.149158 13913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.047s	sys 0.074s
I20260709 13:06:13.151904 14016 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11. Ops: [8.7-10.8]
W20260709 13:06:13.225145 13913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.047s	sys 0.074s
I20260709 13:06:13.112581 18898 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.237679 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.075616 18157 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:12.956293 18525 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.136736 18890 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:13.078418 18712 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.251699 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.148226 18559 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:13.178529 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:13.255524 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.178031 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:13.256809 14551 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:13.177403 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:13.181005 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.182091 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.156293 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.188571 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:13.176002 18914 log.cc:927] Time spent T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.525s	user 0.000s	sys 0.002s
I20260709 13:06:13.170089 18832 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:13.199338 13875 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:13.171324 18894 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:13.179194 18916 consensus_queue.cc:1243] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:06:13.231673 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.233759 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.235484 13866 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:06:13.193099 18801 consensus_meta.cc:287] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.138s	user 0.008s	sys 0.006s
I20260709 13:06:13.258701 14015 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:13.262285 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:13.281286 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 242285us
I20260709 13:06:13.262841 13867 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:06:13.255854 14554 raft_consensus.cc:2376] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 18. Re-sending same reply.
I20260709 13:06:13.227093 18840 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:13.289096 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.288965 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:13.189425 18492 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.175037 18920 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:06:13.247543 18898 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:13.279893 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.295557 13873 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:13.281903 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.296612 13874 raft_consensus.cc:2410] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:06:13.283035 13870 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:13.273054 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.293761 14015 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:13.281525 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.296846 13866 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:13.187881 18874 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.265995 18559 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.300741 18927 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.332679 18927 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.334270 18741 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:13.297897 18921 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:13.335070 18712 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:13.335785 18921 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:13.339192 18896 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.341324 18749 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:13.344537 18882 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:13.345975 18806 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.351217 18806 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.352073 18874 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.352862 18874 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.302088 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:13.365557 13915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.002s	sys 0.031s
W20260709 13:06:13.365952 13915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.002s	sys 0.031s
I20260709 13:06:13.297997 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:13.316699 14018 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:13.353442 18874 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:13.328289 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.355094 13944 pending_rounds.cc:85] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 7
I20260709 13:06:13.357304 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:13.375435 13944 pending_rounds.cc:107] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:06:13.375743 13944 raft_consensus.cc:2889] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:13.378518 13948 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:06:13.360222 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 438745us
W20260709 13:06:13.379591 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 79860us
I20260709 13:06:13.395448 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:13.398440 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:13.302134 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:13.413364 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:13.299707 18907 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
W20260709 13:06:13.310595 18909 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:06:13.319594 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:13.357234 18782 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.167s	user 0.013s	sys 0.000s
I20260709 13:06:13.419351 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:13.429034 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:13.308780 18832 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:13.303798 18559 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:13.324373 18794 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.995s	user 0.004s	sys 0.008s
I20260709 13:06:13.359587 18911 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 16. Current term is 17.
I20260709 13:06:13.311944 18810 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:13.331732 18729 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.330415 18405 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.331126 18831 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.430559 13839 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:13.353991 18806 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.422075 18896 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:13.465327 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:13.297991 18840 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:13.475209 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 44736.7us
I20260709 13:06:13.479676 13839 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:13.399515 18920 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:13.354451 18922 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:06:13.417140 18712 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:06:13.309763 18919 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.333s
I20260709 13:06:13.437892 18832 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.483660 13839 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:13.471091 13988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.010s	sys 0.032s
W20260709 13:06:13.489570 13988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.010s	sys 0.032s
I20260709 13:06:13.321332 18505 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:13.480669 18751 consensus_meta.cc:287] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.194s	user 0.015s	sys 0.000s
I20260709 13:06:13.491595 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.492552 13950 raft_consensus.cc:2410] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 16 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:06:13.492813 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:13.481076 18920 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:13.432750 18488 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.000s	sys 0.029s
I20260709 13:06:13.480223 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:06:13.489930 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 201606us
I20260709 13:06:13.435015 18782 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.482561 18405 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:13.482791 18887 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:13.495437 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.483156 18896 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:13.436851 18791 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:06:13.498791 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.609599 13945 raft_consensus.cc:2410] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:06:13.499049 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.472327 18794 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.502996 18492 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.503742 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.612589 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.506150 18901 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.507038 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.613274 13945 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:13.613524 13950 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
W20260709 13:06:13.608760 18785 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.765s	user 0.014s	sys 0.000s
I20260709 13:06:13.614535 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.614760 18785 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.615348 13950 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:13.615868 18794 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.508985 18505 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:13.616207 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:13.501219 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.510478 18940 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:13.617416 18940 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:13.570302 18810 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:13.508249 18852 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:13.614310 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.496276 18832 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:13.567431 18887 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:13.623104 18887 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:13.512351 18695 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:13.512997 18157 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:13.512609 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.486012 18712 consensus_queue.cc:1243] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:13.517633 14559 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:13.478310 18786 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.626685 18874 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.627940 18786 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:13.519462 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.515098 18927 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.521591 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:13.488653 18882 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:13.617846 18940 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:13.567940 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:13.482088 18716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.017s	sys 0.025s
I20260709 13:06:13.311122 18894 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:13.524566 18488 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.001s	sys 0.029s
I20260709 13:06:13.482326 18831 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:06:13.489782 18931 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:06:13.617019 18923 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
W20260709 13:06:13.642870 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 121387us
I20260709 13:06:13.644045 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:13.645310 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:13.633858 18716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.081s	user 0.018s	sys 0.029s
I20260709 13:06:13.648017 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.648698 13874 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:13.649627 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.650357 13875 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:13.573534 18751 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:13.574206 18930 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:13.575781 18782 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.524165 18920 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:13.577780 18928 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.647068 18794 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:13.662480 18928 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.663105 18794 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.665376 18794 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.662503 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.666204 18931 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:13.667371 18695 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:13.667871 18695 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:13.667332 18931 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:13.581033 18921 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:13.668787 18695 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:13.581418 18889 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.004s	sys 0.012s
I20260709 13:06:13.582928 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2736) took 1176 ms. Trace:
I20260709 13:06:13.584686 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:13.591451 18777 raft_consensus.cc:3055] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:13.596247 18788 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:06:13.569521 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 89376.1us
W20260709 13:06:13.585848 18935 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.079s	user 0.001s	sys 0.001s
I20260709 13:06:13.682672 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.690414 13875 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:13.677798 18941 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:13.687243 18751 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:13.692075 18751 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:13.611374 13941 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:13.525743 18835 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:13.698733 18835 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.699316 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.701105 18835 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.615798 18840 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.702008 13948 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:13.616907 18785 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.607510 18781 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:13.703629 18781 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:13.565994 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 73267.7us
I20260709 13:06:13.690837 18898 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:13.707603 13988 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:13.708743 18898 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.708877 13988 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:13.617197 13950 raft_consensus.cc:2410] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:06:13.711212 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.631017 13942 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:13.637645 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.645713 18492 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:13.712446 18832 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:13.713804 18788 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:13.714241 13866 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:13.714859 18788 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.715960 18788 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:13.717078 18744 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.717360 18932 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:13.638821 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.717638 18744 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.718154 18932 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:13.640653 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.719105 13870 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:13.720283 18781 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:13.720449 18744 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.721765 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:13.722884 18716 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:13.725417 18803 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:13.677518 18930 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.727397 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2158) took 1360 ms. Trace:
I20260709 13:06:13.729303 18930 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.607098 18896 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:13.652814 13868 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:13.653396 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.654275 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:13.744750 13872 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:13.745194 13872 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:13.630342 18882 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.675639 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:13.678581 14561 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:13.678278 14015 rpcz_store.cc:276] 0709 13:06:12.406565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.831268 (+424703us) service_pool.cc:224] Handling call
0709 13:06:13.582694 (+751426us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54001220d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:13.582843 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":980832000}
I20260709 13:06:13.640013 18901 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:13.665741 18928 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.749024 18896 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:13.686352 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:13.682999 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:13.678066 18889 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.005s	sys 0.012s
I20260709 13:06:13.727862 14022 rpcz_store.cc:276] 0709 13:06:12.366849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.368587 (+  1738us) service_pool.cc:224] Handling call
0709 13:06:12.383436 (+ 14849us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:12.551089 (+167653us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:12.585706 (+ 34617us) log.cc:844] Serialized 14 byte log entry
0709 13:06:13.186561 (+600855us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:13.236068 (+ 49507us) log.cc:844] Serialized 26 byte log entry
0709 13:06:13.236266 (+   198us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:13.524695 (+288429us) log.cc:844] Serialized 14 byte log entry
0709 13:06:13.570422 (+ 45727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.570538 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:13.727130 (+156592us) raft_consensus.cc:1685] finished
0709 13:06:13.727156 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.727322 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52218,"prepare.queue_time_us":128600,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":98,"raft.queue_time_us":168204,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":5368,"spinlock_wait_cycles":1998592,"thread_start_us":891037,"threads_started":3,"wal-append.queue_time_us":771418}
I20260709 13:06:13.738458 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:13.755386 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 69147us
W20260709 13:06:13.572610 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 51145.9us
I20260709 13:06:13.743855 13867 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:13.756484 14524 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:13.757272 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 81787.6us
I20260709 13:06:13.759416 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:13.759779 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.760442 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.761003 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.748606 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:13.750191 18832 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:13.764187 13949 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 12: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:13.769964 13840 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:13.682639 18777 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:13.781199 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:13.783105 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:13.788347 13871 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:13.782238 13838 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:13.785912 18820 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:06:13.789881 18852 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:13.790917 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:13.787086 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:13.743603 18405 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:13.794867 13944 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:13.780944 18818 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:13.806366 18818 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:13.806731 18818 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:13.808445 18820 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:13.777295 18791 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:13.765024 18930 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.812055 18882 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.811884 18930 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:13.813336 18777 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:13.814813 18777 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:13.815876 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:13.816579 13942 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:13.764941 18927 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:13.784363 13872 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:13.794162 18898 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.004s
W20260709 13:06:13.800364 18913 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.143s	user 0.002s	sys 0.000s
I20260709 13:06:13.786198 18781 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:13.804687 18797 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.780433 18831 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:13.748714 18695 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:13.819823 13872 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:13.819103 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:13.824826 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:13.833213 14561 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:06:13.833765 13947 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 12: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:13.826705 14558 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:13.831525 13949 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:13.833971 13838 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:13.843895 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.844889 13947 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:13.846257 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.854619 13871 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
W20260709 13:06:13.894543 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.825361 18924 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.005s	sys 0.019s
I20260709 13:06:13.910998 13947 raft_consensus.cc:2410] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:06:13.911204 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 76309.2us
I20260709 13:06:13.911424 13944 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:13.931643 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:13.932330 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 98438.3us
I20260709 13:06:13.824405 18852 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:13.938251 18887 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:13.826313 18831 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:13.939388 18831 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:13.940806 18887 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:13.910386 18492 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:13.942478 18492 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.942860 18492 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:13.932793 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 145907us
I20260709 13:06:13.943774 13947 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:13.822455 18744 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:13.947697 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:13.938699 18852 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.824632 18791 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:13.951820 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.951992 18112 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:06:13.824536 18781 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:13.964695 13872 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:06:13.951388 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:13.972239 13949 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:13.982569 18112 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.983188 18112 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:13.949258 18924 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.005s	sys 0.019s
I20260709 13:06:13.951661 18785 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:13.989523 13839 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:13.958130 18852 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:13.998960 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 273982us
I20260709 13:06:13.993228 13942 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:13.975739 18781 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:14.010545 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:14.011647 14559 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:13.995597 18785 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.025051 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:14.026286 13942 raft_consensus.cc:2410] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:06:13.975567 18791 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:13.973362 18940 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:14.006779 18853 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:14.032838 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.041044 18112 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.030274 18785 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:14.070711 14018 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:06:14.049151 18853 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:14.072674 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.075726 14022 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:14.074077 14016 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:14.048985 18916 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:14.080354 18916 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.082695 18916 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:14.088657 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.091550 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.092424 14018 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:14.093341 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.095515 14022 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:14.108076 18791 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.114044 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:14.121376 14024 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:06:14.121871 14024 pending_rounds.cc:85] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 6
I20260709 13:06:14.122131 14024 pending_rounds.cc:107] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:06:14.122365 14024 raft_consensus.cc:2889] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:14.122406 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.123513 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.124203 14020 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:14.128487 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.128686 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:14.128783 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.129236 14018 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:14.130116 14022 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:14.135296 14017 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:14.137324 14019 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:06:14.139775 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:14.140395 14021 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:14.142295 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.142917 14020 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:14.143708 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:14.144228 14020 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:14.148026 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:14.148799 14023 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:14.147285 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:14.150272 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:14.152338 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.152966 14024 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:14.120478 18786 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:14.165737 14019 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:14.151448 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:14.163885 14022 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:06:14.172632 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.174067 14018 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:14.127255 18405 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:14.167196 18788 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:14.167483 18876 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:14.181591 18488 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.166182 18797 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:06:14.185580 18749 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:14.219820 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:14.223189 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.166455 18920 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.201689 18788 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:14.226897 18788 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:14.227754 18874 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:14.228355 18874 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:14.212525 18488 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:14.230952 18751 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:14.232453 18488 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:14.232849 18953 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.165s	user 0.001s	sys 0.001s
I20260709 13:06:14.202317 18889 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:06:14.222573 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:14.242681 18955 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:06:14.242712 18793 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:14.243471 18793 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:14.244590 18946 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:06:14.244405 18793 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.246403 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:14.251701 14558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 829) took 1926 ms (client timeout 1916 ms). Trace:
I20260709 13:06:14.179456 18786 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.260497 14522 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:14.214668 18947 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:06:14.267653 14522 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:14.252384 14558 rpcz_store.cc:269] 0709 13:06:12.285613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.289112 (+  3499us) service_pool.cc:224] Handling call
0709 13:06:14.211575 (+1922463us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540018c9d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:14.211715 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3358880512}
I20260709 13:06:14.201067 18876 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.186437 18405 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.269800 13838 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:14.209959 18749 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:14.274616 14016 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:14.212670 18940 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:14.223858 14018 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:06:14.298871 18831 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.270983 18720 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.304965 18720 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.306006 18831 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.267290 18894 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:14.292496 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:14.315171 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.358428 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.359266 14018 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:14.297762 18840 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:14.359724 18405 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.360203 18840 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:14.360387 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:14.360716 18840 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:14.361014 14018 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:14.362143 18853 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.275220 18919 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:14.363377 18919 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.275511 18876 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:14.366235 18559 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:14.366712 18559 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:14.367102 14021 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:06:14.367192 18559 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:14.368201 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:14.368816 14021 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:14.301478 18749 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:14.277901 18820 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:14.275002 18887 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:14.359417 18927 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.541s	user 0.006s	sys 0.003s
I20260709 13:06:14.372650 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:14.373347 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58953.7us
I20260709 13:06:14.371846 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:14.369822 18896 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.615s	user 0.012s	sys 0.000s
I20260709 13:06:14.383550 14019 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:14.384887 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:14.398172 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.398958 14020 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:06:14.386126 14018 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:14.414032 14017 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:14.365876 18919 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:14.273980 18788 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:14.388535 18818 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.002s
I20260709 13:06:14.310178 18744 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:06:14.389014 18803 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.663s	user 0.005s	sys 0.006s
W20260709 13:06:14.373349 18695 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.531s	user 0.004s	sys 0.006s
I20260709 13:06:14.387672 18927 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:14.391259 18930 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:14.443024 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.415763 18820 raft_consensus.cc:697] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:14.433205 14021 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:14.456976 13867 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:06:14.466965 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.468230 14015 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 12: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:14.461905 18803 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:14.462455 18882 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.403409 18896 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:14.458830 14018 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
W20260709 13:06:14.509341 18492 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.005s
I20260709 13:06:14.457801 18744 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.454084 18818 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:14.455090 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.524616 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:14.571369 13945 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:06:14.533044 13874 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:14.573236 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:14.536180 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:14.575242 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:14.518018 14019 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:14.576313 14015 raft_consensus.cc:2410] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:06:14.542867 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:14.603662 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.518893 18803 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:14.523531 18901 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.567368 18920 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:14.604867 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 358525us
I20260709 13:06:14.612721 13947 pending_rounds.cc:85] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 6
I20260709 13:06:14.632472 13947 pending_rounds.cc:107] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:06:14.632903 13947 raft_consensus.cc:2889] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:14.615286 13950 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:14.567173 18786 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.627395 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:14.570940 18808 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.628994 14558 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:14.629069 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:14.576890 18916 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:14.641003 13941 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:06:14.604544 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:14.645817 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:14.646224 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.518694 18882 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:14.569875 18492 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.605559 18811 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.626573 18919 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:14.658589 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:14.643267 18893 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.648258 18920 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:14.652597 18786 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.681957 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:14.653517 18525 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.632262 18832 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:06:14.645128 18713 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:06:14.705430 14015 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:14.567871 18820 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:14.661155 18874 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:14.708847 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 336272us
I20260709 13:06:14.650241 18808 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.726616 13838 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:14.682889 18788 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.739127 13838 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:14.672490 18811 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:14.760016 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:14.761176 13950 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
W20260709 13:06:14.763664 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 350118us
I20260709 13:06:14.766247 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:14.675793 18893 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.770910 13950 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
W20260709 13:06:14.760118 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 128479us
W20260709 13:06:14.754357 18793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.009s	sys 0.006s
I20260709 13:06:14.777944 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:14.783449 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:14.746199 18488 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.696771 18786 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.745903 18894 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:14.753376 18405 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:14.788537 13948 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:06:14.768569 18896 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:14.791745 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 139559us
I20260709 13:06:14.790664 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:14.783524 18793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.010s	sys 0.007s
I20260709 13:06:14.743361 18916 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:06:14.776607 13838 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:14.803719 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:14.814622 14558 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:14.776598 18890 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:14.802083 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:14.808917 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 181608us
I20260709 13:06:14.816546 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:14.780074 13840 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:14.817798 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:14.821959 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:14.796393 18894 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:14.832819 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 98321.4us
I20260709 13:06:14.833719 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:14.834468 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:14.754069 18803 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:14.838213 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 61696.6us
W20260709 13:06:14.840310 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 60389.8us
I20260709 13:06:14.841138 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:14.817250 18818 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:14.820492 18744 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.820453 18916 consensus_queue.cc:1243] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:06:14.867290 18965 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:06:14.831537 18788 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:14.798784 18488 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:14.854298 18964 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
W20260709 13:06:14.855060 18963 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:06:14.856740 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:14.857326 18973 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:14.844332 18889 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:14.819905 18525 consensus_queue.cc:1059] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:06:14.859647 18893 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.849349 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:14.835479 18894 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:14.829123 18405 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:14.825357 18874 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:06:14.858021 18966 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:14.874585 18896 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:14.895985 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:14.896912 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:14.898849 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:14.901559 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52304.1us
W20260709 13:06:14.904345 18781 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.874s	user 0.009s	sys 0.005s
W20260709 13:06:14.893641 18749 consensus_meta.cc:287] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.519s	user 0.012s	sys 0.002s
I20260709 13:06:14.894119 18492 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.918213 14558 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:14.885510 18788 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.917766 18966 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:14.933977 18966 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:14.934027 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:14.935176 18808 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:14.935515 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:14.936671 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:14.917930 18896 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:14.920585 18559 consensus_meta.cc:287] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.553s	user 0.015s	sys 0.000s
I20260709 13:06:14.902318 18811 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:14.909265 18793 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:14.938935 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 829) took 1219 ms. Trace:
I20260709 13:06:14.938867 18559 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.939385 18793 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.939882 18793 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:14.939507 13943 rpcz_store.cc:276] 0709 13:06:13.719901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.952436 (+232535us) service_pool.cc:224] Handling call
0709 13:06:14.118295 (+165859us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:14.118691 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:14.118723 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:14.127445 (+  8722us) log.cc:844] Serialized 26 byte log entry
0709 13:06:14.440008 (+312563us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:14.440072 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:14.440169 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:14.938697 (+498528us) raft_consensus.cc:1685] finished
0709 13:06:14.938723 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:14.938875 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":4847,"thread_start_us":266069,"threads_started":1,"wal-append.queue_time_us":492929}
I20260709 13:06:14.941160 18559 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:14.918004 18916 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:14.924346 18893 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:14.901096 18744 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:14.954653 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:14.894164 18889 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.920347 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:14.844704 18876 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.957521 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:14.906814 18988 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:14.927536 13840 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:14.929689 13914 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:14.938294 18896 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:14.923399 18818 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:14.902817 18969 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:14.943010 18788 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:14.887974 18980 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:06:14.912722 18982 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:14.968680 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.002136 13869 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:15.003794 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.004519 13869 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:15.005364 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:14.969909 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:14.972297 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.006412 13945 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:15.005918 13869 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:06:15.009356 18890 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:14.978556 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 48962.3us
W20260709 13:06:14.979467 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 145872us
I20260709 13:06:14.981976 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:14.942811 14016 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
W20260709 13:06:14.983373 13915 connection.cc:256] RPC call timeout handler was delayed by -1.2396613720000005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1656s for a response
I20260709 13:06:14.938877 18786 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:14.932499 18781 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:14.996193 13868 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:06:14.996978 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 69706.6us
I20260709 13:06:14.932744 18749 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:14.974427 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:15.037739 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.976179 18811 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:06:15.103634 18990 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:06:15.035601 18949 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.010720 18405 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.105635 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:15.106859 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:15.107668 13947 raft_consensus.cc:2410] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:06:14.981706 18525 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:15.108096 18525 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.108467 18525 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:15.110386 14559 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:15.011849 14555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 669) took 1008 ms. Trace:
I20260709 13:06:14.977252 13943 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:15.114475 13945 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:15.115062 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:15.115608 18749 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.116657 13943 pending_rounds.cc:85] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 7
I20260709 13:06:15.116951 13943 pending_rounds.cc:107] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:06:15.117194 13943 raft_consensus.cc:2889] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:14.995378 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:14.981946 18970 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:14.961622 18889 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:14.980085 18893 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:15.119670 18893 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:15.125322 18840 consensus_meta.cc:287] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.764s	user 0.008s	sys 0.004s
I20260709 13:06:15.127063 18525 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:15.037151 13915 connection.cc:256] RPC call timeout handler was delayed by -1.085761371999979s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
I20260709 13:06:15.129362 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:15.084793 13875 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:15.051856 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.052501 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.130733 14018 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:15.130892 18488 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:15.131626 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.132731 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.133409 14018 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:15.133828 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.134325 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.135076 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.135529 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.135699 13867 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:06:15.136644 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:15.137661 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.139298 18811 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:15.140367 18811 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.140969 18492 consensus_queue.cc:1059] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:15.141412 13944 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:15.142620 18492 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:15.142992 18492 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:15.143833 18970 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:15.145465 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:15.139093 14022 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:06:15.147797 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:15.148649 14558 raft_consensus.cc:2376] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11. Re-sending same reply.
I20260709 13:06:15.136646 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:15.151744 13867 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:15.152602 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:15.066345 13869 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:15.065193 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:15.064924 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.001586 18982 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.103199 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.116132 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:14.960706 18744 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.107494 18890 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:15.086762 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 269002us
I20260709 13:06:15.158813 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:15.057227 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:15.157768 13866 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:15.114099 14555 rpcz_store.cc:276] 0709 13:06:14.003945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.004168 (+   223us) service_pool.cc:224] Handling call
0709 13:06:14.129868 (+125700us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:14.130324 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:14.130354 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:14.164587 (+ 34233us) log.cc:844] Serialized 52 byte log entry
0709 13:06:14.438732 (+274145us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:14.438780 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:14.438872 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:15.011600 (+572728us) raft_consensus.cc:1685] finished
0709 13:06:15.011623 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.011785 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14346,"raft.queue_time_us":6098,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":7875,"thread_start_us":260929,"threads_started":1,"wal-append.queue_time_us":466634}
I20260709 13:06:15.155067 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.146126 13949 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:15.127352 18889 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.160240 18744 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:15.250180 18788 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.250428 18966 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.252391 18788 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.252617 18966 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.210405 18722 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:15.253314 18722 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.169068 18931 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:15.254518 18931 raft_consensus.cc:3055] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:15.254318 18894 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.254942 18931 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:15.255553 18722 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:15.255856 18931 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.256467 18894 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.160262 18982 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:15.144809 18970 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.104553 18949 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:15.233201 18785 consensus_meta.cc:287] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.164s	user 0.008s	sys 0.012s
I20260709 13:06:15.231227 18786 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.277360 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:15.164943 14020 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:15.167219 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.001737 18969 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.166218 18991 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:15.011186 18897 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:06:15.175704 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 46423.6us
I20260709 13:06:15.163106 13869 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:15.296063 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.296737 13869 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:06:15.059363 18999 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.067286 18876 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:15.233933 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.297848 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.233249 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:15.215003 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:15.125860 18840 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.143865 18492 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.247148 13988 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:15.247392 18525 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:15.247666 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.178906 18890 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.248471 18916 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.161334 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:15.266695 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 117724us
I20260709 13:06:15.162917 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:15.284446 13866 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:15.179373 18919 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:15.288993 14555 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:06:15.166164 18852 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.154s	user 0.012s	sys 0.009s
I20260709 13:06:15.292320 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:15.328661 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:15.328822 18852 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.294605 14561 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:06:15.329411 13942 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:15.173417 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.331069 18852 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.331459 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.331941 18916 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.167397 18801 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.116s
I20260709 13:06:15.299467 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:15.338950 18983 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:15.339537 18983 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:15.340495 18983 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.300240 13944 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:15.346691 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2757) took 1227 ms. Trace:
I20260709 13:06:15.347332 14017 rpcz_store.cc:276] 0709 13:06:14.119703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.149351 (+ 29648us) service_pool.cc:224] Handling call
0709 13:06:14.399846 (+250495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:14.431947 (+ 32101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.346385 (+914438us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b5400020dd0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:15.346599 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503895424}
I20260709 13:06:15.349196 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:15.269596 18989 consensus_queue.cc:1059] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:06:15.350673 18989 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:15.351102 18989 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:15.349961 14017 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:15.352059 18989 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.353507 18989 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:15.354924 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 192071us
I20260709 13:06:15.301146 18897 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:15.355850 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:15.317242 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.356542 13866 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:15.356899 14024 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
W20260709 13:06:15.358120 18948 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.231s	user 0.003s	sys 0.009s
I20260709 13:06:15.282020 18996 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:15.356416 18781 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.356374 18897 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:15.319911 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 72866.7us
I20260709 13:06:15.319072 18492 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:15.324344 18559 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:15.324985 18919 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.326306 13871 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:15.276003 18785 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:15.352463 13874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 697) took 1680 ms (client timeout 1633 ms). Trace:
I20260709 13:06:15.355000 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2185) took 1027 ms. Trace:
I20260709 13:06:15.331640 14559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 918) took 1097 ms. Trace:
I20260709 13:06:15.298486 13869 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:15.264438 18744 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:15.429862 13874 rpcz_store.cc:269] 0709 13:06:13.671711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.674247 (+  2536us) service_pool.cc:224] Handling call
0709 13:06:15.352216 (+1677969us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54000ea454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:15.352383 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1786519808}
I20260709 13:06:15.447276 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.448379 13874 raft_consensus.cc:2368] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13.
I20260709 13:06:15.309217 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:15.431348 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.451663 13869 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:06:15.416056 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.281440 18786 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.430613 14559 rpcz_store.cc:276] 0709 13:06:14.234470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.234643 (+   173us) service_pool.cc:224] Handling call
0709 13:06:14.235003 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.110855 (+875852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.331357 (+220502us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b540040a6d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:15.331541 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2012678528}
I20260709 13:06:15.429975 14019 rpcz_store.cc:276] 0709 13:06:14.327961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.709540 (+381579us) service_pool.cc:224] Handling call
0709 13:06:14.710043 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:14.912248 (+202205us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:14.912356 (+   108us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:14.922443 (+ 10087us) log.cc:844] Serialized 26 byte log entry
0709 13:06:15.175145 (+252702us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:15.175199 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.175293 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:15.354722 (+179429us) raft_consensus.cc:1685] finished
0709 13:06:15.354757 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.354927 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32373,"raft.queue_time_us":164438,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":13601,"thread_start_us":342921,"threads_started":2,"wal-append.queue_time_us":356832}
I20260709 13:06:15.421969 18559 raft_consensus.cc:697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:15.435891 18525 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.463518 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.430310 18781 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:15.294421 18488 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:06:15.482021 18995 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:15.464286 18781 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.435844 18785 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.442477 18720 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:15.486215 18995 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:15.497606 18995 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:15.463589 18525 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.486685 18788 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:15.497344 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.342013 18983 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:15.515347 14022 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:15.518635 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.423264 18933 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:15.526831 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:15.528821 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:15.531402 13873 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:15.333161 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:15.357676 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:15.554910 13866 raft_consensus.cc:2410] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:06:15.555316 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:15.555621 13867 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:15.557302 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:15.558372 13950 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:15.558660 13946 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 13:06:15.557377 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 248275us
I20260709 13:06:15.561394 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:15.565754 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:15.562266 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 400965us
W20260709 13:06:15.567518 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.568606 13989 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:15.525951 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 202487us
I20260709 13:06:15.557420 13866 pending_rounds.cc:85] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 4
I20260709 13:06:15.619382 13866 pending_rounds.cc:107] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:06:15.619809 13866 raft_consensus.cc:2889] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:15.620636 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:15.624740 13867 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:15.499040 18781 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:15.629350 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 60839us
I20260709 13:06:15.311286 18999 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.653391 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:15.635136 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.654762 14019 raft_consensus.cc:2410] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:06:15.644641 13942 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
W20260709 13:06:15.632975 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 71665.3us
I20260709 13:06:15.632006 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:15.657979 14559 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:15.658203 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:15.463915 18559 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.670710 14020 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:06:15.687935 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.688153 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:15.662931 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:15.661846 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 378282us
I20260709 13:06:15.622931 18920 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:15.702025 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2184) took 1374 ms. Trace:
I20260709 13:06:15.702731 14015 rpcz_store.cc:276] 0709 13:06:14.327634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.644298 (+316664us) service_pool.cc:224] Handling call
0709 13:06:14.662713 (+ 18415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:14.705961 (+ 43248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.701660 (+995699us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b5400411754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:15.701929 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77817216}
I20260709 13:06:15.711889 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:15.713552 14022 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:06:15.713608 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:15.692910 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:15.718472 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:15.652794 18876 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:15.661212 18908 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:15.662200 18933 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:15.696578 18811 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.008s	sys 0.035s
I20260709 13:06:15.743791 14559 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
W20260709 13:06:15.749785 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 86966.8us
I20260709 13:06:15.758073 13989 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:15.645900 18949 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.694358 18852 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:15.666271 18840 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.689697 18988 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:15.710743 18981 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:15.739637 19003 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:15.694679 18785 consensus_queue.cc:1059] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:06:15.743600 18898 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:15.749166 18887 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:06:15.769833 18793 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.829s	user 0.017s	sys 0.004s
I20260709 13:06:15.798736 18887 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.799243 18887 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:15.798981 18793 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.801333 18793 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.727523 18788 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:15.744230 18818 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:15.802657 18818 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:15.803071 18818 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:15.757503 18811 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.008s	sys 0.035s
I20260709 13:06:15.739832 18749 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:06:15.808262 18785 raft_consensus.cc:3055] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:15.808722 18785 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:15.755309 18777 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:15.755424 18908 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.815039 18908 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:15.774155 18916 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:15.817852 13941 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:15.711333 18831 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:06:15.754765 18970 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:15.818823 18831 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:06:15.780978 18896 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.784s	user 0.014s	sys 0.002s
I20260709 13:06:15.819416 18831 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:15.819710 18896 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.820354 18831 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.821664 18831 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:15.822316 13870 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:15.773021 18920 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:15.823361 18982 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.003s
I20260709 13:06:15.823323 18920 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.824060 18920 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:15.824007 18982 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.824484 18898 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:15.789465 18999 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:15.825497 18898 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.790616 18882 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:15.759063 18893 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:15.774099 18988 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.827140 18893 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.765425 18897 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:06:15.761274 18933 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.818749 18970 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:15.779212 18525 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:15.816362 19003 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.770910 18969 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:15.771399 18981 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:15.840389 18970 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:15.771909 18890 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.842300 14017 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:15.769454 18852 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:15.844780 18933 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:15.840847 18896 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.855969 18982 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:15.816100 18744 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.876065 13914 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:15.836000 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2180) took 1509 ms. Trace:
I20260709 13:06:16.044691 14016 rpcz_store.cc:276] 0709 13:06:14.326377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.560619 (+234242us) service_pool.cc:224] Handling call
0709 13:06:14.561077 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.031492 (+470415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.835596 (+804104us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54002990d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:15.835909 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":877364608}
I20260709 13:06:15.816075 18786 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:15.893700 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2351) took 1032 ms. Trace:
I20260709 13:06:16.048743 13941 rpcz_store.cc:276] 0709 13:06:14.839836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.898601 (+ 58765us) service_pool.cc:224] Handling call
0709 13:06:15.871640 (+973039us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b5400229a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:15.871898 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2061124992}
W20260709 13:06:15.976114 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.809645 18785 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:15.918902 18889 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:16.060896 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:16.035334 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.041963 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:15.905841 18981 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:16.039497 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:16.057463 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 181517us
I20260709 13:06:16.039302 18896 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.028237 18988 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:16.043977 18982 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:16.066974 18989 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.707s	user 0.020s	sys 0.005s
I20260709 13:06:16.065332 18559 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:16.066395 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 373488us
W20260709 13:06:16.083567 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 337785us
I20260709 13:06:16.098480 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:16.085114 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 371574us
I20260709 13:06:16.092749 13839 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:16.065102 18525 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:16.100593 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:16.104269 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:16.100687 13839 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:16.105751 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:16.102965 13838 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:16.105911 18853 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:06:16.107797 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 349822us
I20260709 13:06:16.110934 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:16.113656 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:16.119498 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:16.121711 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.122855 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:16.106698 18987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.001s	user 0.022s	sys 0.018s
I20260709 13:06:16.146124 13989 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:16.142697 18995 consensus_meta.cc:287] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.645s	user 0.018s	sys 0.012s
I20260709 13:06:16.156354 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:16.110576 18811 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:16.116513 18853 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.172647 13989 leader_election.cc:400] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13.
I20260709 13:06:16.173027 13989 leader_election.cc:403] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:16.145313 18987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.039s	user 0.022s	sys 0.018s
I20260709 13:06:16.175889 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:16.084319 18896 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.097810 18982 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.150126 18999 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:16.190105 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:16.144824 18876 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:16.198283 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:16.106815 18559 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:16.149645 18793 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:16.154158 18893 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:16.156702 18989 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:16.202497 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:16.203022 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:16.193432 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:16.156742 18525 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:16.193212 18988 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:06:16.251411 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:16.157068 18969 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:06:16.187332 18844 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.158103 19014 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:16.174998 18788 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:16.178143 18853 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:16.246899 18983 consensus_meta.cc:287] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.721s	user 0.013s	sys 0.000s
I20260709 13:06:16.168619 18811 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:16.252507 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:16.253141 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:16.170465 18980 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:06:16.207641 18405 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:16.248621 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:16.197252 18999 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:16.246304 18793 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:16.260941 14017 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:16.265731 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:16.405081 14019 raft_consensus.cc:2393] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:16.405265 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:16.406764 14017 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 12: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:16.408185 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:16.408960 14017 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:16.266940 14023 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:16.200145 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:16.413322 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 213134us
I20260709 13:06:16.414306 14558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 841) took 1251 ms. Trace:
I20260709 13:06:16.414947 14558 rpcz_store.cc:276] 0709 13:06:15.162680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.162963 (+   283us) service_pool.cc:224] Handling call
0709 13:06:15.163371 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:15.163678 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:15.163714 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:15.164325 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 13:06:15.690393 (+526068us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:15.690432 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.690507 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:16.413447 (+722940us) raft_consensus.cc:1685] finished
0709 13:06:16.413493 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.413909 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":414632,"threads_started":1,"wal-append.queue_time_us":1013444}
I20260709 13:06:16.389633 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2846) took 1275 ms. Trace:
I20260709 13:06:16.390409 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2841) took 1278 ms. Trace:
I20260709 13:06:16.209657 18720 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:16.431248 18720 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:16.433126 18916 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:16.435262 18896 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:16.435418 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:16.434698 18525 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:16.436399 14019 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:16.418574 13867 rpcz_store.cc:276] 0709 13:06:15.114117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.652314 (+538197us) service_pool.cc:224] Handling call
0709 13:06:15.652769 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:16.052534 (+399765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:16.052588 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:16.082803 (+ 30215us) log.cc:844] Serialized 26 byte log entry
0709 13:06:16.388143 (+305340us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:16.388348 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.388586 (+   238us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:16.388735 (+   149us) raft_consensus.cc:1685] finished
0709 13:06:16.388778 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.389242 (+   464us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46661,"raft.queue_time_us":86637,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":12362,"thread_start_us":584800,"threads_started":2,"wal-append.queue_time_us":223321}
I20260709 13:06:16.324018 18852 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:16.309831 18793 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:16.311776 18781 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.640s	user 0.018s	sys 0.014s
I20260709 13:06:16.263162 18844 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.464656 18844 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:16.262178 18811 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:16.335181 18982 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:16.324654 18893 raft_consensus.cc:2955] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 LEADER]: Committing config change with OpId 14.13: config changed from index 7 to 13, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:16.470570 18781 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:16.388262 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 285018us
I20260709 13:06:16.474377 19018 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.475560 19018 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.477766 14553 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:16.400542 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:16.206252 18991 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:16.169286 18995 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:16.481105 13875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2887) took 1513 ms (client timeout 1759 ms). Trace:
I20260709 13:06:16.309384 19003 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:16.201244 18876 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:06:16.334280 19012 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.268s	user 0.004s	sys 0.006s
I20260709 13:06:16.482923 18876 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:16.483603 13873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2890) took 1510 ms (client timeout 1681 ms). Trace:
W20260709 13:06:16.327591 19017 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.179s	user 0.002s	sys 0.000s
W20260709 13:06:16.328043 19020 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:06:16.486162 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2885) took 1519 ms. Trace:
I20260709 13:06:16.327673 19023 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:16.486814 13872 rpcz_store.cc:276] 0709 13:06:14.966825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.979838 (+ 13013us) service_pool.cc:224] Handling call
0709 13:06:16.485567 (+1505729us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54002e68d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:16.485874 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1660089600}
I20260709 13:06:16.487686 19023 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:16.489351 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:16.328063 19028 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:06:16.502816 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:16.503008 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2190) took 1255 ms. Trace:
I20260709 13:06:16.504310 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 568) took 1258 ms. Trace:
I20260709 13:06:16.504994 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:16.504928 14022 rpcz_store.cc:276] 0709 13:06:15.245489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.539967 (+294478us) service_pool.cc:224] Handling call
0709 13:06:15.540288 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.714139 (+173851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.503657 (+789518us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54000ac154. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:16.504062 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":397271808}
I20260709 13:06:16.271044 18999 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:16.401254 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:16.418697 13866 rpcz_store.cc:276] 0709 13:06:15.111579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.556589 (+445010us) service_pool.cc:224] Handling call
0709 13:06:15.557108 (+   519us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:06:15.740464 (+183356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:15.740500 (+    36us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:06:15.741297 (+   797us) log.cc:844] Serialized 104 byte log entry
0709 13:06:16.069540 (+328243us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:16.069593 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.069675 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:16.389588 (+319913us) raft_consensus.cc:1685] finished
0709 13:06:16.389653 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.390156 (+   503us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113130,"raft.queue_time_us":112982,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":8411,"spinlock_wait_cycles":1019264,"thread_start_us":300597,"threads_started":1,"wal-append.queue_time_us":570755}
W20260709 13:06:16.437953 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 289369us
I20260709 13:06:16.508358 13866 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:16.509181 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2847) took 1393 ms. Trace:
I20260709 13:06:16.260941 14015 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:16.510788 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:16.509578 13866 rpcz_store.cc:276] 0709 13:06:15.115923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.507380 (+1391457us) service_pool.cc:224] Handling call
0709 13:06:16.507949 (+   569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.508861 (+   912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.509079 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:16.209481 18949 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:16.512710 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 566) took 1267 ms. Trace:
I20260709 13:06:16.513165 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2848) took 1396 ms. Trace:
I20260709 13:06:16.513679 13866 rpcz_store.cc:276] 0709 13:06:15.116837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.511837 (+1395000us) service_pool.cc:224] Handling call
0709 13:06:16.512410 (+   573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.512777 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:16.512806 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:16.512870 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:16.512909 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.512972 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.513106 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:16.515401 18995 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:16.517544 19003 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.504329 13944 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:16.453796 18980 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:16.454706 14554 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:16.456488 18981 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.457207 18720 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:16.464735 18559 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:16.222222 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:16.481644 13875 rpcz_store.cc:269] 0709 13:06:14.967670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.969535 (+  1865us) service_pool.cc:224] Handling call
0709 13:06:16.480589 (+1511054us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b54000da554. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:16.480828 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113735936}
W20260709 13:06:16.484184 13873 rpcz_store.cc:269] 0709 13:06:14.973344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.973519 (+   175us) service_pool.cc:224] Handling call
0709 13:06:16.483069 (+1509550us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54000a8054. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:16.483404 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":938776448}
I20260709 13:06:16.260962 19014 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.553027 19014 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:16.328506 19031 raft_consensus.cc:2764] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:06:16.556820 18949 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:16.557675 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 364192us
I20260709 13:06:16.486626 14018 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:16.489527 19023 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.498041 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2826) took 1212 ms. Trace:
I20260709 13:06:16.502950 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:16.339918 18988 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13.
I20260709 13:06:16.447564 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:16.583377 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 715) took 1407 ms. Trace:
I20260709 13:06:16.339692 18970 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.584024 13875 rpcz_store.cc:276] 0709 13:06:15.175707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.566015 (+1390308us) service_pool.cc:224] Handling call
0709 13:06:16.582450 (+ 16435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.582926 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:16.582964 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:16.583064 (+   100us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:16.583099 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.583154 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.583285 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:16.586882 18970 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:16.521514 18980 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.587826 18980 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:16.523470 19003 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:16.527884 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 127323us
I20260709 13:06:16.593791 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:16.594188 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:16.596063 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:16.599332 18929 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:16.599820 18929 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.599908 18840 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:16.600231 18929 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:16.600509 18840 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:16.602133 18874 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:16.602857 18874 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:16.604287 18874 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:16.528828 18405 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:16.530365 19032 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:16.531191 19024 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.530153 13839 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:16.607223 19024 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.532203 19033 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.609092 18894 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:16.609668 19033 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:16.610538 18781 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:16.504253 14020 rpcz_store.cc:276] 0709 13:06:15.246863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.693660 (+446797us) service_pool.cc:224] Handling call
0709 13:06:15.694006 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.762495 (+ 68489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:15.762549 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:15.762615 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:15.762675 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.502395 (+739720us) spinlock_profiling.cc:243] Waited 68.2 ms on lock 0x7b54003250d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:16.502493 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.502726 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156801408}
I20260709 13:06:16.557855 18949 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.513190 14024 rpcz_store.cc:276] 0709 13:06:15.244980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.247390 (+  2410us) service_pool.cc:224] Handling call
0709 13:06:16.512177 (+1264787us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b54003255d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:16.512450 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94191872}
I20260709 13:06:16.614244 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:16.558208 13869 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:06:16.615916 18931 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:16.558903 13949 rpcz_store.cc:276] 0709 13:06:15.286897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.287131 (+   234us) service_pool.cc:224] Handling call
0709 13:06:15.313655 (+ 26524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:15.313996 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:15.314037 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:15.431612 (+117575us) log.cc:844] Serialized 26 byte log entry
0709 13:06:15.744451 (+312839us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:15.744488 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.744550 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:16.497369 (+752819us) raft_consensus.cc:1685] finished
0709 13:06:16.497409 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.497728 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":146392,"threads_started":1,"wal-append.queue_time_us":497899}
I20260709 13:06:16.577281 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:16.577849 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 713) took 1419 ms. Trace:
I20260709 13:06:16.601933 19022 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:16.612159 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:16.558946 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:16.624691 13950 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:16.558018 18981 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:16.624922 13873 rpcz_store.cc:276] 0709 13:06:15.158393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.563397 (+1405004us) service_pool.cc:224] Handling call
0709 13:06:16.573495 (+ 10098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.577430 (+  3935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:16.577458 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:16.577529 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:16.577563 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.577633 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.577761 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:16.618510 19003 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:16.630594 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 100728us
I20260709 13:06:16.652299 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:16.653553 14017 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:16.658587 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:16.658865 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 241323us
I20260709 13:06:16.697417 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:16.702426 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:16.658017 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:16.659179 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:16.672823 13945 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:06:16.665510 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.763866 14019 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:16.768467 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 712) took 1621 ms. Trace:
I20260709 13:06:16.771337 14018 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:16.772446 13943 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
W20260709 13:06:16.782425 18831 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.952s	user 0.013s	sys 0.008s
W20260709 13:06:16.782871 19043 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 13:06:16.783645 18820 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.491s	user 0.022s	sys 0.020s
W20260709 13:06:16.784055 18820 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.651s	user 0.022s	sys 0.020s
I20260709 13:06:16.659370 18996 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:16.785152 18996 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:16.786224 14021 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:16.786945 19024 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:16.790834 13944 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:06:16.791791 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:16.660320 18919 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:16.699424 19018 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:16.799750 13871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2889) took 1827 ms (client timeout 1801 ms). Trace:
I20260709 13:06:16.880693 18781 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:06:16.881139 13871 rpcz_store.cc:269] 0709 13:06:14.971748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.971928 (+   180us) service_pool.cc:224] Handling call
0709 13:06:16.799145 (+1827217us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54002e81d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:16.799476 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734311168}
I20260709 13:06:16.881525 18781 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:16.882812 19040 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.808751 19033 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:16.799598 18820 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:16.883666 19040 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:16.884311 18820 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:16.812074 18818 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.008s	user 0.012s	sys 0.011s
I20260709 13:06:16.830365 13949 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
W20260709 13:06:16.838080 18887 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.038s	user 0.022s	sys 0.001s
W20260709 13:06:16.846123 18492 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.425s	user 0.020s	sys 0.011s
I20260709 13:06:16.846664 18973 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:16.667687 18840 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:16.853698 13947 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:16.862493 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:16.863543 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 89281.6us
I20260709 13:06:16.790616 18916 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:16.659384 18832 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.152s
I20260709 13:06:16.872394 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:16.852326 18876 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:16.877996 18853 consensus_meta.cc:287] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.608s	user 0.012s	sys 0.003s
I20260709 13:06:16.799914 13874 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:16.882786 18919 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:16.823113 18793 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.761930 18949 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:16.907379 13950 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:13.327525 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:16.914322 18920 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.089s	user 0.018s	sys 0.002s
W20260709 13:06:16.922155 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 444357us
I20260709 13:06:16.922546 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2836) took 1815 ms. Trace:
I20260709 13:06:16.922943 18820 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:16.924645 18786 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
W20260709 13:06:16.929296 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 331269us
I20260709 13:06:16.930456 13869 rpcz_store.cc:276] 0709 13:06:15.107528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.452393 (+344865us) service_pool.cc:224] Handling call
0709 13:06:16.534742 (+1082349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:16.615841 (+ 81099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.922313 (+306472us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000da550. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:16.922463 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2385552256}
I20260709 13:06:16.939417 13943 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:16.945704 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:16.946864 18876 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:16.947647 18876 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:16.938194 18853 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:16.948211 18876 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:16.949807 13942 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:16.951428 18788 consensus_meta.cc:287] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.688s	user 0.007s	sys 0.010s
I20260709 13:06:16.951908 18788 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:16.946887 13871 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:16.957487 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:16.909938 18929 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.911101 18818 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.911437 18980 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.925896 18887 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:16.937573 18781 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:16.951584 18853 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:16.970774 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:16.974308 18811 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.975157 13950 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:16.975522 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.976652 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:16.977367 18811 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:16.977751 13950 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:16.976769 18713 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.925638 18916 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.978309 13945 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:06:16.985128 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 374826us
I20260709 13:06:16.966176 18995 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:16.988191 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.992581 13948 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 12: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:16.980417 19018 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:16.994015 18713 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:16.994668 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 122472us
I20260709 13:06:16.995198 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:16.995479 13946 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:16.995679 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:16.994278 19018 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:16.937841 18920 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:16.995986 13838 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:16.991787 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:17.000712 18713 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:16.992574 18928 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:06:17.004673 18749 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:17.006757 18916 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:17.007288 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2888) took 2039 ms. Trace:
I20260709 13:06:17.007908 13944 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:06:17.012748 18876 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:17.016469 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:17.017313 13945 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:16.987977 18919 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.019394 18793 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:17.020411 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:17.020659 18919 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:17.021010 18785 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:17.022399 19040 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:17.023167 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:17.026486 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:17.026836 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:17.027446 14553 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:17.028028 14558 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:06:17.030393 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2855) took 1370 ms. Trace:
I20260709 13:06:17.034533 18818 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:17.034440 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:17.002722 18788 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.035699 13873 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:17.035893 18788 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.036674 18887 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:17.036723 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:17.037781 13873 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:17.038582 14555 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:17.038792 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:17.038980 18920 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:17.046922 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 88278.4us
I20260709 13:06:17.048418 19022 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:17.049404 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:17.051617 13873 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:17.049250 19021 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:17.054358 18786 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:17.063256 18786 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:16.769191 13866 rpcz_store.cc:276] 0709 13:06:15.147095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.514896 (+1367801us) service_pool.cc:224] Handling call
0709 13:06:16.515414 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.515910 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:16.568170 (+ 52260us) log.cc:844] Serialized 14 byte log entry
0709 13:06:16.767348 (+199178us) log.cc:844] Serialized 14 byte log entry
0709 13:06:16.767893 (+   545us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:16.767957 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:16.768029 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.768138 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.768339 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":107020,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":8378,"raft.queue_time_us":35829,"spinlock_wait_cycles":47328768,"thread_start_us":203997,"threads_started":2}
I20260709 13:06:17.064447 18786 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.065095 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:17.052758 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:17.066860 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71322.8us
I20260709 13:06:17.067655 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:17.069581 19025 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.756s	user 0.001s	sys 0.005s
I20260709 13:06:17.070363 18981 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:17.060994 18949 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:17.072029 18949 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:17.072896 13873 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:17.036521 18840 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:17.060714 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 72931us
W20260709 13:06:17.074837 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 68008.8us
I20260709 13:06:17.077762 18492 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:17.079262 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:17.071259 18893 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:06:17.079844 18749 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:17.081179 18749 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:17.081662 18972 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:17.082157 18972 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:17.082639 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2837) took 1974 ms. Trace:
I20260709 13:06:17.083051 13874 rpcz_store.cc:276] 0709 13:06:15.107835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.455239 (+347404us) service_pool.cc:224] Handling call
0709 13:06:16.799409 (+1344170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.935172 (+135763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.082415 (+147243us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54002e81d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:17.082571 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3090253184}
I20260709 13:06:17.083526 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:17.079437 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:17.084712 13874 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:17.079805 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:17.021243 13947 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:17.087074 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.082590 18972 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:17.091419 18874 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:17.049283 19022 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:06:17.095144 18749 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.097839 13946 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
W20260709 13:06:17.099677 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.100085 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:17.107474 18929 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:17.101715 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.102326 18749 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:17.102419 18788 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:17.073246 18949 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.059983 19018 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:17.058320 18887 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:06:17.112404 18781 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:17.112829 18781 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:17.114069 18781 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:17.115604 18781 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:17.109468 13948 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:17.118109 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.091562 18980 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:17.121625 13946 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:17.102437 13950 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:06:17.123127 18931 consensus_meta.cc:287] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.005s
I20260709 13:06:17.123718 19023 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.007694 13870 rpcz_store.cc:276] 0709 13:06:14.968066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.970616 (+  2550us) service_pool.cc:224] Handling call
0709 13:06:14.971044 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.822880 (+851836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.006810 (+1183930us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b54003abbd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:17.007094 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1957073792}
I20260709 13:06:17.124255 19023 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.125524 13870 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:06:17.030810 13875 rpcz_store.cc:276] 0709 13:06:15.660403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.590472 (+930069us) service_pool.cc:224] Handling call
0709 13:06:16.591148 (+   676us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:16.591787 (+   639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:16.591826 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:16.631385 (+ 39559us) log.cc:844] Serialized 26 byte log entry
0709 13:06:16.957608 (+326223us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:16.979019 (+ 21411us) log.cc:844] Serialized 14 byte log entry
0709 13:06:16.979279 (+   260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.979364 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:17.030127 (+ 50763us) raft_consensus.cc:1685] finished
0709 13:06:17.030154 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.030313 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23229,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":66,"raft.queue_time_us":2476,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":7738,"thread_start_us":178831,"threads_started":1,"wal-append.queue_time_us":266686}
I20260709 13:06:17.066859 13866 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:17.128371 19023 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:17.129834 18999 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.623s	user 0.012s	sys 0.002s
I20260709 13:06:17.122272 13946 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:17.131345 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48327.8us
I20260709 13:06:17.129904 13771 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 9 to 10, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 10 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.133615 18893 raft_consensus.cc:3055] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:06:17.141925 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71949.6us
I20260709 13:06:17.137210 13988 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:17.140152 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:17.141019 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:17.096776 19014 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.543s	user 0.004s	sys 0.008s
I20260709 13:06:17.130721 18852 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:06:17.137526 14524 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.900s (SENT) [suppressed 8 similar messages]
I20260709 13:06:17.171963 13866 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:17.172119 13838 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:17.168213 13945 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:06:17.065727 18996 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.169763 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:17.177737 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:17.157984 18893 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:17.168598 18405 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:06:17.200011 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 151997us
I20260709 13:06:17.200289 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.201043 13943 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:17.201639 13874 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:17.193480 13771 catalog_manager.cc:5697] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:17.195504 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.171586 18852 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:17.162400 18999 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:17.204968 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 123732us
I20260709 13:06:17.208491 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.209416 13950 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:17.168241 18781 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 13:06:17.213759 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 76820.9us
I20260709 13:06:17.214087 13875 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:17.169235 19014 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:17.204195 13946 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:17.218588 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 68117.5us
I20260709 13:06:17.218953 18989 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:17.219022 13868 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:17.219039 19021 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:17.223419 13988 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:17.227083 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:17.206239 18852 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:17.229990 13945 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:17.231050 13838 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:17.244997 14561 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:17.228415 13771 catalog_manager.cc:5697] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 10 to 11, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 11 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:17.249303 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.260901 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 717) took 1196 ms. Trace:
I20260709 13:06:17.261415 13872 rpcz_store.cc:276] 0709 13:06:16.064222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.614875 (+550653us) service_pool.cc:224] Handling call
0709 13:06:16.615433 (+   558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.615869 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:06:16.901254 (+285385us) log.cc:844] Serialized 14 byte log entry
0709 13:06:17.208142 (+306888us) log.cc:844] Serialized 14 byte log entry
0709 13:06:17.219535 (+ 11393us) log.cc:844] Serialized 14 byte log entry
0709 13:06:17.260210 (+ 40675us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b18003d1260. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c0348a40 00007fb2c03434e4 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287 00007fb2c344e208
0709 13:06:17.260387 (+   177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:17.260441 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:17.260507 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.260575 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.260824 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71799,"prepare.queue_time_us":168198,"prepare.run_cpu_time_us":158,"prepare.run_wall_time_us":164,"raft.queue_time_us":124016,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":7532160,"thread_start_us":451220,"threads_started":2}
I20260709 13:06:17.262974 13943 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:17.264432 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.265175 13949 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:17.245285 18916 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:17.262514 19015 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:06:17.228964 18874 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:06:17.267431 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:17.279142 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48194.4us
I20260709 13:06:17.277844 13945 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
W20260709 13:06:17.285022 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 61746.2us
I20260709 13:06:17.302910 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.282290 18940 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:17.276327 19021 raft_consensus.cc:697] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:17.315222 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.316651 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:17.296834 19022 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:17.321791 13942 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
W20260709 13:06:17.324689 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 102169us
I20260709 13:06:17.246807 18893 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.308917 18940 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:17.340993 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:17.340998 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 203636us
I20260709 13:06:17.323412 19022 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.345764 13945 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:17.320613 18989 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:17.297307 18405 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:06:17.351089 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:17.316815 18920 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:17.327919 13771 catalog_manager.cc:5697] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 7 to 8, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 8 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:17.359688 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:17.345122 18940 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:17.363090 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:17.365332 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:17.365911 14553 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:06:17.346493 19023 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:17.368331 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:17.352550 19022 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:17.371981 14021 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:17.351178 18996 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:17.373479 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:17.374158 14018 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:17.369979 13773 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57642 (request call id 273) took 45650 ms (client timeout 15000 ms). Trace:
I20260709 13:06:17.370124 19042 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.381770 19014 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:17.382195 14019 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:17.383325 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.384016 13941 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:17.390599 18919 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:17.394548 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.394578 18893 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:17.398556 19055 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
W20260709 13:06:17.399411 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 48435.1us
I20260709 13:06:17.401223 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:17.407181 14018 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:17.363915 18989 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:17.419032 19056 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:06:17.441594 13771 catalog_manager.cc:5697] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 7 to 8, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 8 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
W20260709 13:06:17.474403 19061 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:06:17.364454 18920 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.543332 18972 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:17.374752 13773 rpcz_store.cc:269] 0709 13:05:31.660823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.661077 (+   254us) service_pool.cc:224] Handling call
0709 13:06:17.306527 (+45645450us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:07.192668 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:07.231357 (+ 38689us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:07.231431 (+    74us) write_op.cc:435] Acquired schema lock
0709 13:06:07.231461 (+    30us) tablet.cc:662] Decoding operations
0709 13:06:07.232899 (+  1438us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:07.233042 (+   143us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:07.233068 (+    26us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:06:07.319070 (+ 86002us) tablet.cc:701] Row locks acquired
0709 13:06:07.343332 (+ 24262us) write_op.cc:260] PREPARE: finished
0709 13:06:07.399112 (+ 55780us) write_op.cc:270] Start()
0709 13:06:07.418905 (+ 19793us) write_op.cc:276] Timestamp: P: 1783602367382588 usec, L: 0
0709 13:06:07.449154 (+ 30249us) op_driver.cc:348] REPLICATION: starting
0709 13:06:07.454044 (+  4890us) log.cc:844] Serialized 4801 byte log entry
0709 13:06:08.400594 (+946550us) op_driver.cc:464] REPLICATION: finished
0709 13:06:09.429612 (+1029018us) write_op.cc:301] APPLY: starting
0709 13:06:09.462036 (+ 32424us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:09.797895 (+335859us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:09.798024 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:10.062551 (+264527us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:10.062932 (+   381us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:06:10.062982 (+    50us) write_op.cc:312] APPLY: finished
0709 13:06:10.063679 (+   697us) log.cc:844] Serialized 113 byte log entry
0709 13:06:10.566739 (+503060us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c03290ac 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b
0709 13:06:10.942512 (+375773us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:10.942685 (+   173us) write_op.cc:454] Released schema lock
0709 13:06:10.942761 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:10.943239 (+   478us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:10.943270 (+    31us) write_op.cc:454] Released schema lock
0709 13:06:11.434967 (+491697us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
Metrics: {"mutex_wait_us":8915302,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":910801,"apply.run_cpu_time_us":58791,"apply.run_wall_time_us":1570769,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54215,"num_ops":12,"prepare.queue_time_us":521006,"prepare.run_cpu_time_us":62433,"prepare.run_wall_time_us":481060,"raft.queue_time_us":437985,"raft.run_cpu_time_us":46567,"raft.run_wall_time_us":691393,"replication_time_us":951194,"spinlock_wait_cycles":226949888,"thread_start_us":2175382,"threads_started":5,"wal-append.queue_time_us":1066260,"wal-append.run_cpu_time_us":34004,"wal-append.run_wall_time_us":3104633}]]}
I20260709 13:06:17.376003 19042 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:17.578581 13874 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:06:17.561555 13866 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:06:17.560672 13947 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:06:17.584251 13871 raft_consensus.cc:2410] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:06:17.546012 18972 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:17.603338 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 202797us
I20260709 13:06:17.613266 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:17.614187 14016 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:17.614768 14018 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:06:17.615700 14022 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:06:17.602166 13868 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2898) took 2535 ms (client timeout 1624 ms). Trace:
I20260709 13:06:17.599241 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:17.554320 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 156352us
I20260709 13:06:17.601701 18989 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:17.618139 13868 rpcz_store.cc:269] 0709 13:06:15.066814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.114299 (+ 47485us) service_pool.cc:224] Handling call
0709 13:06:17.601540 (+2487241us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b540026bbd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:17.602014 (+   474us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4067903744}
I20260709 13:06:17.626956 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:17.571290 13771 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 13 to 15, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 15 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.627912 14017 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:17.546662 18785 consensus_meta.cc:287] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.525s	user 0.012s	sys 0.000s
W20260709 13:06:17.637856 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 176914us
I20260709 13:06:17.637746 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:17.646894 14021 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:06:17.647562 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.647473 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:17.636801 19056 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:06:17.647835 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:17.648758 13946 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:17.648963 13941 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:17.650830 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.651348 19023 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:17.652083 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:17.672524 14015 raft_consensus.cc:1813] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Handling vote request from an unknown peer 3779649036364744823fc035dac892c7
I20260709 13:06:17.653836 19014 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:17.654105 18887 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:17.683719 13947 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:17.684248 14015 raft_consensus.cc:2410] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:06:17.624872 19033 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:17.551518 19042 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:17.553709 18920 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:17.658103 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:17.679139 14018 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:17.703307 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:17.678763 13771 catalog_manager.cc:5697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 11 to 12, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 12 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.636276 18999 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:17.715963 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 57955.8us
I20260709 13:06:17.723089 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2852) took 2067 ms. Trace:
I20260709 13:06:17.725378 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.727598 13771 catalog_manager.cc:5697] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 7 to 9, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 9 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:17.627130 18940 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.687885 14561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 843) took 1056 ms. Trace:
I20260709 13:06:17.723575 13867 rpcz_store.cc:276] 0709 13:06:15.655619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.583352 (+927733us) service_pool.cc:224] Handling call
0709 13:06:16.583901 (+   549us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:16.632676 (+ 48775us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:16.935031 (+302355us) log.cc:844] Serialized 14 byte log entry
0709 13:06:17.305572 (+370541us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:17.323404 (+ 17832us) log.cc:844] Serialized 52 byte log entry
0709 13:06:17.323581 (+   177us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:17.323897 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 13:06:17.324110 (+   213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.555088 (+230978us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b18002b02a0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:17.599821 (+ 44733us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b54002e59d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:17.599904 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:17.722832 (+122928us) raft_consensus.cc:1685] finished
0709 13:06:17.722854 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.723023 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13802,"prepare.queue_time_us":210834,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":94,"raft.queue_time_us":7884,"raft.run_cpu_time_us":470,"raft.run_wall_time_us":55640,"spinlock_wait_cycles":84048000,"thread_start_us":599868,"threads_started":2,"wal-append.queue_time_us":404077}
I20260709 13:06:17.690196 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.743191 13945 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:17.740115 18887 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:17.765195 18887 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:17.708902 18989 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.766815 18989 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.717204 19003 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:17.771682 19003 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.741425 14561 rpcz_store.cc:276] 0709 13:06:16.610583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.626989 (+ 16406us) service_pool.cc:224] Handling call
0709 13:06:17.666552 (+1039563us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400710f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:17.666734 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1215186688}
W20260709 13:06:17.695511 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 144493us
I20260709 13:06:17.702904 13948 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:17.680732 14019 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:06:17.724115 18820 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:06:17.734699 13771 catalog_manager.cc:5697] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.708633 13840 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:17.665314 14017 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
W20260709 13:06:17.782179 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 220904us
I20260709 13:06:17.708578 19040 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:17.786895 13771 catalog_manager.cc:5697] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 8 to 10, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 10 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:17.769910 18989 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:17.548494 19021 consensus_queue.cc:237] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.725323 18785 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:06:17.801222 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.797633 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:17.799633 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:17.809841 13771 catalog_manager.cc:5697] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 6 to 8, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 8 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:17.815215 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 112779us
I20260709 13:06:17.821136 19041 consensus_queue.cc:1059] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:17.821579 18920 consensus_queue.cc:1059] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:17.822480 19042 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:17.822700 18713 raft_consensus.cc:3055] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:17.823238 18713 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:17.800141 19040 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:17.827010 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:17.828837 19040 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:17.822851 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:17.830937 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.823314 18920 raft_consensus.cc:3055] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:17.834532 18744 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:17.834640 18713 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.838516 18874 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:17.817198 18852 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.005s
I20260709 13:06:17.807653 18989 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:17.805346 18785 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.823419 19042 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:17.859213 13871 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:17.861205 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:17.807523 18999 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.830534 18492 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:17.819293 18887 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:17.836643 19003 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:17.842993 19040 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:17.860400 13771 catalog_manager.cc:5697] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 10 to 11, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 11 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:17.865767 13869 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:17.874228 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:17.869295 18744 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:17.837054 18920 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:17.863610 18874 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.880962 18940 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:17.881696 18874 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:17.884859 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 175646us
I20260709 13:06:17.885155 13945 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:17.886488 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:17.854701 18785 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:17.892419 13771 catalog_manager.cc:5697] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 8 to 11, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 11 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:17.894349 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 71684.1us
I20260709 13:06:17.895073 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:17.863981 19042 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:17.870338 18492 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.899669 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:17.905737 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:17.869431 18989 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:17.912072 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:17.912186 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:17.912524 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:17.891252 18887 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:17.882123 18405 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:06:17.873775 18999 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.865162 18852 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:17.923921 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:17.924577 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:17.921159 13870 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:17.927640 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:17.928346 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 60891.5us
I20260709 13:06:17.930768 18820 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:17.914912 18972 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:17.932991 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:17.906108 19040 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:17.909683 18492 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:17.913595 18713 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:17.944005 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:17.948773 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:17.962929 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:17.908546 18893 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:06:17.973169 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:17.973954 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:17.971268 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:17.944856 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0449374999999463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260709 13:06:17.977298 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.937955 19003 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:17.977058 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:17.981053 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2901) took 1037 ms. Trace:
W20260709 13:06:17.982723 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.984557 13869 rpcz_store.cc:276] 0709 13:06:16.943572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.958822 (+ 15250us) service_pool.cc:224] Handling call
0709 13:06:17.980835 (+1022013us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b54000a8f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:17.980987 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1573275904}
I20260709 13:06:17.986588 13771 catalog_manager.cc:5697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
W20260709 13:06:17.987526 19022 consensus_meta.cc:287] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.002s
I20260709 13:06:17.988016 19022 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:17.990294 13866 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:17.989279 13872 raft_consensus.cc:2368] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11.
I20260709 13:06:17.941807 19040 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:17.989239 13771 catalog_manager.cc:5697] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 4 to 7, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 7 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.997061 18853 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:18.005630 18777 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:17.978461 18972 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:17.983566 19003 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.921406 18920 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.011179 13873 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:17.950026 19041 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:06:18.002763 19040 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:18.025228 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:18.026450 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:18.029343 13946 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:17.978222 18986 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:06:17.977890 18919 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:06:18.009382 18777 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:18.041915 13875 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:18.042328 13771 catalog_manager.cc:5697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 7 to 9, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 9 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:18.018599 19014 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:18.021230 19003 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:18.049942 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:18.051167 13771 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:18.017602 18853 raft_consensus.cc:697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:18.056417 13771 catalog_manager.cc:5697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 8 to 10, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 10 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:18.053632 13866 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:06:18.033481 18940 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:06:18.044960 18777 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:18.066800 13771 catalog_manager.cc:5697] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
W20260709 13:06:18.073360 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.076807 13771 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 8 to 11, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 11 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:18.068624 19074 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:06:18.059803 18749 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:17.997524 19042 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.081770 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:18.085445 13873 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:18.020421 18744 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:18.095508 19066 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.178s	user 0.001s	sys 0.001s
I20260709 13:06:18.096004 19077 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:18.096664 19077 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:18.097783 19077 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.102082 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:18.105429 13875 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:18.106354 18980 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:18.079099 18920 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:18.111281 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 110224us
I20260709 13:06:18.117606 18940 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:18.119650 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:18.120878 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.124366 18999 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:18.048640 18887 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:18.137053 18989 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:18.153563 14524 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:18.156721 13870 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:18.178874 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.179806 13870 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:18.181525 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.142133 18749 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:18.144891 18999 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:18.100574 18853 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:18.154844 19042 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:18.157446 18405 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.110363 18972 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:18.210229 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 118336us
I20260709 13:06:18.152348 18852 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:18.209872 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 234936us
I20260709 13:06:18.204746 18999 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:18.217628 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:18.218477 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:18.157034 18940 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:18.223135 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.210243 18781 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:18.224561 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.214136 19073 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:18.214804 13870 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:18.224689 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 87987.7us
I20260709 13:06:18.213161 18749 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:18.237528 18996 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:18.223414 18980 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:18.246186 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.226323 18852 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:18.251489 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.252552 19082 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:18.253913 19082 raft_consensus.cc:3055] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:18.254349 19082 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:18.255311 19082 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:18.256796 19082 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:18.233771 18744 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:18.263041 19023 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.271057 18852 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:18.279793 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.223827 18405 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.287070 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:18.285238 13875 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:18.289983 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:18.261440 18777 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.290560 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:18.298115 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.301748 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:18.282657 19023 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:18.302587 18749 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.303925 13870 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:18.304443 13875 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:18.304888 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:18.306520 13873 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:18.266450 18744 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.311345 13943 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:18.315404 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:18.325464 19084 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.326574 19084 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.329483 19084 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:18.330687 19087 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:18.339321 18405 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:18.347625 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.348658 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:18.349695 14024 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:06:18.290362 18999 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.300449 18781 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:06:18.355856 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:18.307307 18713 consensus_queue.cc:1059] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:06:18.358490 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.360675 18777 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:18.360483 14018 raft_consensus.cc:2955] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 7 to 13, VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:18.370041 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:18.372334 14020 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:18.372596 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:18.373225 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:18.373385 14021 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:18.358574 18785 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.371001 14022 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:18.375419 18744 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:18.376017 18785 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:18.377409 13873 raft_consensus.cc:2368] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12.
I20260709 13:06:18.370357 14017 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:06:18.384095 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 73651us
W20260709 13:06:18.389833 19079 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:06:18.393433 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.376762 18986 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:18.366411 18999 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:18.368350 19023 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:18.400509 13875 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:18.370333 18713 raft_consensus.cc:3055] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:18.367841 18749 raft_consensus.cc:2764] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:06:18.394953 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.371482 18852 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:18.414600 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:18.423992 14017 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:18.425709 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:18.435372 14022 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:18.441332 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:18.442600 13869 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:18.397665 18986 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:18.444341 13868 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:06:18.450831 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:18.457677 14021 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:18.417626 18713 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:18.460450 18713 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.463877 18820 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:18.465131 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:18.465859 13872 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:18.466173 19041 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:18.471218 19022 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:18.471668 14021 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:06:18.473150 19078 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.388s	user 0.005s	sys 0.000s
I20260709 13:06:18.473495 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:18.474402 14020 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:18.479188 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:18.479954 14021 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:18.470745 18986 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.459453 14022 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:18.483868 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:18.493906 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:18.494632 13875 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 12. Current term is 13. Ops: []
W20260709 13:06:18.495997 18492 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.513s	user 0.011s	sys 0.002s
I20260709 13:06:18.496232 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.496469 18492 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.497676 13867 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:18.494750 18989 raft_consensus.cc:3055] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:18.499094 18989 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:18.504362 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:18.504716 19070 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.531s	user 0.002s	sys 0.002s
I20260709 13:06:18.505244 14023 raft_consensus.cc:2368] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12.
I20260709 13:06:18.506132 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:18.505683 14021 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:18.520311 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:18.521116 14021 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:18.500134 18989 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:18.549511 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:18.549597 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:18.550738 14023 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:18.525779 14017 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:18.555094 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:18.513990 18749 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.575466 14021 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:18.471962 19022 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:06:18.582185 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:18.583546 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:18.527149 18986 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:18.590523 14553 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:18.597786 13872 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:18.608632 14017 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:06:18.632871 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:18.531111 19087 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:18.622406 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:18.621793 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.578450 19041 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:18.611581 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 160889us
I20260709 13:06:18.630635 14559 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:18.621328 19084 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.642292 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:18.644505 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:18.648998 14555 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:06:18.653880 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 78227.4us
W20260709 13:06:18.655220 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.585683 18749 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.657797 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:18.659014 14021 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:18.660838 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.642450 18940 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:18.666560 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.667807 13872 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:18.671274 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:18.672683 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:18.674500 13949 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:06:18.652788 18989 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:18.706753 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:18.708022 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:18.706771 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 49387.2us
I20260709 13:06:18.652973 18492 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:18.651185 19087 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:18.711877 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 60088us
I20260709 13:06:18.724082 13949 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:18.727694 14020 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:18.727864 13870 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:06:18.729728 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:18.645957 19041 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.725477 14559 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:18.652470 19084 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.733466 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:18.736061 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:18.689707 19003 consensus_meta.cc:287] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.002s
I20260709 13:06:18.761696 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:18.765475 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.767704 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.768541 13872 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 8.
I20260709 13:06:18.770745 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:18.772192 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:18.778185 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:18.778924 13942 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:18.781515 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:18.784371 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 62311.2us
I20260709 13:06:18.786872 14020 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:18.787417 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.793267 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:18.794003 13945 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:18.795691 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:18.798954 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:18.799667 19095 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:06:18.800662 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:18.805218 19094 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:06:18.806831 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:18.807067 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:18.810479 14024 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:18.811757 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:18.812664 13950 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:18.814424 18492 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:18.814994 18492 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:18.817396 18492 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:18.817508 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:18.818663 13944 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:18.819926 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:18.783856 19077 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
W20260709 13:06:18.824082 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.834720 14020 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:18.852216 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.859743 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.789398 18874 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:18.867166 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:18.866685 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.868233 13872 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:18.761087 19003 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:18.871590 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.871762 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:18.879299 19084 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:18.878943 19087 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:18.880791 18749 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:18.866539 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:18.853725 19040 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.806s	user 0.013s	sys 0.001s
I20260709 13:06:18.885080 13944 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:18.866729 18996 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:18.891553 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:18.892448 13872 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:18.894016 14019 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:18.895241 13945 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:18.895399 13870 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:18.902004 13874 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:18.903600 19112 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:18.904336 19112 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:18.911860 19003 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:18.912811 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:18.917025 13942 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:18.929320 13871 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:18.939054 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.942761 13950 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:18.945437 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:18.905330 19112 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:18.955080 13915 leader_election.cc:400] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11.
I20260709 13:06:18.955657 13915 leader_election.cc:403] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 13:06:18.933629 19098 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:06:18.965956 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 73104.8us
W20260709 13:06:18.966362 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:18.952284 18920 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.823s	user 0.013s	sys 0.001s
I20260709 13:06:18.966818 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:18.970959 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.971531 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:18.972276 13945 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:18.973857 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:18.952445 18874 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.888223 19040 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:18.962402 19042 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.750s	user 0.013s	sys 0.000s
I20260709 13:06:18.980516 13945 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:18.983600 13946 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:18.985947 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:18.987025 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:19.004457 13872 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:18.987865 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:18.996807 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:18.989694 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:18.999428 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:19.004060 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:19.004690 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:19.007326 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:19.009673 13838 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:19.010116 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:19.007901 14559 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:06:19.009130 18781 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:19.013764 18749 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:19.014393 18749 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:19.016547 14015 raft_consensus.cc:2410] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:06:18.978427 18996 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11.
I20260709 13:06:18.991255 19021 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.024689 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:19.027539 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:19.027271 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:19.021615 13942 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:19.028970 14555 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:19.031399 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:19.030123 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:19.037714 19082 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.774s	user 0.014s	sys 0.000s
I20260709 13:06:19.039281 13946 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:19.039199 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:19.024413 18852 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:19.029738 19023 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:19.006042 18874 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.031852 18492 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:19.044497 19040 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:19.038693 18820 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:19.034618 18785 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:19.027068 18972 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:06:19.033872 19021 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.062157 19109 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:19.063249 18405 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:19.047590 19041 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:19.056918 18852 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:19.047212 19103 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:06:19.054770 18996 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:19.057433 18893 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:19.065433 13989 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:19.067281 19040 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:19.060958 19023 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:19.084307 19110 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:19.088304 19115 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:19.086793 19083 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:19.087203 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:19.089140 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 79127.3us
I20260709 13:06:19.092859 18405 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:19.036114 18887 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:19.094313 19114 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:06:19.095139 18785 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:19.090670 18820 raft_consensus.cc:697] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:06:19.103226 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.110154 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:19.050119 19042 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:19.111764 13989 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:19.113905 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 71354.5us
I20260709 13:06:19.109613 19022 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.122722 18887 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.124332 18874 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:19.124753 18999 consensus_meta.cc:287] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.707s	user 0.008s	sys 0.004s
I20260709 13:06:19.156877 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:19.157680 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 70579us
I20260709 13:06:19.107627 19110 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:19.161406 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:19.130367 13943 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:19.097218 18744 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:06:19.146230 14023 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:19.101794 18893 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.125093 19126 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:19.154250 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 157558us
I20260709 13:06:19.171339 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:19.110399 18996 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:19.179625 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:19.152678 18919 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:19.092832 18749 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.175269 13989 leader_election.cc:400] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12.
I20260709 13:06:19.155061 19003 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:19.186822 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 161664us
I20260709 13:06:19.187598 13989 leader_election.cc:403] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:19.176393 18713 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:19.176847 18777 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:19.129289 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 90223.2us
I20260709 13:06:19.172129 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:19.190228 14558 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:19.185949 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:19.156021 18492 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:19.194391 13989 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:19.195036 19023 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.195523 18820 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.196013 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:19.197333 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:19.194383 13942 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:19.161881 19082 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.199317 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:19.202157 14017 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:19.168913 19083 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:19.173237 19084 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:19.173700 18893 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:19.207623 13914 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:19.207985 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 57336.7us
I20260709 13:06:19.152352 19022 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.226562 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:19.233050 14018 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:19.161099 19103 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:06:19.116487 19115 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.185096 18919 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:19.235132 18919 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.236774 18919 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:19.230242 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 44411.5us
I20260709 13:06:19.240330 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:19.246209 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:19.198181 18920 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:19.201292 18492 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:19.255362 13989 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:19.257979 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:19.213829 19084 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.213702 18999 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:19.215113 19042 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:06:19.210302 19083 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:19.230345 19040 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:19.250401 19112 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:19.246111 19003 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:19.185810 19126 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.223774 19082 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:19.286872 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 93726.8us
I20260709 13:06:19.289220 14018 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:19.268877 19041 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:19.203434 18777 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.276684 19084 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:19.181099 18852 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.228070 19109 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
W20260709 13:06:19.343358 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 118137us
I20260709 13:06:19.291353 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:19.345238 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:19.263176 18492 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:19.357059 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:19.286923 19083 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:19.361704 14018 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:19.269441 18940 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:06:19.296058 18999 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:19.214710 18405 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.281184 18980 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
W20260709 13:06:19.364480 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 73227.4us
I20260709 13:06:19.541401 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:19.302568 18920 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:19.371899 13915 connection.cc:256] RPC call timeout handler was delayed by -1.5027541939999765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 13:06:19.550925 13915 connection.cc:256] RPC call timeout handler was delayed by -1.4829541939999444s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
W20260709 13:06:19.551306 13915 connection.cc:256] RPC call timeout handler was delayed by -1.472154193999927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
I20260709 13:06:19.346773 19040 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:19.373637 19128 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.327s
I20260709 13:06:19.507982 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:19.296484 18996 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:19.448750 19130 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:06:19.451109 19123 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:19.452123 19071 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:19.276201 18785 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.509603 18887 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:19.450023 19132 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:19.438927 18999 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.341876 19125 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:06:19.462292 19041 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:19.350587 18744 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 13:06:19.601110 19022 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:19.604703 13943 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:19.606441 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:19.606920 13872 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:06:19.609053 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:19.611845 19082 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:19.613375 13943 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:06:19.613983 13943 pending_rounds.cc:85] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 7
I20260709 13:06:19.614356 13943 pending_rounds.cc:107] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:06:19.614785 13943 raft_consensus.cc:2889] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:19.618484 13872 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:19.618803 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:19.553188 19040 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.622326 19040 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:19.623554 13871 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:19.625054 13873 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
W20260709 13:06:19.575289 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 211266us
W20260709 13:06:19.629165 14016 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.001s
I20260709 13:06:19.447114 19124 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:19.346520 19112 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:19.440387 19122 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:19.485157 19021 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:19.522873 18777 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:19.523600 19140 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:19.532570 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:19.533432 14018 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
W20260709 13:06:19.409363 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.630684 18405 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:19.551663 18492 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.497534 19135 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:19.554173 14023 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:06:19.522895 19129 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:06:19.566749 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:19.576937 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 216310us
I20260709 13:06:19.525871 19110 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:19.582823 19134 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 13:06:19.584501 18989 consensus_meta.cc:287] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.861s	user 0.011s	sys 0.004s
I20260709 13:06:19.590489 18986 raft_consensus.cc:3055] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:19.591509 19042 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12.
I20260709 13:06:19.594331 19142 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.598832 19126 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:19.629739 14016 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:19.636612 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:19.639148 13873 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:19.565838 19123 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.566071 19071 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:19.652248 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:19.537518 14022 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:19.657181 14024 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:19.660881 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:19.632287 18940 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.664925 13871 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:06:19.665072 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:19.579362 19131 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:06:19.669359 13950 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:06:19.694855 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:19.638540 19124 consensus_queue.cc:1243] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:06:19.669950 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 103318us
I20260709 13:06:19.701515 13989 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:19.670892 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:19.703120 13872 raft_consensus.cc:2410] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
W20260709 13:06:19.703644 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 67122.3us
I20260709 13:06:19.675417 13874 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:19.647610 19115 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:19.678300 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:19.678742 14021 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:06:19.683436 14555 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:19.681449 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:19.710755 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:19.706670 18887 consensus_queue.cc:1059] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:06:19.669060 18777 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.713277 18887 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:19.713793 18887 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:19.713565 18777 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.714789 18887 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.715858 18777 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:19.716228 18887 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:19.716583 19083 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.717185 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:19.717815 14015 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:06:19.653168 19122 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:19.746328 14022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2790) took 1265 ms (client timeout 1679 ms). Trace:
W20260709 13:06:19.746994 14022 rpcz_store.cc:269] 0709 13:06:18.480621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.487217 (+  6596us) service_pool.cc:224] Handling call
0709 13:06:19.746130 (+1258913us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b5400020654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:19.746258 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1554258688}
I20260709 13:06:19.689051 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:19.748667 14017 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:19.697026 13988 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:19.652052 19140 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:19.673159 18492 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:19.705070 13839 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:19.757388 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 52445.6us
I20260709 13:06:19.675207 18986 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:19.710804 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:19.762300 14551 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:19.762583 18986 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.763152 19084 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.764205 18986 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:19.765481 19084 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:19.751255 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 54451.4us
I20260709 13:06:19.753750 18777 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:19.718410 18989 raft_consensus.cc:2764] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:06:19.770391 19122 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.772193 19083 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:19.718101 19126 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:19.773661 19126 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:19.718312 19082 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:19.776475 19003 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:19.776755 19132 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:19.683202 19041 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:19.778214 19112 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.780954 18999 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:19.760340 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 49710.1us
I20260709 13:06:19.783206 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:19.684163 19123 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:19.754784 18893 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.784149 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 580) took 1105 ms. Trace:
W20260709 13:06:19.754534 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55954.4us
I20260709 13:06:19.788409 18492 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:19.819823 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:19.820817 14024 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:19.820956 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2798) took 1151 ms. Trace:
I20260709 13:06:19.822726 14023 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:19.824899 14020 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:06:19.825387 14020 pending_rounds.cc:85] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 8
I20260709 13:06:19.825670 14020 pending_rounds.cc:107] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:06:19.825966 14020 raft_consensus.cc:2889] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:19.795941 14021 rpcz_store.cc:276] 0709 13:06:18.678441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.792112 (+113671us) service_pool.cc:224] Handling call
0709 13:06:19.783886 (+991774us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b540004b354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:19.784083 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1972398336}
I20260709 13:06:19.830431 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:19.830857 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:19.821549 14016 rpcz_store.cc:276] 0709 13:06:18.669241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.800362 (+131121us) service_pool.cc:224] Handling call
0709 13:06:19.820666 (+1020304us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b540023c154. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:19.820867 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":678364928}
I20260709 13:06:19.723289 18744 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:06:19.832832 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 153460us
I20260709 13:06:19.831583 14022 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:19.836355 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:19.758498 19138 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:19.836707 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:19.843770 14016 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:06:19.846251 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47674.7us
W20260709 13:06:19.846658 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 63594.8us
I20260709 13:06:19.785383 18713 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:19.851779 13870 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:06:19.699276 18940 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.855624 14561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 856) took 1308 ms. Trace:
I20260709 13:06:19.722663 19022 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.131s
W20260709 13:06:19.788822 19097 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.012s	sys 0.022s
W20260709 13:06:19.867452 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:19.850939 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 225710us
I20260709 13:06:19.827440 18777 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.874624 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:19.841727 18986 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.831250 19082 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:19.884526 13949 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:19.722184 19142 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:19.884799 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 167710us
I20260709 13:06:19.849922 19138 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.886654 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:19.847733 19136 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:19.890174 13840 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:19.816247 19073 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 13:06:19.876899 19097 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.012s	sys 0.023s
W20260709 13:06:19.880838 18781 consensus_meta.cc:287] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.867s	user 0.009s	sys 0.005s
I20260709 13:06:19.864007 14561 rpcz_store.cc:276] 0709 13:06:18.547263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.547438 (+   175us) service_pool.cc:224] Handling call
0709 13:06:18.582667 (+ 35229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:18.615047 (+ 32380us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:18.639482 (+ 24435us) log.cc:844] Serialized 14 byte log entry
0709 13:06:19.183178 (+543696us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b1800134e00. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd323a 00007fb2c343da2a
0709 13:06:19.667466 (+484288us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:19.667524 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:19.677973 (+ 10449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:19.855291 (+177318us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b5400635454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:19.855368 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:19.855544 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18729,"prepare.queue_time_us":18056,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":9084,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":20349,"spinlock_wait_cycles":49583616,"thread_start_us":474250,"threads_started":1}
I20260709 13:06:19.912634 18492 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:19.861634 19014 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:06:19.946973 18492 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:19.903760 13915 connection.cc:256] RPC call timeout handler was delayed by -1.4181541939999533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1869s for a response
I20260709 13:06:19.875250 18999 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:19.918258 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 159499us
I20260709 13:06:19.868237 18744 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:19.930265 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:19.924425 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:19.897449 19138 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:19.947791 13915 connection.cc:256] RPC call timeout handler was delayed by -1.2966541939999843s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
I20260709 13:06:19.955411 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:19.963652 13950 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
W20260709 13:06:19.963891 13915 connection.cc:256] RPC call timeout handler was delayed by -1.2327541939999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
I20260709 13:06:19.885416 19143 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.962383 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:19.962767 14017 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:19.979295 13941 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:19.964938 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:19.896251 19136 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:19.960690 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 74125.5us
I20260709 13:06:19.984573 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:19.926163 18919 consensus_meta.cc:287] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.682s	user 0.016s	sys 0.000s
I20260709 13:06:19.952386 18713 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.989075 19149 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:19.910647 19125 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:06:19.989753 19149 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.895586 18986 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.992305 19149 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:19.937199 18989 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:19.994270 18986 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:19.994807 18989 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.976534 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:19.980331 13948 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:19.842001 19140 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.000715 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:19.951732 19003 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:19.988795 18713 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.970489 19071 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.000730 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:19.996606 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:20.003311 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 78992.6us
I20260709 13:06:20.006896 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:20.006999 18492 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.955713 19077 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.886124 18777 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:19.963176 18781 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:20.045182 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48652.7us
I20260709 13:06:20.045399 18492 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.046307 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:20.017900 18887 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:06:20.047559 19153 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:20.048100 19153 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.048450 19153 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:20.049922 18713 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:20.050779 19071 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:20.050864 19077 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.053977 19077 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:20.054198 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 69710.1us
I20260709 13:06:20.055105 13989 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:20.055143 18744 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.058799 14559 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:20.059399 18989 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:20.059938 19163 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:20.061520 19132 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:20.066839 18492 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:20.068888 18940 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:20.069732 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:20.073094 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 117774us
I20260709 13:06:20.073714 19083 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:20.074236 19083 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.074581 19083 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:20.075842 19125 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:20.076367 19125 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.076766 19125 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:19.982734 19109 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.017675 19141 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:19.947988 19126 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:19.976617 18980 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:06:20.083120 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:20.085937 19087 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:20.086426 19087 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:19.996759 19136 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:20.087786 13950 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:06:20.087412 19087 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:20.089484 19154 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:06:20.007179 18986 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:20.106068 13946 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:20.107913 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62889.5us
I20260709 13:06:20.109308 13840 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:20.101774 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.060012 18989 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:20.111901 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:20.114148 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:20.115207 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:20.118093 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:20.120574 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:20.122874 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:20.124699 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:20.078920 19109 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.133879 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:20.122720 19084 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:20.018020 19003 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:20.142069 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:20.142206 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:20.145754 14551 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
W20260709 13:06:20.143239 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.155637 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:20.112636 18713 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:20.101788 18996 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:06:19.997536 19143 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.127655 18919 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:20.175597 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:20.171442 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:20.174998 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:20.183467 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 49674.3us
I20260709 13:06:20.183993 13948 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:20.184341 14561 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:20.101473 19115 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:06:20.132179 19087 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:20.184765 13840 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:20.104555 19082 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:20.189429 19146 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:06:20.190368 19142 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:20.153061 19021 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:20.204548 19143 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:20.207460 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 68258.3us
I20260709 13:06:20.137562 19128 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:20.226403 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 59356.1us
I20260709 13:06:20.137533 19084 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.144673 18986 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:20.234529 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:20.235047 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:20.235366 13866 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:20.235699 13867 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:20.218695 13946 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:20.219440 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:20.236765 13873 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:20.228528 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:20.164911 18893 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:20.230837 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:20.242257 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:20.237982 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:20.243017 13869 raft_consensus.cc:2376] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13. Re-sending same reply.
I20260709 13:06:20.246019 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:20.214567 19077 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:06:20.227262 19040 consensus_meta.cc:287] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.604s	user 0.006s	sys 0.006s
I20260709 13:06:20.251112 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:20.216571 19121 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:20.234140 19084 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:20.252583 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:20.226363 19124 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:20.262351 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:20.263031 13867 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:20.264341 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:20.187299 18713 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.280076 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:20.283465 14559 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:20.241289 19021 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:20.246199 18919 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:20.286585 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:20.284883 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:20.285203 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:20.286149 13870 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:20.256085 19160 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:20.238979 19014 consensus_queue.cc:1059] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:06:20.240051 19151 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:06:20.322091 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:20.325730 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:20.322870 13867 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:20.332885 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:20.334782 14016 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:20.336074 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:20.336661 14016 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:20.337605 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:20.339330 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:20.340025 14015 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:20.342695 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:20.343549 13869 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:20.348673 19153 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.349910 14017 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:20.352051 19110 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:20.353778 19167 log.cc:927] Time spent T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:06:20.333794 14021 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:20.375854 13869 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
W20260709 13:06:20.391063 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 69548.8us
I20260709 13:06:20.396169 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:20.294211 18919 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.399012 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:20.261214 19040 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:20.405210 18405 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.741s	user 0.008s	sys 0.004s
I20260709 13:06:20.457832 14018 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:20.403033 18919 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:20.339345 19083 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.435478 18989 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:20.388484 19125 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.388429 19127 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.330s
I20260709 13:06:20.399269 18980 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
W20260709 13:06:20.518525 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.515213 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:20.519946 14021 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:20.401070 19077 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:06:20.320065 19003 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:20.531912 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 98563us
I20260709 13:06:20.533015 13915 leader_election.cc:400] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12.
I20260709 13:06:20.533346 13915 leader_election.cc:403] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:06:20.511979 19112 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:20.515375 19158 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:20.538947 13866 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:20.541584 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:20.547489 19170 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:06:20.550897 19168 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:06:20.552513 19159 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:20.552985 19159 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:20.535324 19109 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:20.553951 19159 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.555011 18713 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:20.555428 19159 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:20.556959 19083 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:20.561720 19138 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.000s
I20260709 13:06:20.534933 19021 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.563264 19151 raft_consensus.cc:3055] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:20.558609 19125 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:20.522274 18989 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:20.603581 18989 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:20.609634 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 284021us
I20260709 13:06:20.610764 13989 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:20.616107 19124 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:20.616963 19136 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.529s	user 0.007s	sys 0.009s
I20260709 13:06:20.585577 19151 raft_consensus.cc:740] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:20.621120 19151 consensus_queue.cc:260] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.624599 19083 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:20.624881 19151 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:20.625828 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:20.626884 18713 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:20.628799 18919 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.621315 19084 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.598842 19153 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:20.661298 14021 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:20.567780 19158 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:20.667351 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.667364 14019 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:20.617507 19136 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.669226 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:20.586802 19138 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.678920 14022 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:20.684541 19084 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:20.687429 18919 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:20.688498 18777 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:20.688879 19123 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.869s	user 0.013s	sys 0.002s
I20260709 13:06:20.691740 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:20.692545 14561 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:20.697831 19040 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:20.699182 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:20.701412 19127 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:20.701682 14018 pending_rounds.cc:85] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 5
I20260709 13:06:20.701885 19127 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:20.703490 14561 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:20.705207 14018 pending_rounds.cc:107] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:06:20.706419 14018 raft_consensus.cc:2889] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:20.705873 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:20.689409 19123 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.717268 19041 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:20.717916 19041 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:20.719159 19041 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:20.727195 19179 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:06:20.727751 19145 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.738s	user 0.004s	sys 0.003s
I20260709 13:06:20.727988 19123 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:20.730067 19164 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.740s	user 0.002s	sys 0.000s
I20260709 13:06:20.732628 19136 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:20.733891 19158 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:20.738227 19193 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.738936 19193 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.741463 19193 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:20.757314 19155 raft_consensus.cc:3055] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:20.757937 19155 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:20.758909 19155 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.760295 19155 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:20.762941 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 230073us
I20260709 13:06:20.763959 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:20.766893 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:20.713589 19071 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.003s
I20260709 13:06:20.772678 14023 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:06:20.771620 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:20.780104 13949 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:20.805895 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:20.809131 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:20.816502 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 51236.4us
I20260709 13:06:20.815446 13950 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:20.817451 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:20.818473 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:20.822928 14023 raft_consensus.cc:1240] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 13:06:20.805110 19198 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:06:20.811163 19040 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.823612 18713 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:20.835157 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.703109 19127 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.773772 18874 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:06:20.836555 19152 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.848s	user 0.003s	sys 0.000s
I20260709 13:06:20.838331 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2805) took 1125 ms. Trace:
I20260709 13:06:20.852140 14024 rpcz_store.cc:276] 0709 13:06:19.712540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.830417 (+117877us) service_pool.cc:224] Handling call
0709 13:06:20.838120 (+1007703us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b5400325ad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:20.838258 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2037285120}
I20260709 13:06:20.839529 13915 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:20.839883 14023 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:20.774818 19077 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12.
I20260709 13:06:20.783142 18989 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.850395 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:20.854388 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:20.859879 18989 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.860778 19110 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:06:20.861152 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:20.861709 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.855182 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:20.822737 19136 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.862612 19136 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.863435 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:20.864099 14554 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:20.864454 19184 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
W20260709 13:06:20.865674 19190 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:06:20.856588 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:20.836615 19160 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:20.858134 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:20.858450 19156 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.855278 18749 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.841559 19193 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:20.862380 19192 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:20.873289 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:20.850885 19040 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.918599 18749 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.929733 14554 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:20.876240 19193 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.921676 19127 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:20.920681 19156 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:20.923339 19199 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:06:20.859500 19158 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:20.977501 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:20.846709 18853 consensus_queue.cc:1059] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
I20260709 13:06:20.853287 18713 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:20.984416 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 111220us
I20260709 13:06:20.986024 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:20.987418 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:20.992899 18749 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:20.994417 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:20.979580 19190 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:06:21.000502 18749 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:21.001112 18749 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:21.002131 18749 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:20.995328 19112 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:21.006474 19112 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:21.006871 19112 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:21.007416 19040 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:21.008184 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:21.009125 19122 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:21.009564 19122 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:21.009943 19122 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:21.010658 19087 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.788s	user 0.012s	sys 0.002s
I20260709 13:06:21.018828 19122 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.020233 19156 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:21.021234 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:21.020948 19087 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:06:20.925748 19192 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:20.984105 19193 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:20.995231 18986 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:21.031749 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:20.998627 18996 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:21.039938 19121 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:06:21.046033 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.046214 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:21.047555 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.050938 13945 raft_consensus.cc:2410] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:21.051800 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.029752 18713 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:21.049892 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:21.051083 13914 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:21.053849 19147 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:21.055908 19192 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.050633 19041 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:21.031491 19084 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:21.064779 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.031718 18989 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:21.048703 19204 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:06:21.038725 19158 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:06:21.073107 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:21.103264 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 53511.4us
W20260709 13:06:21.112350 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.036062 19138 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:21.091614 19123 consensus_queue.cc:1059] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:06:21.049472 19156 consensus_queue.cc:1059] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:06:21.132918 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:21.129218 19158 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:21.161017 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 110038us
I20260709 13:06:21.158294 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:21.124444 19121 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:21.164839 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:21.157285 19156 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:21.143463 19041 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:21.144305 18919 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:21.126034 19084 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.170913 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:21.175814 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:21.162829 19158 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:21.168905 18749 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:21.171640 19159 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.169785 19156 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:21.187664 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:21.188318 14015 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:21.189092 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:21.191102 19191 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.209s	user 0.001s	sys 0.001s
W20260709 13:06:21.191668 19200 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:06:21.186456 18749 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:21.193295 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:21.193634 19201 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:06:21.194032 14016 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:21.193779 18749 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.195552 14016 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:06:21.195858 18749 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:21.196563 19087 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:21.182077 19210 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:06:21.179231 19123 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:06:21.202621 19159 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.215925 14016 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 13:06:21.230852 14015 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:21.238577 14015 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:06:21.241354 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 60492.9us
I20260709 13:06:21.213390 19159 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:21.229607 19123 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:21.221009 19122 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.234351 19121 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:21.236114 19210 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:21.254339 19123 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.267762 18820 raft_consensus.cc:3055] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:21.268328 18820 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:21.269542 18820 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.271217 18820 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:21.272157 19123 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:21.257839 19210 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:21.274478 19135 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.274966 19210 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.275527 14024 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:21.277135 19163 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.277361 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:21.278571 19208 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:06:21.275050 19135 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.280395 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:21.280989 19155 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.520s	user 0.015s	sys 0.000s
I20260709 13:06:21.281240 13873 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:21.281692 19135 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.276376 19210 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:21.282771 13867 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:21.296186 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.297205 13874 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:21.299710 13870 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:21.303510 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.286731 19121 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.305653 13872 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:21.306336 13874 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:21.307099 13867 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:21.307740 13871 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:21.249687 19156 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.308912 19156 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:21.309131 13869 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:21.314137 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:21.313902 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:21.315620 13874 raft_consensus.cc:2393] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:21.316723 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:21.320402 13874 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:21.328016 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:21.329875 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.322357 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.331804 13874 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:21.333248 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.328125 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.277676 19163 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.338177 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:21.340612 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:21.368160 13871 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:21.341470 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:21.343813 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.370113 13870 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
W20260709 13:06:21.319249 19159 consensus_queue.cc:1175] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e1649107c781428692458f10597c5742" responder_term: 10 status { last_received { term: 8 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: [9.7-9.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:21.347770 19023 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.349421 19143 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.350242 18940 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.350744 18744 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.354624 18781 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.355002 19148 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.355780 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:21.361871 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:21.353839 19162 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.374032 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:21.299546 19112 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:21.382129 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.385425 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.356374 19160 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:06:21.389094 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.361799 19155 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.394169 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:21.394913 18405 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:21.400923 13869 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:21.426764 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:21.435631 18492 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:21.436115 18492 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.436465 18492 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:21.437796 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:21.430611 19144 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.400733 19193 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.416836 19122 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:21.461197 14019 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:21.397233 18781 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.458401 19144 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.463583 18781 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.464269 19144 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.465359 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.441182 19214 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.466008 13944 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:06:21.443272 19140 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:21.466399 19214 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.416687 18405 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.446619 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.447036 19084 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:21.490435 13866 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:21.490743 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2878) took 1527 ms. Trace:
I20260709 13:06:21.491163 13946 rpcz_store.cc:276] 0709 13:06:19.963849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.967559 (+  3710us) service_pool.cc:224] Handling call
0709 13:06:21.490489 (+1522930us) spinlock_profiling.cc:243] Waited 61.3 ms on lock 0x7b5400091354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:21.490642 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140956544}
I20260709 13:06:21.447260 19215 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:21.492492 18405 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:21.492676 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2237) took 1202 ms. Trace:
I20260709 13:06:21.492920 19215 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.493144 14017 rpcz_store.cc:276] 0709 13:06:20.290585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.319639 (+ 29054us) service_pool.cc:224] Handling call
0709 13:06:20.343022 (+ 23383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:20.350346 (+  7324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:21.492407 (+1142061us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b5400411c54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:21.492585 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15389568}
I20260709 13:06:21.495412 19215 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.495600 18989 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.496050 19215 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:21.438846 19126 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:06:21.434818 18785 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:21.499543 18785 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.499969 18785 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:21.500847 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2248) took 1191 ms. Trace:
W20260709 13:06:21.500909 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 187127us
I20260709 13:06:21.502395 18919 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.502532 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:21.501325 14018 rpcz_store.cc:276] 0709 13:06:20.309506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.701012 (+391506us) service_pool.cc:224] Handling call
0709 13:06:20.701420 (+   408us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:20.764958 (+ 63538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:20.765000 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:20.765806 (+   806us) log.cc:844] Serialized 78 byte log entry
0709 13:06:20.994489 (+228683us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:21.030516 (+ 36027us) log.cc:844] Serialized 14 byte log entry
0709 13:06:21.030924 (+   408us) log.cc:844] Serialized 14 byte log entry
0709 13:06:21.031090 (+   166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:21.031187 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:21.497832 (+466645us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b1800374da0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:21.500587 (+  2755us) raft_consensus.cc:1685] finished
0709 13:06:21.500612 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:21.500781 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74029,"prepare.queue_time_us":145124,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":82,"raft.queue_time_us":39094,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":275,"spinlock_wait_cycles":23913984,"thread_start_us":220579,"threads_started":1,"wal-append.queue_time_us":356354}
I20260709 13:06:21.502967 18919 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.504307 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:21.505033 19083 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:21.397523 19148 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.505578 13873 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:06:21.507272 19083 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.456190 19136 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:21.392678 19143 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.507788 19083 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.509479 19143 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.445600 19142 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:21.459368 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:21.440315 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:21.392146 18940 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.491998 19158 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.395913 18744 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.399611 19162 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.432502 19155 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:21.511116 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 51822.1us
I20260709 13:06:21.496124 18989 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.497169 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:21.448532 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.469362 18492 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.519083 13838 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:21.523717 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.497803 19163 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:21.501082 19040 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:21.507541 19148 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.387914 19023 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.500910 19153 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.454697 19112 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.539777 13870 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:06:21.512234 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71985.8us
I20260709 13:06:21.499047 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2895) took 1278 ms. Trace:
I20260709 13:06:21.511591 19214 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.462163 19193 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:21.510011 19083 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:21.542856 18492 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:21.525835 13948 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:06:21.505340 19084 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:21.529613 19210 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.548087 19023 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:21.511294 19142 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:21.541293 13875 rpcz_store.cc:276] 0709 13:06:20.220208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.220347 (+   139us) service_pool.cc:224] Handling call
0709 13:06:21.498785 (+1278438us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54003acad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:21.498983 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1001022336}
I20260709 13:06:21.545869 19158 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.523871 19143 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:21.513883 18940 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.556108 13871 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:21.532377 19138 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:21.549383 13874 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:06:21.538859 19153 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.560333 19022 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:06:21.564473 18405 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.584926 18405 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.579552 18744 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.563197 18919 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:21.585496 19122 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:21.561218 19113 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:06:21.586161 18749 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:21.565842 19210 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.561877 19125 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:21.590233 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:21.590862 19163 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:21.592224 18989 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:21.599490 19162 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:21.601701 19155 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:21.655769 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 65649.9us
I20260709 13:06:21.610801 19153 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:21.564329 19084 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:21.613826 13941 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:21.670631 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:21.671033 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:21.675302 19223 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:06:21.662216 18989 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.674470 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:21.683212 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.684974 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:21.685760 13946 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:21.690196 13944 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:06:21.690665 13944 pending_rounds.cc:85] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 8
I20260709 13:06:21.690914 13944 pending_rounds.cc:107] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:06:21.691200 13944 raft_consensus.cc:2889] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:21.694545 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.695685 13950 raft_consensus.cc:2410] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:06:21.696695 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.700848 14019 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:21.662346 19148 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:21.721427 13941 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:21.676404 18989 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.737004 18893 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:21.737768 18893 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.750623 18820 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:21.751654 18820 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.751672 18980 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:21.754519 18980 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:21.755699 18980 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.757154 18980 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:21.760013 18820 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:21.760964 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.644469 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:21.646929 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:21.654026 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 109574us
I20260709 13:06:21.570256 19143 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.659096 18996 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:21.799407 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:21.799726 18996 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:21.800387 14024 raft_consensus.cc:2410] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:06:21.661566 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:21.801339 14551 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:21.799414 19143 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:21.793567 19083 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:06:21.799660 14553 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:21.651546 18749 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.777691 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:21.669312 19153 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.784878 14554 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:21.785035 14561 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:21.813987 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:21.785940 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.796023 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.797930 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.766657 14522 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:21.817239 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 50718.5us
I20260709 13:06:21.782578 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:21.822614 13941 raft_consensus.cc:2410] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:06:21.812444 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:21.812994 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.816344 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:21.823654 14021 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:06:21.810531 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:21.827754 13873 raft_consensus.cc:2410] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:06:21.811339 13914 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:21.828629 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.655094 19125 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.829469 13872 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:06:21.825057 14016 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:06:21.832743 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:21.834864 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.835892 13950 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:21.836899 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:21.840472 14553 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:21.794194 18852 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:21.638370 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 234496us
I20260709 13:06:21.872244 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:21.848287 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:21.851074 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:21.854214 13946 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:21.861913 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:21.874643 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.835090 18820 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:21.834357 18713 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:21.810348 19077 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:21.870903 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:21.890774 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:21.873817 13944 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:21.876063 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.840826 19041 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:21.873937 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.881603 13946 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:06:21.901152 13946 pending_rounds.cc:85] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 6
I20260709 13:06:21.900978 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:21.901578 13946 pending_rounds.cc:107] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:06:21.901962 13946 raft_consensus.cc:2889] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:21.881542 13949 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:06:21.902817 13949 pending_rounds.cc:85] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 7
I20260709 13:06:21.903147 13949 pending_rounds.cc:107] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:06:21.903448 13949 raft_consensus.cc:2889] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:21.904294 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.904250 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.844607 19142 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.900646 13872 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:21.881692 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:21.828848 18781 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.908051 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:21.891769 13866 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:21.879671 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:21.893953 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:21.893956 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:21.895584 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.845394 19082 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.835018 19193 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:21.841302 19040 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:21.883720 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.911194 13941 raft_consensus.cc:2410] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:06:21.911756 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.910307 14021 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:21.913765 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:21.934643 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.844971 19215 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:21.914835 18492 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:06:21.916460 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.841220 19138 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:06:21.917735 13871 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:21.905560 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:21.887506 19224 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:21.905779 19003 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:21.919176 19083 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:21.919628 19159 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:21.907120 19158 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:06:21.877131 19144 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:21.928138 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.913923 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:21.980741 13945 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:21.931905 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:21.934898 14554 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:21.869364 19110 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:21.885957 18820 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.882441 19155 raft_consensus.cc:2764] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:06:21.901965 14015 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:22.019568 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:21.943463 19219 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:21.948782 19224 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.942215 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.893039 19135 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:21.946048 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:21.906312 18777 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:21.896239 19222 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:06:21.907636 19077 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.952739 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:21.953648 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:21.887311 19122 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:06:21.954133 19143 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.933019 18781 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.947021 19215 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.898432 19084 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.914737 13868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:06:21.938534 19136 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:21.905097 19213 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:06:21.957625 18749 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:21.962136 19125 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:21.940256 19193 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:21.939752 19082 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:22.064103 13875 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:21.896409 18874 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:06:22.065246 13871 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:21.933087 19142 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:21.906807 19087 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:06:21.952809 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:21.913201 19127 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:21.982031 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 67251.2us
I20260709 13:06:21.977955 19003 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.071902 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:22.072302 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 187507us
I20260709 13:06:21.976323 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:21.834743 19153 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.024735 13943 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:22.027514 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:22.062603 19122 consensus_queue.cc:1243] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:06:22.026232 13873 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:21.974296 19159 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.890241 18852 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:22.093465 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:22.094949 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:22.049862 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:21.989732 19144 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:22.047161 14553 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:22.052245 13874 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:21.989630 19110 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:22.128441 14017 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:22.022354 18820 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:22.022892 18980 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:06:22.026973 19207 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.846s	user 0.006s	sys 0.000s
I20260709 13:06:21.970873 18785 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.060249 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:22.092689 19136 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:22.039750 19224 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:22.065838 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 171950us
I20260709 13:06:22.068022 14551 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:22.188989 13989 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:22.190763 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:22.074193 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:22.075089 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 98923us
I20260709 13:06:22.042960 19083 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:06:22.148728 18980 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:22.084515 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 57096.1us
I20260709 13:06:22.028390 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:22.089550 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:22.099184 14015 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:22.098763 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:22.198333 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 99658.6us
I20260709 13:06:22.129884 19144 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:22.102350 13868 raft_consensus.cc:606] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:06:22.204002 13868 raft_consensus.cc:614] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:22.108448 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:22.130650 19143 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.145948 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:22.211508 13949 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:22.149149 13942 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:22.088205 18853 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:22.087843 19121 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:22.153976 14552 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:22.036149 19135 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.096801 19112 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:06:22.093698 19159 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:22.182686 13947 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:06:22.225427 13947 pending_rounds.cc:85] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 8
I20260709 13:06:22.225762 13947 pending_rounds.cc:107] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:06:22.226068 13947 raft_consensus.cc:2889] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:22.128010 18777 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:22.186403 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:22.227900 14023 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:22.227790 18777 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:22.228978 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:22.229743 14023 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:22.230245 18777 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:22.230623 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:22.230865 18777 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:22.042534 19219 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:22.137763 19110 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:22.232919 19219 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:22.233453 18777 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:22.136543 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:22.234618 14021 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:22.088721 19127 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:22.148393 19222 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.092334 19158 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:06:22.215975 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:22.128520 19115 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:06:22.205188 19040 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:22.204023 18980 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:22.236198 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 162083us
I20260709 13:06:22.196249 18781 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:22.205855 19136 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:22.260329 19136 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.248651 13941 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:22.246099 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 111094us
I20260709 13:06:22.252259 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:22.206243 19041 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:22.207420 19213 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:06:22.246802 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 47673.5us
I20260709 13:06:22.297389 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:22.221246 19135 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:22.224509 19193 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:22.256431 19022 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:22.256189 19040 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:22.305020 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 112677us
I20260709 13:06:22.235842 19240 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.233971 18777 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:22.473008 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 165053us
W20260709 13:06:22.444865 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.511868 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:22.304567 13871 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:22.319113 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:22.520152 14021 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:22.297171 18781 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:22.321343 19229 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
I20260709 13:06:22.327766 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:22.551512 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:22.557247 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 82298.5us
W20260709 13:06:22.569083 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 235384us
I20260709 13:06:22.570587 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:22.324185 19155 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:22.332358 19040 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:22.486554 19193 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:22.615018 19077 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.591987 19241 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:06:22.544762 19149 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.568835 18781 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:22.617408 19149 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.619866 19149 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:22.621009 18781 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.541133 19236 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:22.567584 19083 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:22.622295 19082 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:22.622535 19083 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.623255 19083 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:22.555471 19215 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:22.566012 18405 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:06:22.630723 18785 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.553779 19240 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.636386 19240 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.636245 19135 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:22.318289 19022 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.299479 18996 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.638902 19159 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.640654 18996 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.575901 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:22.641044 19159 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.264488 19222 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:22.641922 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 66144.3us
I20260709 13:06:22.595206 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:22.581336 19155 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:22.598518 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:22.585834 13989 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:22.607355 19127 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:22.622658 19207 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:06:22.668936 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 83249.5us
I20260709 13:06:22.670192 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:22.595785 19233 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.440s	user 0.000s	sys 0.001s
I20260709 13:06:22.591841 19125 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:22.642091 19234 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:06:22.676455 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:22.654991 18785 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:22.688295 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:22.614410 18852 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:22.638965 19022 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:22.612005 19003 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:22.614418 19153 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:22.602180 19219 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:22.726938 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:22.609294 19142 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:22.606545 19158 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.645103 19239 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:22.685217 14552 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:22.667953 18989 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:22.706265 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:22.738687 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:22.739425 14018 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:22.706688 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 411001us
I20260709 13:06:22.712984 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2246) took 2404 ms. Trace:
I20260709 13:06:22.690431 19149 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
W20260709 13:06:22.700760 13945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.056s	sys 0.119s
I20260709 13:06:22.744964 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:22.745623 13945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.057s	sys 0.119s
I20260709 13:06:22.678915 19082 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:22.690995 19125 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.614763 19193 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.705277 18986 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:22.733762 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:22.701208 18785 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.745865 13946 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:22.741887 14020 rpcz_store.cc:276] 0709 13:06:20.308928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.710207 (+401279us) service_pool.cc:224] Handling call
0709 13:06:20.766330 (+ 56123us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:20.811659 (+ 45329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:20.811690 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:20.848821 (+ 37131us) log.cc:844] Serialized 52 byte log entry
0709 13:06:21.094613 (+245792us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:21.129245 (+ 34632us) log.cc:844] Serialized 14 byte log entry
0709 13:06:21.136306 (+  7061us) log.cc:844] Serialized 14 byte log entry
0709 13:06:21.136601 (+   295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:21.136726 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:21.495390 (+358664us) spinlock_profiling.cc:243] Waited 6.53 ms on lock 0x7b1800428bc0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c730758f 00007fb2c0346571 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd
0709 13:06:21.970648 (+475258us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:22.580670 (+610022us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b1800428bc0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:22.712686 (+132016us) raft_consensus.cc:1685] finished
0709 13:06:22.712719 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.712905 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64970,"prepare.queue_time_us":63692,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":136,"raft.queue_time_us":67750,"raft.run_cpu_time_us":524,"raft.run_wall_time_us":33679,"spinlock_wait_cycles":45966720,"thread_start_us":152623,"threads_started":1,"wal-append.queue_time_us":304077}
I20260709 13:06:22.727335 19219 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.606096 19041 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.704692 19143 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:06:22.622483 19236 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.729972 19238 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:22.735488 19239 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.765359 13871 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:06:22.764954 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 107484us
I20260709 13:06:22.722991 19084 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:06:22.764076 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:22.783260 14018 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:22.769411 19219 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:22.748019 19153 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:22.725283 19155 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:22.789135 14021 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:22.762660 19125 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:22.761305 18989 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.767693 18785 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:22.741405 19158 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.801107 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:22.819461 14020 raft_consensus.cc:1813] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Handling vote request from an unknown peer 3779649036364744823fc035dac892c7
W20260709 13:06:22.803622 19110 consensus_meta.cc:287] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.572s	user 0.008s	sys 0.003s
I20260709 13:06:22.821925 19110 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:22.766309 19159 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.765209 19253 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:22.801944 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:22.804766 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:22.804906 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 464213us
I20260709 13:06:22.763242 19149 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:22.786010 18777 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:22.760337 19003 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 13:06:22.814613 14015 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.613s	user 0.011s	sys 0.001s
I20260709 13:06:22.800976 13950 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:06:22.799551 19136 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:22.820370 14020 raft_consensus.cc:2410] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:06:22.702380 19235 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:06:22.801766 19239 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:22.809514 19153 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.814371 19222 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.847271 14015 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:22.852200 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:22.888562 19222 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.889084 19222 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.859579 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:22.891055 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:22.891316 19222 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.891716 13871 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:22.892730 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:22.892953 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:22.893780 13873 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:06:22.895638 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2896) took 1757 ms. Trace:
I20260709 13:06:22.896067 13943 rpcz_store.cc:276] 0709 13:06:21.138149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.138329 (+   180us) service_pool.cc:224] Handling call
0709 13:06:21.143424 (+  5095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.088367 (+944943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.895381 (+807014us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b5400087354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:22.895571 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2020323200}
I20260709 13:06:22.898988 19135 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.900681 19236 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.848897 19155 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:22.803791 19238 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.905156 19238 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:22.907233 19110 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:22.870386 19262 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.874791 19136 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:22.877389 14016 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:22.851186 19253 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.878756 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2256) took 1059 ms. Trace:
I20260709 13:06:22.856595 19149 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:22.880513 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2259) took 1059 ms. Trace:
I20260709 13:06:22.922434 14022 rpcz_store.cc:276] 0709 13:06:21.820890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.879584 (+1058694us) service_pool.cc:224] Handling call
0709 13:06:22.879959 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.880180 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.880207 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.880252 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:22.880291 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.880347 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.880454 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.886622 18996 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.889415 19249 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:22.900767 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2918) took 1028 ms. Trace:
I20260709 13:06:22.930161 13875 rpcz_store.cc:276] 0709 13:06:21.872592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.875642 (+  3050us) service_pool.cc:224] Handling call
0709 13:06:22.900554 (+1024912us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54002e81d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:22.900699 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350342528}
W20260709 13:06:22.862687 19252 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 13:06:22.864698 19255 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:06:22.915051 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:22.857684 18820 consensus_meta.cc:287] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.664s	user 0.009s	sys 0.005s
I20260709 13:06:22.916512 13871 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 13:06:22.920091 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.921066 14020 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:06:22.917021 14019 rpcz_store.cc:276] 0709 13:06:21.819833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.835389 (+ 15556us) service_pool.cc:224] Handling call
0709 13:06:21.867495 (+ 32106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.062434 (+194939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.062474 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.062524 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:22.062574 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.878489 (+815915us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54003250d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:22.878542 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.878691 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":447732480}
I20260709 13:06:22.916507 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2258) took 1096 ms. Trace:
I20260709 13:06:22.957268 14016 rpcz_store.cc:276] 0709 13:06:21.820565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.876645 (+1056080us) service_pool.cc:224] Handling call
0709 13:06:22.877004 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.916202 (+ 39198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.916437 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:22.884425 19251 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:06:22.821045 18989 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.910351 19153 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:22.892984 19158 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.954206 19247 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.989637 14016 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:06:22.995241 18893 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:22.996501 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2820) took 1325 ms. Trace:
I20260709 13:06:22.996945 14017 rpcz_store.cc:276] 0709 13:06:21.671315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.671842 (+   527us) service_pool.cc:224] Handling call
0709 13:06:21.704119 (+ 32277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.129032 (+424913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.996198 (+867166us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b540023f854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:22.996425 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":975218432}
I20260709 13:06:23.000429 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:22.991868 19224 consensus_meta.cc:287] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.779s	user 0.012s	sys 0.000s
I20260709 13:06:22.893781 19193 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:22.932536 19113 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:22.950835 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:22.951681 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:22.952364 18980 consensus_meta.cc:287] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.655s	user 0.007s	sys 0.004s
I20260709 13:06:22.952446 19083 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.952703 18785 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.968791 18820 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:22.924583 19262 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.932753 18996 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.966931 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:22.969316 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2257) took 1149 ms. Trace:
I20260709 13:06:23.099833 14020 rpcz_store.cc:276] 0709 13:06:21.820238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.920344 (+1100106us) service_pool.cc:224] Handling call
0709 13:06:22.920707 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.968943 (+ 48236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.969236 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:23.100713 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:22.970966 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:23.101395 14561 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:06:22.971619 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:22.978919 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:23.102710 13871 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:23.044613 19113 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:22.991535 19144 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.773s	user 0.009s	sys 0.003s
I20260709 13:06:23.103586 19144 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.104283 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:23.104867 19113 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.105443 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:23.056826 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:23.006273 19247 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.106406 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:22.992997 19264 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:06:23.036387 14556 raft_consensus.cc:2410] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:06:22.848484 19159 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:22.966813 19003 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
W20260709 13:06:23.039779 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 196305us
W20260709 13:06:23.128263 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.046522 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 744) took 1717 ms. Trace:
I20260709 13:06:23.051905 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 699) took 1169 ms. Trace:
I20260709 13:06:23.099206 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:23.131235 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:23.102146 13943 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:23.129626 13870 rpcz_store.cc:276] 0709 13:06:21.329166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.370944 (+ 41778us) service_pool.cc:224] Handling call
0709 13:06:21.371314 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:21.540202 (+168888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.046202 (+1506000us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540026c0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:23.046446 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386456320}
I20260709 13:06:23.056224 14522 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:23.107745 14555 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:23.108115 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:23.113126 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:23.133152 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 77017.8us
I20260709 13:06:23.133586 13869 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:23.127832 19122 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:23.042580 19158 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:23.116319 18996 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.012099 19265 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:23.168686 19265 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.169232 19265 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:23.169631 19140 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:23.118482 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:23.119380 13945 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:23.119758 19258 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:23.111972 13914 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
W20260709 13:06:23.176751 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 68097.3us
I20260709 13:06:23.171394 13950 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:23.124255 14016 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:23.129748 14560 raft_consensus.cc:2393] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:23.130089 14551 rpcz_store.cc:276] 0709 13:06:21.882816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.882937 (+   121us) service_pool.cc:224] Handling call
0709 13:06:21.883259 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.188234 (+304975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.051541 (+863307us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400530f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:23.051821 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411952896}
I20260709 13:06:23.139545 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:22.935668 14021 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:23.096164 19163 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:23.159121 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:23.097219 19253 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:23.180718 13914 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:23.064746 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:23.153122 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2457) took 1266 ms. Trace:
I20260709 13:06:23.162230 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:23.198421 13944 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:23.042304 19224 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.201596 19224 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:23.137773 19122 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.165340 19022 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.158573 19113 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.166320 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 748) took 1360 ms. Trace:
I20260709 13:06:23.158133 19112 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.117494 19239 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.108331 19014 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.172534 13988 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:23.237810 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 65372.6us
I20260709 13:06:23.055071 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:23.239217 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 184223us
I20260709 13:06:23.068446 13949 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:06:23.060950 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:23.005041 18893 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:23.202490 14023 raft_consensus.cc:2393] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:23.163408 19262 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.207142 19135 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.219591 13869 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:23.158531 19259 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:23.223575 13867 rpcz_store.cc:276] 0709 13:06:21.805719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.806079 (+   360us) service_pool.cc:224] Handling call
0709 13:06:21.806479 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.151802 (+345323us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:22.198600 (+ 46798us) log.cc:844] Serialized 14 byte log entry
0709 13:06:22.931602 (+733002us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.931664 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:22.931726 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.164813 (+233087us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b1800385840. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:23.166027 (+  1214us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54000daa54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:23.166096 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.166260 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55242,"prepare.queue_time_us":46937,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":1553,"raft.queue_time_us":90301,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"spinlock_wait_cycles":797907968,"thread_start_us":676633,"threads_started":2,"wal-append.queue_time_us":530156}
I20260709 13:06:23.047271 18980 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.177040 19264 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:23.225845 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 73022us
I20260709 13:06:23.198508 13948 rpcz_store.cc:276] 0709 13:06:21.886587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.886838 (+   251us) service_pool.cc:224] Handling call
0709 13:06:21.913756 (+ 26918us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.093860 (+180104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.093895 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.095484 (+  1589us) log.cc:844] Serialized 26 byte log entry
0709 13:06:22.542442 (+446958us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:22.542481 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.542563 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:23.139500 (+596937us) raft_consensus.cc:1685] finished
0709 13:06:23.139533 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.153038 (+ 13505us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1941,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":12663,"thread_start_us":520004,"threads_started":2,"wal-append.queue_time_us":573300}
I20260709 13:06:23.126696 19040 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.216239 18996 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:23.206529 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:23.186893 19163 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:23.295419 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:23.189414 19215 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:23.249655 13872 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:06:23.249454 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:23.243754 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:23.297824 13945 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:23.278285 14552 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:23.286713 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:23.300323 13914 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:23.300777 13867 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:23.170270 19158 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.297217 13949 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:06:23.185092 19140 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:23.291337 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 230484us
I20260709 13:06:23.304426 19162 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:23.255023 19259 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:23.334769 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:23.320421 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:23.321141 14018 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:23.321290 14020 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
W20260709 13:06:23.320501 14015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2825) took 1314 ms (client timeout 1712 ms). Trace:
I20260709 13:06:23.323846 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:23.304711 19215 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:23.404956 13942 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:23.250236 19156 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:23.250988 19224 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:23.405896 13946 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:23.275244 19214 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:23.244818 19112 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.247257 19014 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.407955 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:23.296983 19222 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:23.254304 19110 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:06:23.395402 14552 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:23.323316 19125 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.004s
I20260709 13:06:23.236672 19113 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.325714 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:23.403817 14015 rpcz_store.cc:269] 0709 13:06:22.006463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.019246 (+ 12783us) service_pool.cc:224] Handling call
0709 13:06:23.320301 (+1301055us) spinlock_profiling.cc:243] Waited 78.9 ms on lock 0x7b540030cad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:23.320436 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181383424}
I20260709 13:06:23.315718 19247 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.254701 18893 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.334427 18744 raft_consensus.cc:3055] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:23.322721 18749 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:23.278347 19135 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.395653 19149 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:23.391947 19259 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:23.418365 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:23.419994 19140 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:23.421972 18980 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:23.422736 19274 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:06:23.413672 19083 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:23.431043 19239 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:23.432210 19022 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.318945 19267 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 13:06:23.405397 19087 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:23.434558 19219 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.632s	user 0.008s	sys 0.004s
I20260709 13:06:23.407799 19215 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:23.416164 19222 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.425163 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2960) took 1350 ms. Trace:
W20260709 13:06:23.556620 19272 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:06:23.439332 19112 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.439715 19125 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.438174 19265 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.440860 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:23.574611 14554 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 17: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:23.443289 19210 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:23.450760 14560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 876) took 1673 ms (client timeout 2058 ms). Trace:
I20260709 13:06:23.453394 19144 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:23.455643 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2264) took 1594 ms. Trace:
I20260709 13:06:23.456393 19014 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.457233 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57836 (request call id 18) took 1616 ms. Trace:
I20260709 13:06:23.578258 13868 rpcz_store.cc:276] 0709 13:06:21.840575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.841321 (+   746us) service_pool.cc:224] Handling call
0709 13:06:23.457002 (+1615681us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b54002e5ed4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c342d853 00007fb2c83fb60a 00007fb2c19817d4 00007fb2c1981762 00007fb2c1981691 00007fb2c198160c 00007fb2c19808c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:23.457123 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112300928}
I20260709 13:06:23.461987 19040 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:23.461876 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:23.474156 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 153823us
I20260709 13:06:23.397433 19142 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:06:23.411108 18989 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:23.483199 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 157619us
W20260709 13:06:23.460497 19155 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.000s
I20260709 13:06:23.462664 19224 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:23.587373 19224 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.482863 18744 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:23.499631 14558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 700) took 1611 ms. Trace:
I20260709 13:06:23.557629 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2964) took 1351 ms. Trace:
W20260709 13:06:23.558871 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 426823us
I20260709 13:06:23.416651 19214 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.560122 14551 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:06:23.561287 13867 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:23.483445 19021 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:23.564178 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2926) took 1237 ms. Trace:
I20260709 13:06:23.482167 19263 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:23.413407 19136 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:23.482687 18785 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.514726 19122 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:23.567835 13873 rpcz_store.cc:276] 0709 13:06:22.074818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.894362 (+819544us) service_pool.cc:224] Handling call
0709 13:06:22.894691 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.935022 (+ 40331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.935056 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.954797 (+ 19741us) log.cc:844] Serialized 26 byte log entry
0709 13:06:23.137007 (+182210us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:23.230712 (+ 93705us) log.cc:844] Serialized 14 byte log entry
0709 13:06:23.305170 (+ 74458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.305256 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:23.424892 (+119636us) raft_consensus.cc:1685] finished
0709 13:06:23.424917 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.425089 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":196117,"prepare.queue_time_us":93784,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":159963,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":27903,"thread_start_us":138432,"threads_started":1,"wal-append.queue_time_us":298958}
I20260709 13:06:23.557163 19159 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.515756 18749 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:23.576761 14023 rpcz_store.cc:276] 0709 13:06:21.870902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.230363 (+359461us) service_pool.cc:224] Handling call
0709 13:06:23.455401 (+1225038us) spinlock_profiling.cc:243] Waited 971 ms on lock 0x7b54002ea754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:23.455569 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2232901888}
W20260709 13:06:23.576116 14560 rpcz_store.cc:269] 0709 13:06:21.777406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.861333 (+ 83927us) service_pool.cc:224] Handling call
0709 13:06:23.450411 (+1589078us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54003372d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:23.450633 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2877666176}
I20260709 13:06:23.590374 14558 rpcz_store.cc:276] 0709 13:06:21.888054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.888527 (+   473us) service_pool.cc:224] Handling call
0709 13:06:21.920291 (+ 31764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.294793 (+374502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.294827 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.294874 (+    47us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:22.294910 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.499329 (+1204419us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54004b0fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:23.499384 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.499551 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860635392}
I20260709 13:06:23.458490 14017 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:23.592824 19214 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.605625 19265 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:23.583451 19239 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.606319 18749 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.606714 19239 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.572269 19219 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.594026 13838 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:23.593832 18893 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:23.488147 19153 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.613104 13875 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:06:23.563275 18820 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.581211 13839 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:23.590548 13870 rpcz_store.cc:276] 0709 13:06:22.205788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.149831 (+944043us) service_pool.cc:224] Handling call
0709 13:06:23.153486 (+  3655us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:23.179882 (+ 26396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:23.179919 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:23.218748 (+ 38829us) log.cc:844] Serialized 26 byte log entry
0709 13:06:23.409567 (+190819us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:23.409617 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.409709 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:23.500471 (+ 90762us) raft_consensus.cc:1685] finished
0709 13:06:23.500497 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.557533 (+ 57036us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56766,"raft.queue_time_us":26076,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":19580,"thread_start_us":128769,"threads_started":1,"wal-append.queue_time_us":253137}
I20260709 13:06:23.601037 13869 rpcz_store.cc:276] 0709 13:06:22.327091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.561389 (+1234298us) service_pool.cc:224] Handling call
0709 13:06:23.561752 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.563763 (+  2011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:23.563807 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:23.563860 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:23.563904 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.563980 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.564116 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:23.625943 13867 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:06:23.591095 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:23.592764 13948 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:06:23.598737 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2260) took 1777 ms. Trace:
I20260709 13:06:23.594122 19022 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:06:23.576427 19112 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:23.627107 14022 rpcz_store.cc:276] 0709 13:06:21.821193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.923745 (+1102552us) service_pool.cc:224] Handling call
0709 13:06:22.928072 (+  4327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.928482 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.928514 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.936275 (+  7761us) log.cc:844] Serialized 26 byte log entry
0709 13:06:23.036655 (+100380us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:23.036751 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.036863 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:23.598456 (+561593us) raft_consensus.cc:1685] finished
0709 13:06:23.598482 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.598655 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":35752,"thread_start_us":46447,"threads_started":1,"wal-append.queue_time_us":212615}
W20260709 13:06:23.640851 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 49835.7us
I20260709 13:06:23.572762 19087 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.642225 14524 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:23.590013 19224 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:23.584424 19110 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:23.614802 18893 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:23.648840 19159 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:23.651399 18893 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:23.651969 18893 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.651886 19159 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.654497 19159 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:23.631043 19122 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:23.658915 19122 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.659719 19153 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:23.603140 19021 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.612851 19219 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.609788 19239 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:23.709647 19110 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:23.592559 19125 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:23.699172 13944 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:23.715245 13944 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:23.590869 18744 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.703130 19021 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:23.660674 19128 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:23.701913 18785 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:23.706951 19135 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.951880 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:23.942111 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2965) took 1734 ms. Trace:
I20260709 13:06:23.948957 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:23.967669 13872 rpcz_store.cc:276] 0709 13:06:22.207894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.185710 (+977816us) service_pool.cc:224] Handling call
0709 13:06:23.186720 (+  1010us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:23.296478 (+109758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.941669 (+645191us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b54000a8f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:23.941989 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144122624}
I20260709 13:06:23.731086 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2272) took 1472 ms. Trace:
I20260709 13:06:23.969213 14020 rpcz_store.cc:276] 0709 13:06:22.259204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.100834 (+841630us) service_pool.cc:224] Handling call
0709 13:06:23.321094 (+220260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.403450 (+ 82356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.730870 (+327420us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b540030cad0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:23.731014 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":505801088}
W20260709 13:06:23.970984 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 328909us
I20260709 13:06:23.943246 19122 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.725530 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:23.731860 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2270) took 1755 ms. Trace:
I20260709 13:06:23.971915 14551 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:06:23.726652 13868 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:23.975643 14551 raft_consensus.cc:606] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
W20260709 13:06:23.976413 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 395289us
I20260709 13:06:23.724989 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:23.977921 13839 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:23.716239 14021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2822) took 1710 ms (client timeout 2174 ms). Trace:
W20260709 13:06:23.979169 14021 rpcz_store.cc:269] 0709 13:06:22.005415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.556192 (+550777us) service_pool.cc:224] Handling call
0709 13:06:23.716053 (+1159861us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54002eac54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:23.716165 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329275776}
I20260709 13:06:23.632035 18980 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:23.982190 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:23.983896 13915 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.712s (SENT) [suppressed 10 similar messages]
I20260709 13:06:23.973016 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:23.723315 19041 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:23.986773 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:23.972179 19258 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:23.715499 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2828) took 1510 ms. Trace:
I20260709 13:06:23.976667 14551 raft_consensus.cc:614] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:23.974510 14018 rpcz_store.cc:276] 0709 13:06:21.977045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.801687 (+824642us) service_pool.cc:224] Handling call
0709 13:06:23.731644 (+929957us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b540020d0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:23.731793 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1097542400}
I20260709 13:06:24.007392 13941 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:24.010571 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:23.641822 19158 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:23.934350 19084 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:23.732499 18940 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:24.063781 13872 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:06:24.064898 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 48662us
I20260709 13:06:24.016152 13839 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:24.007071 18744 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:23.748224 18820 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:23.973451 19128 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.729038 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:23.922199 19237 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:24.074913 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.075687 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.101529 13868 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:24.079095 14016 rpcz_store.cc:276] 0709 13:06:22.205067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.000026 (+794959us) service_pool.cc:224] Handling call
0709 13:06:23.012629 (+ 12603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.180288 (+167659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.715225 (+534937us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54000ff5d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:23.715433 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256216448}
W20260709 13:06:24.082206 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 66142us
I20260709 13:06:23.744843 19266 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:24.088881 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:23.982333 19239 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:24.072971 18785 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:24.100186 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2827) took 2080 ms. Trace:
I20260709 13:06:24.008671 19122 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:24.077075 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 2963) took 1875 ms. Trace:
I20260709 13:06:24.012945 19041 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:24.103113 13839 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:24.067461 19265 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:24.142378 18492 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:23.995688 19286 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.153s
I20260709 13:06:24.074834 19149 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:23.950039 19280 consensus_queue.cc:1059] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.139s
W20260709 13:06:24.159284 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.006618 19253 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:24.015779 19224 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.108798 13872 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:24.080741 19123 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.108460 19266 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:24.109813 19285 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:06:24.080941 18940 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:24.094636 19073 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:24.087548 19084 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.110347 19237 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.094901 18919 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:06:24.111984 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.113998 19273 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:24.092222 19269 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:24.104283 19128 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:23.646710 19112 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:23.718640 19156 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:24.125344 14017 rpcz_store.cc:276] 0709 13:06:22.019938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.997984 (+978046us) service_pool.cc:224] Handling call
0709 13:06:23.458279 (+460295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.605216 (+146937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.099937 (+494721us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b54002ea750. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:24.100115 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057250688}
I20260709 13:06:24.092656 19113 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:24.066349 18980 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.133699 13875 rpcz_store.cc:276] 0709 13:06:22.202126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.113822 (+911696us) service_pool.cc:224] Handling call
0709 13:06:23.114227 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:23.619539 (+505312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.076779 (+457240us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b540026cfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:24.077009 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146699520}
I20260709 13:06:24.102749 13867 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:24.207831 13872 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
W20260709 13:06:24.066566 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 56122.6us
I20260709 13:06:24.103698 19136 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.021981 19259 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.154644 13838 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:24.223667 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:24.226816 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 72283.8us
I20260709 13:06:24.227974 13838 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:24.140475 19122 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.229080 13838 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:24.162856 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:24.162432 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:24.231132 13941 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:06:24.232031 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:24.172065 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.127844 18986 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:24.183876 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:24.212458 18893 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.560s	user 0.013s	sys 0.001s
I20260709 13:06:24.140945 18785 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:24.120616 19126 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:06:24.128106 19239 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.207765 19073 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.274080 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:24.218562 19084 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:24.233692 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.230772 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:24.276377 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:24.277189 13944 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:24.235463 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:24.225782 13868 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:24.280470 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:24.261883 19271 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:06:24.181649 19253 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.186019 19224 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.285002 19239 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:24.193801 19123 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.224687 19222 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:24.249281 13871 raft_consensus.cc:2393] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 19: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:24.249087 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:24.189006 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:24.325022 13948 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:06:24.252481 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:24.252825 19122 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:24.253189 19262 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:24.253593 19023 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:24.255484 19153 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:24.256300 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2934) took 1454 ms. Trace:
I20260709 13:06:24.270817 18893 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.258180 19113 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:24.259652 18492 consensus_queue.cc:1059] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:24.260145 13838 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:24.260859 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2485) took 1131 ms. Trace:
I20260709 13:06:24.210142 19269 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:24.208343 19273 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:24.209537 19149 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:24.271960 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:24.287954 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 304079us
W20260709 13:06:24.265240 18996 consensus_meta.cc:287] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.961s	user 0.013s	sys 0.000s
I20260709 13:06:24.334494 19262 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:24.230710 19265 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.313184 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:24.285833 19239 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.319018 14555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:06:24.322314 19123 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:24.285020 19253 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:24.335898 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 86908.1us
I20260709 13:06:24.391301 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:24.391999 14018 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:24.250344 19136 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.350806 19269 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:24.392419 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.394266 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:24.395092 14022 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:24.395525 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.396427 14020 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:24.398236 14015 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:06:24.252507 18980 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:24.401690 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.349699 13866 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:06:24.248987 18940 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.247476 13867 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
W20260709 13:06:24.348156 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 88099.9us
I20260709 13:06:24.409373 14018 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:24.348631 19153 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.364082 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:24.262331 18986 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.370666 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.362295 19234 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:24.376559 19073 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:24.308164 19266 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.378664 19224 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:24.379191 18996 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:24.380648 18893 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:24.342512 19023 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.341698 13874 rpcz_store.cc:276] 0709 13:06:22.801712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.585277 (+783565us) service_pool.cc:224] Handling call
0709 13:06:23.585705 (+   428us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:23.599931 (+ 14226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:23.599969 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:23.607101 (+  7132us) log.cc:844] Serialized 52 byte log entry
0709 13:06:23.778978 (+171877us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:23.939222 (+160244us) log.cc:844] Serialized 14 byte log entry
0709 13:06:23.977168 (+ 37946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.977251 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:24.255992 (+278741us) raft_consensus.cc:1685] finished
0709 13:06:24.256018 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.256229 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111221,"prepare.queue_time_us":160212,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":52151,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":20859,"thread_start_us":171313,"threads_started":1,"wal-append.queue_time_us":565249}
I20260709 13:06:24.443300 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:24.399269 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.399735 14015 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:06:24.400295 19293 log.cc:927] Time spent T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:06:24.401221 19237 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:24.216152 19156 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:24.348297 13947 rpcz_store.cc:276] 0709 13:06:23.129373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.614497 (+485124us) service_pool.cc:224] Handling call
0709 13:06:23.614797 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:23.615122 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:23.615153 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:23.615649 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 13:06:23.978314 (+362665us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:23.978376 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.978557 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:24.260589 (+282032us) raft_consensus.cc:1685] finished
0709 13:06:24.260613 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.260786 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1133,"raft.queue_time_us":6653,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":8104,"thread_start_us":362166,"threads_started":1,"wal-append.queue_time_us":527025}
I20260709 13:06:24.409109 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.430327 19234 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:24.384361 19262 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:24.450754 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.371810 19143 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:24.417178 14020 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:24.462667 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:24.464138 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:24.445562 19266 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:24.426391 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62415.3us
I20260709 13:06:24.422238 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.468631 14018 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:24.470388 13988 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:24.470649 19299 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.471148 19299 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.471740 13988 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:24.473398 19299 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:24.429307 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.422020 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:24.430622 14022 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:24.390136 19265 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.396493 19239 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.447160 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:24.484328 14015 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:24.452138 19234 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:24.448931 14017 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:06:24.486621 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:24.364001 19113 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:24.386394 18785 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.424033 19297 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:24.493283 19136 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:24.434455 19149 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.493542 19297 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:24.494925 19021 consensus_meta.cc:287] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.001s
I20260709 13:06:24.495456 19021 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.495970 19112 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:24.435128 18980 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:06:24.484827 19292 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.166s	user 0.001s	sys 0.001s
I20260709 13:06:24.374960 19041 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.498158 19041 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:24.450601 19073 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:24.499013 19073 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.421203 19273 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:24.500679 19215 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.924s	user 0.013s	sys 0.000s
I20260709 13:06:24.442126 19224 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:24.501364 19073 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:24.501173 19215 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.454649 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.464792 14020 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:24.348112 18492 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:24.506287 18492 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:24.507236 18492 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.503494 13945 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:24.508271 18492 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:24.477218 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.510277 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2483) took 1442 ms. Trace:
I20260709 13:06:24.479450 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.428229 18940 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:24.510735 13949 rpcz_store.cc:276] 0709 13:06:23.068326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.399286 (+330960us) service_pool.cc:224] Handling call
0709 13:06:23.414720 (+ 15434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:23.576037 (+161317us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:23.618441 (+ 42404us) log.cc:844] Serialized 14 byte log entry
0709 13:06:24.075491 (+457050us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:24.081359 (+  5868us) log.cc:844] Serialized 26 byte log entry
0709 13:06:24.081546 (+   187us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:24.081579 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.081672 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:24.501962 (+420290us) raft_consensus.cc:1685] finished
0709 13:06:24.501985 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.510204 (+  8219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2657,"prepare.queue_time_us":27293,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":27726,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":18635,"thread_start_us":605475,"threads_started":2,"wal-append.queue_time_us":604282}
W20260709 13:06:24.513396 19253 consensus_queue.cc:1175] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3779649036364744823fc035dac892c7" responder_term: 17 status { last_received { term: 17 index: 14 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:24.496953 18980 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:24.462592 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 2939) took 1233 ms. Trace:
I20260709 13:06:24.445786 19023 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:24.463457 19143 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:24.507645 19286 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:24.492345 19122 consensus_queue.cc:1059] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:06:24.476564 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 54611.2us
I20260709 13:06:24.544945 13941 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:24.451045 19259 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:24.515655 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.549495 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.550594 14017 raft_consensus.cc:2393] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:24.523568 14020 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:24.526135 14015 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:24.529717 13947 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 9.
I20260709 13:06:24.528541 13873 rpcz_store.cc:276] 0709 13:06:23.228743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.620727 (+391984us) service_pool.cc:224] Handling call
0709 13:06:23.621176 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:23.621683 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:23.621728 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:23.648209 (+ 26481us) log.cc:844] Serialized 26 byte log entry
0709 13:06:24.017042 (+368833us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:24.017092 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.017167 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:24.462167 (+445000us) raft_consensus.cc:1685] finished
0709 13:06:24.462204 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.462500 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1865,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":10111,"thread_start_us":278510,"threads_started":1,"wal-append.queue_time_us":445916}
I20260709 13:06:24.531862 19021 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:24.530932 14016 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:06:24.483783 19237 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:24.483776 19299 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:24.552948 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.557654 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.552948 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.558452 13873 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:24.558576 14020 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:24.559443 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.561082 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:24.532538 18980 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:24.565946 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.566778 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.564090 13949 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:06:24.567080 13873 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:24.568244 14020 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:06:24.568576 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.569942 19122 raft_consensus.cc:3055] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:24.570427 19122 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:24.571435 19122 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.572822 19122 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:24.579473 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.580416 14020 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:24.544858 18986 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:24.539055 19300 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:24.587507 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.592706 13948 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:24.556989 19237 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:24.593940 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.598344 14017 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:24.591861 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.591307 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:24.561589 19299 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:24.599098 14020 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:24.599114 19299 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:24.600219 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:24.601584 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:24.602349 14020 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 17: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:24.605353 19286 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:24.596954 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:24.608428 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.613415 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2912) took 1484 ms. Trace:
I20260709 13:06:24.614012 13946 rpcz_store.cc:276] 0709 13:06:23.129046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.611843 (+482797us) service_pool.cc:224] Handling call
0709 13:06:24.118328 (+506485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.118555 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:24.118586 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:24.118636 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:24.118678 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.118741 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.613137 (+494396us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b54003b01d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:24.613314 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1163991424}
I20260709 13:06:24.616120 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.617620 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:24.618299 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.618829 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.618991 14015 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:24.620105 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.621198 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.621557 14015 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:24.623584 14017 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:24.623939 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.624607 13868 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:24.624929 14017 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:06:24.625631 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:24.626127 14017 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:24.520202 19156 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.629611 13869 raft_consensus.cc:2368] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:06:24.621785 13870 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:24.628557 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:24.605850 19286 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:24.667022 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.664163 13868 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 17: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:24.596560 19280 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:24.687299 19286 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:24.607266 19237 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:24.699787 19115 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:24.701155 19153 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:24.665932 18893 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:24.703246 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2268) took 2774 ms. Trace:
I20260709 13:06:24.696585 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.708606 13868 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: [9.8-12.10]
W20260709 13:06:24.709861 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.723995 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.695696 19280 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:24.718957 14019 rpcz_store.cc:276] 0709 13:06:21.929130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.957092 (+1027962us) service_pool.cc:224] Handling call
0709 13:06:22.976809 (+ 19717us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:23.924810 (+948001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:23.924858 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:23.934398 (+  9540us) log.cc:844] Serialized 52 byte log entry
0709 13:06:24.355813 (+421415us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:24.562766 (+206953us) log.cc:844] Serialized 14 byte log entry
0709 13:06:24.563055 (+   289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.702601 (+139546us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b540023cb54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:24.702696 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:24.702775 (+    79us) raft_consensus.cc:1685] finished
0709 13:06:24.702915 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.703157 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":272755,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":60810,"raft.run_cpu_time_us":482,"raft.run_wall_time_us":49763,"spinlock_wait_cycles":1771808640,"thread_start_us":539829,"threads_started":2,"wal-append.queue_time_us":429485}
I20260709 13:06:24.687987 13870 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:06:24.717085 19115 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.725972 14017 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:06:24.690430 13869 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:24.690845 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:24.749521 13868 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 13:06:24.683947 19295 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:06:24.607721 19143 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.751760 13869 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:24.759341 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:24.760341 14019 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:24.738272 19280 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:24.707904 19269 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:24.688627 18996 consensus_queue.cc:1175] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" responder_term: 11 status { last_received { term: 11 index: 8 } last_committed_idx: 7 last_received_current_leader { term: 11 index: 8 } }
I20260709 13:06:24.608331 19113 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.733889 19239 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:24.748538 19115 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:24.722823 18893 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.783129 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.778525 13872 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:24.787837 13875 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:06:24.802037 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:24.803025 14553 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:24.808112 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.785094 19215 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:24.812103 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 183537us
I20260709 13:06:24.812897 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.816380 14019 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:24.814486 14021 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:24.818665 14024 raft_consensus.cc:2368] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14.
I20260709 13:06:24.824194 13870 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:06:24.824321 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.821062 13915 leader_election.cc:400] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:06:24.825687 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:24.827237 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:24.835112 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:24.840211 19112 consensus_queue.cc:1059] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:06:24.843751 19112 raft_consensus.cc:3055] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:24.843616 19222 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:24.844200 19112 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:24.844682 19222 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:24.844795 18893 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:24.845067 19289 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.201s	user 0.005s	sys 0.006s
I20260709 13:06:24.845171 19112 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:24.846503 19128 consensus_meta.cc:287] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.612s	user 0.013s	sys 0.000s
I20260709 13:06:24.846874 19112 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:24.846949 19128 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.849440 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:24.850683 13838 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:24.853404 19126 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:24.853580 19110 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:24.853948 19126 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:24.854959 19126 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.854959 14555 raft_consensus.cc:606] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:24.856750 14555 raft_consensus.cc:614] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:24.856805 19128 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:24.866129 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:24.869571 19307 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:06:24.872326 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:24.874130 19310 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:06:24.845091 19222 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:24.877043 19158 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:24.876780 19222 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.877522 19158 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:24.878424 19023 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:24.881295 19023 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:24.882668 13873 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:24.878985 19297 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:24.881976 19306 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:06:24.885784 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2909) took 1757 ms. Trace:
I20260709 13:06:24.886224 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:24.840647 19215 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:24.893852 14553 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:24.889348 13942 rpcz_store.cc:276] 0709 13:06:23.128208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.607533 (+479325us) service_pool.cc:224] Handling call
0709 13:06:23.607937 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:23.608343 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:23.608374 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:23.608857 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:06:23.644129 (+ 35272us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:23.644175 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.644263 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:24.844427 (+1200164us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:24.885483 (+ 41056us) raft_consensus.cc:1685] finished
0709 13:06:24.885507 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.885715 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1842,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":15197056,"thread_start_us":34899,"threads_started":1,"wal-append.queue_time_us":35250}
I20260709 13:06:24.906639 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:24.907698 13949 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:24.909446 19314 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:24.914366 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:24.915416 14021 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:24.917476 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.918958 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:24.887660 19222 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:24.921489 14024 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:24.923745 19282 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:24.924394 19282 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:24.928905 19280 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:24.934312 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 75431.7us
I20260709 13:06:24.939769 13949 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:24.955808 14021 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:24.935899 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:24.941800 13838 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:24.865164 19299 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:24.961855 19314 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:24.962452 19314 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.962881 19314 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:24.976992 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:24.977250 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:24.978645 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:24.979578 19087 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.019198 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:24.893179 19110 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:06:25.019965 14019 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:24.977249 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:25.020980 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:25.021483 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44334us
W20260709 13:06:24.959216 19306 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:06:25.022936 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 81274.2us
I20260709 13:06:25.023753 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:25.026669 14021 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 8.
I20260709 13:06:25.027012 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:25.022307 14522 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:24.980945 19318 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:24.878466 19158 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.036867 13947 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:24.925580 19282 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:25.018328 19149 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:25.047472 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.047163 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:24.920226 19163 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.055804 19299 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:25.055989 19163 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:25.039628 19087 consensus_queue.cc:1059] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:06:25.063371 19273 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:25.064496 19312 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:06:25.066582 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:25.067289 14020 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:25.022747 19128 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:25.027348 18785 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:25.027140 19239 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.039892 19259 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:25.069131 13948 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:25.074936 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2482) took 2007 ms. Trace:
W20260709 13:06:25.076146 19315 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.098s	user 0.000s	sys 0.004s
I20260709 13:06:25.077446 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:25.092453 14017 raft_consensus.cc:2376] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 13. Re-sending same reply.
I20260709 13:06:25.093582 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:25.094955 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:25.095674 14017 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 9: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:25.096422 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:25.097424 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:25.097747 18980 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.001s
W20260709 13:06:25.061766 19308 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:06:25.085247 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.114465 13944 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:25.044814 19319 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 13:06:25.107707 19324 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.145s	user 0.000s	sys 0.005s
I20260709 13:06:25.040591 19269 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:25.093642 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:25.099573 19280 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:25.082154 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.122814 13942 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:06:25.065903 19234 consensus_meta.cc:287] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.559s	user 0.004s	sys 0.005s
I20260709 13:06:25.101174 19286 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.087600 19297 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:25.102536 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:25.103132 19321 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:25.087915 19259 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:25.115701 19041 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.617s	user 0.013s	sys 0.000s
I20260709 13:06:25.093014 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.126991 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.128005 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.128767 13948 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:25.088101 13943 rpcz_store.cc:276] 0709 13:06:23.068021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.166457 (+ 98436us) service_pool.cc:224] Handling call
0709 13:06:23.166837 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:23.170587 (+  3750us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:23.951843 (+781256us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b18003a9660. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c6e2b3c7 00007fb2c343d9a7 00007fb2c344d0bf 00007fb2c344d038
0709 13:06:24.357409 (+405566us) log.cc:844] Serialized 14 byte log entry
0709 13:06:24.605845 (+248436us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:24.685611 (+ 79766us) log.cc:844] Serialized 26 byte log entry
0709 13:06:24.685905 (+   294us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:24.685967 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.686112 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:25.074617 (+388505us) raft_consensus.cc:1685] finished
0709 13:06:25.074738 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:25.074869 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8992,"prepare.queue_time_us":258268,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":17040,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":22133,"spinlock_wait_cycles":34721536,"thread_start_us":586241,"threads_started":2,"wal-append.queue_time_us":407752}
W20260709 13:06:25.093266 19266 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.625s	user 0.011s	sys 0.001s
I20260709 13:06:25.112799 13949 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:25.131467 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:25.115408 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:25.086645 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2464921009999443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2215s for a response
W20260709 13:06:25.132759 13840 connection.cc:256] RPC call timeout handler was delayed by -1.1096921009999505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23670000000000002s for a response
I20260709 13:06:25.115685 14555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:06:25.087364 19110 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:25.046537 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:25.092032 18785 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:25.124238 14017 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:25.092072 19239 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:25.158635 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 112193us
I20260709 13:06:25.134357 14023 raft_consensus.cc:2410] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:06:25.134380 14555 raft_consensus.cc:606] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:25.125581 19321 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:25.134946 13874 raft_consensus.cc:2410] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:25.119496 19112 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:25.090919 19149 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:25.092692 19282 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:25.140470 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:25.127255 19123 raft_consensus.cc:3055] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:25.135941 19110 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:25.137660 19262 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.669s	user 0.001s	sys 0.011s
I20260709 13:06:25.145081 19280 raft_consensus.cc:2764] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:06:25.145561 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 900) took 1825 ms. Trace:
I20260709 13:06:25.151276 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 760) took 1136 ms. Trace:
I20260709 13:06:25.178211 13875 rpcz_store.cc:276] 0709 13:06:24.014842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.236954 (+222112us) service_pool.cc:224] Handling call
0709 13:06:25.151087 (+914133us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54004510d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:25.151209 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976558336}
I20260709 13:06:25.154533 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.125473 19234 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:25.155678 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:25.190589 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.156577 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.156227 19115 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.156220 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:25.126734 19297 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.132539 19259 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:25.129915 13948 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:25.114342 18980 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.159579 14555 raft_consensus.cc:614] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:25.134774 13944 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:25.159789 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:25.201632 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.176033 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:25.155380 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:25.193027 19122 consensus_meta.cc:287] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.620s	user 0.014s	sys 0.002s
I20260709 13:06:25.183372 19149 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:25.183656 19110 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:25.184489 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:25.184587 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.177929 13950 rpcz_store.cc:276] 0709 13:06:23.319764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.082218 (+762454us) service_pool.cc:224] Handling call
0709 13:06:24.105325 (+ 23107us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.234359 (+129034us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:24.234393 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:24.234459 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:24.234494 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.145275 (+910781us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400091d54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:25.145331 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:25.145494 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296211328}
I20260709 13:06:25.140434 13947 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:25.188717 19237 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:25.182387 19262 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.177412 19123 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:25.208801 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.220676 13914 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:25.205336 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:25.203060 13943 raft_consensus.cc:2410] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:06:25.202821 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 165886us
I20260709 13:06:25.167368 19314 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.218936 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.204222 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:25.167150 19041 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:06:25.238687 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.193879 19126 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:25.203650 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 43934.8us
I20260709 13:06:25.242172 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:25.204010 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 70728.4us
W20260709 13:06:25.243299 13840 connection.cc:256] RPC call timeout handler was delayed by -1.0514538440000933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
I20260709 13:06:25.246249 13867 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:25.206725 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:25.177525 19112 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.226568 13946 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:25.226533 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:25.222038 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:25.252038 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.252146 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:25.231577 13944 raft_consensus.cc:2410] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:06:25.252848 13869 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:06:25.254208 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.255700 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:25.256986 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.240613 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:25.231755 19328 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.258785 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:25.211241 19280 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:25.260036 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:25.250221 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:25.261125 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.211302 19299 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:25.260037 13871 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:25.247877 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:25.266045 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:25.266337 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.269249 13872 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:06:25.270215 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.270488 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:25.271569 13872 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:25.264065 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:25.276057 13867 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:25.279809 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:25.280427 13944 raft_consensus.cc:2410] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:06:25.232475 19110 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:25.286013 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:25.286916 13944 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:25.286926 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:25.232573 19087 consensus_queue.cc:1175] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" responder_term: 13 status { last_received { term: 12 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 12 index: 11 } }
I20260709 13:06:25.242543 19122 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:25.294095 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:25.260331 19149 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:25.296289 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:25.294552 14555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3f6783295084924801a104ff3faf10e"
dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:47240
I20260709 13:06:25.302444 14555 raft_consensus.cc:606] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:25.302556 13872 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:25.302829 14555 raft_consensus.cc:614] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:25.262255 19222 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.307277 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.308238 13950 raft_consensus.cc:2410] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:06:25.308739 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:25.308995 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:25.309505 13872 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:25.297166 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.267004 19299 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:25.297986 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:25.212085 19321 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.285494 19124 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:06:25.297879 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 67148.8us
I20260709 13:06:25.318884 18492 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:25.299072 19286 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.297042 19280 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:25.318132 13870 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
I20260709 13:06:25.326737 13840 leader_election.cc:400] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14.
I20260709 13:06:25.327131 13947 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:25.327320 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:25.321695 14522 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:25.329756 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.233050 18785 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.344219 13869 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:25.345065 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2971) took 1180 ms. Trace:
I20260709 13:06:25.345578 13874 rpcz_store.cc:276] 0709 13:06:24.164961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.443004 (+278043us) service_pool.cc:224] Handling call
0709 13:06:25.344841 (+901837us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54002e86d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:25.344998 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588572544}
I20260709 13:06:25.349023 13874 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:06:25.332109 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:25.314244 19269 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:25.322827 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:25.246034 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:25.314448 19299 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:25.370965 19148 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.371502 19148 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.310274 19122 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:25.374840 19122 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:25.298369 19123 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.379745 19123 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:25.381695 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.353724 13869 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:25.335965 19325 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:25.306653 19270 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:06:25.296545 18852 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:06:25.355405 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2972) took 1184 ms. Trace:
W20260709 13:06:25.405666 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 73645.8us
I20260709 13:06:25.405795 13873 rpcz_store.cc:276] 0709 13:06:24.171462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.663489 (+492027us) service_pool.cc:224] Handling call
0709 13:06:25.355160 (+691671us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54002e72d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:25.355339 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320641280}
I20260709 13:06:25.356778 19237 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:25.357002 19115 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.357975 18980 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.359330 19112 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.314766 19222 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.416458 19222 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.361763 19314 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.280973 19328 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:25.420352 19328 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.364409 19234 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.364531 19262 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.365631 19041 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:25.370286 19239 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:25.398267 19084 raft_consensus.cc:3055] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:25.323202 13944 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:25.398384 13870 raft_consensus.cc:2410] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 13:06:25.353067 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 146853us
I20260709 13:06:25.283439 19126 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:25.318429 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:25.402904 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 138949us
I20260709 13:06:25.407161 13840 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:25.318974 19326 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:06:25.429478 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 181724us
W20260709 13:06:25.434242 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.432403 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:25.436113 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:25.410353 19273 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:25.436985 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:25.437887 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.443279 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:25.443910 14018 raft_consensus.cc:3060] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:25.438529 14019 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:25.433530 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:25.458303 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:25.461912 13868 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:25.466691 13915 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:25.465013 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 115943us
I20260709 13:06:25.468719 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:25.469017 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:25.468737 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.473001 13946 raft_consensus.cc:2410] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:06:25.438004 19125 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:25.429787 19329 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:25.422775 19041 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.430313 19084 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:25.437307 18874 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.479357 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:25.434135 19334 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:06:25.503532 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.511106 18785 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:25.512674 13948 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:25.474130 19110 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:25.514485 19110 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:25.515496 19110 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.518476 19280 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:25.518393 18874 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.468592 19326 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:25.520275 19326 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:25.522377 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 50032.3us
I20260709 13:06:25.467063 19234 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:25.504952 19329 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.525621 19329 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:25.468154 14018 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:25.507714 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:25.463140 19149 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:25.550906 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:06:25.523552 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 44304.8us
I20260709 13:06:25.502440 19124 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:06:25.498649 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 252689us
I20260709 13:06:25.561542 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:25.476007 19297 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:25.563836 13946 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:25.506304 19273 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.520980 18874 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.570083 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:25.573926 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:25.578918 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:25.580096 14524 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:25.509251 19261 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:25.581395 18874 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:25.585762 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 131108us
I20260709 13:06:25.587075 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:25.576411 19335 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:25.579962 19273 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.614459 19265 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:25.577872 19325 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:25.633174 19041 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:25.589924 19261 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:25.608853 19335 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:25.732170 19335 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:25.732774 19299 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:25.599314 19237 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:25.589445 19124 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:25.737802 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 151323us
W20260709 13:06:25.695310 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 85067.3us
I20260709 13:06:25.603031 19149 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.598387 19122 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:25.696020 19115 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:25.697181 19126 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:25.698972 19340 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.122s	user 0.003s	sys 0.007s
I20260709 13:06:25.699358 19128 raft_consensus.cc:2804] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:25.700012 18874 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:25.598788 19339 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:25.614845 19347 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:06:25.702243 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 2982) took 1428 ms. Trace:
I20260709 13:06:25.586532 19328 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:25.706339 13866 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:25.706951 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:25.707151 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 608) took 1313 ms. Trace:
I20260709 13:06:25.708817 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:25.709659 14015 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
W20260709 13:06:25.710649 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 187524us
I20260709 13:06:25.705684 19259 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:25.722729 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:25.725618 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.726368 19156 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:25.729817 19282 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.009s
I20260709 13:06:25.730463 19329 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:25.759748 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:25.731597 19261 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:25.602751 19334 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:25.739593 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:25.718979 19269 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:25.834800 13948 raft_consensus.cc:2368] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14.
I20260709 13:06:25.745141 13871 rpcz_store.cc:276] 0709 13:06:24.273917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.454332 (+180415us) service_pool.cc:224] Handling call
0709 13:06:25.701980 (+1247648us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b54000e9f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:25.702176 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1700985600}
I20260709 13:06:25.747165 14023 rpcz_store.cc:276] 0709 13:06:24.396951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.398937 (+  1986us) service_pool.cc:224] Handling call
0709 13:06:25.706939 (+1308002us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b5400325ad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:25.707091 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1586206080}
W20260709 13:06:25.750736 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 43884.2us
I20260709 13:06:25.836206 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:06:25.832692 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44747.5us
W20260709 13:06:25.766921 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.749493 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2496) took 1585 ms. Trace:
I20260709 13:06:25.747437 14024 raft_consensus.cc:2376] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 14. Re-sending same reply.
I20260709 13:06:25.774703 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:25.775640 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:25.841459 13941 rpcz_store.cc:276] 0709 13:06:24.164288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.331857 (+167569us) service_pool.cc:224] Handling call
0709 13:06:24.332237 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.545576 (+213339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.749193 (+1203617us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54000b9d54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:25.749425 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474963200}
I20260709 13:06:25.775942 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:25.776207 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:25.777045 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:25.772382 14015 pending_rounds.cc:85] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 13
I20260709 13:06:25.764902 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:25.744242 19115 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.740360 19122 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:25.761845 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:25.746286 19328 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.860301 19334 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.861114 19115 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.863807 19123 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:25.864301 19123 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:25.864711 19123 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:25.848990 19041 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:25.872704 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.784376 13869 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:25.789848 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:25.832427 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 92925.7us
I20260709 13:06:25.755887 19286 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:25.736399 19237 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:25.830832 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.744964 18874 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:25.847455 13868 raft_consensus.cc:2410] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:06:25.847529 13838 leader_election.cc:400] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14.
I20260709 13:06:25.751161 19299 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:25.888286 13838 leader_election.cc:403] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:06:25.847827 14015 pending_rounds.cc:107] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:06:25.889160 14015 raft_consensus.cc:2889] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:25.848084 13946 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:25.747273 19126 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.892629 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:25.854771 19122 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:25.854839 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:25.895103 13948 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:25.854951 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.896884 13941 raft_consensus.cc:2410] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:06:25.755339 19273 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.729874 19325 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:25.848628 19318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.010s	sys 0.022s
I20260709 13:06:25.904299 13915 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:25.844599 19350 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.908511 13941 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:25.870368 19328 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:25.929142 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 152995us
I20260709 13:06:25.932369 13987 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:25.920462 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:06:25.910909 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 72009.8us
I20260709 13:06:25.846230 19348 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:25.937733 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:25.943060 13869 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:25.947160 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:25.832147 18785 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:06:25.952574 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:25.955492 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:25.891685 19237 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:25.891988 18874 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:25.965747 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:25.966562 13946 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:25.968237 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 606) took 1576 ms. Trace:
W20260709 13:06:25.909190 19318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.011s	sys 0.022s
I20260709 13:06:25.970347 14016 rpcz_store.cc:276] 0709 13:06:24.392087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.392189 (+   102us) service_pool.cc:224] Handling call
0709 13:06:25.968047 (+1575858us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b5400325fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:25.968170 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":316864896}
I20260709 13:06:25.970482 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:25.972479 13946 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:25.920794 19234 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14.
I20260709 13:06:25.918020 19323 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:25.988445 13948 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:25.762043 19124 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:25.907881 19126 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:25.933627 19115 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:25.987707 13838 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:25.993975 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:26.082808 13868 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
W20260709 13:06:25.994618 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.995276 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:25.962029 19282 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:25.994473 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:25.987821 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:25.994891 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:25.961558 19299 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:26.004745 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:25.949411 19348 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.963081 19273 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:26.095875 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 108294us
I20260709 13:06:25.908792 19338 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 13:06:25.976588 18996 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:25.969141 19041 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:26.096422 13872 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:25.988862 19241 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:26.107677 13838 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:26.091038 19345 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.094574 19115 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:25.930012 19349 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.189s
I20260709 13:06:25.961592 19149 consensus_queue.cc:1059] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.143s
I20260709 13:06:26.091486 19280 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.156519 13948 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:06:26.158190 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 50612.5us
I20260709 13:06:26.168656 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:26.104301 18785 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:26.095590 19282 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:25.995702 19262 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.160s
I20260709 13:06:25.995519 19297 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:26.096959 19222 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.084149 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:26.157085 19326 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.988392 19269 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.164s
W20260709 13:06:26.165959 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 427050us
W20260709 13:06:26.187606 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.188112 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:25.949997 19350 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.192436 19041 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:26.193853 19350 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:25.924015 19346 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.195s
I20260709 13:06:26.093261 19352 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:26.181018 19282 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:26.170178 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 433998us
I20260709 13:06:26.105157 19348 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:26.165457 19128 raft_consensus.cc:697] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:26.199019 19128 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.199755 19143 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:26.120076 19299 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.202706 19299 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.203850 19222 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.204728 19282 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.205482 19299 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:26.207545 19326 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.116053 19273 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.175565 18744 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:26.210911 19273 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:26.172983 19329 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:26.176433 19280 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.214848 19329 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:26.215652 19329 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.176705 19349 consensus_queue.cc:1243] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:26.221603 18785 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:26.227244 19269 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:26.227314 18744 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:26.227782 19269 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:26.186486 13872 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:26.157593 19241 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:26.165576 18852 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:26.160499 19115 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:26.230913 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 142378us
I20260709 13:06:26.227886 18785 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.242830 19241 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.244273 19241 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:26.235167 18852 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.228652 18744 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.219141 19265 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 13:06:26.243029 19335 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
I20260709 13:06:26.241601 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:26.217995 19126 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:26.169425 19345 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.246022 18852 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:26.354837 19329 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:26.359215 19222 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:26.416605 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 367396us
I20260709 13:06:26.220448 19348 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.437579 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:26.438501 13875 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:26.440378 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:26.434389 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:26.442262 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:26.443159 13868 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:06:26.446461 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.448024 19349 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:26.457340 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 278523us
I20260709 13:06:26.462765 18996 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:26.464022 18996 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.441133 13875 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:26.353960 19335 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.486336 13947 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:26.486443 19335 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:26.487591 19335 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.488227 19335 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.448686 19349 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.492388 19349 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:26.477195 19360 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:26.495178 19360 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:26.496464 19360 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.499698 19326 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:26.500586 19326 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:26.503405 19358 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.504235 19358 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.263221 19269 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.511088 19269 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:26.511497 19084 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.512910 19084 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:26.517740 19280 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:26.435945 19266 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:26.540468 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 552675us
W20260709 13:06:26.543556 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 302505us
I20260709 13:06:26.545145 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:26.546075 14555 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:26.546454 14560 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:06:26.550808 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:26.555123 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:26.556460 14551 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:26.561959 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:26.563417 14560 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:26.564160 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:26.564446 14554 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:26.565335 14552 raft_consensus.cc:2410] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:06:26.565899 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:26.566537 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:26.567297 14551 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:26.573906 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.566016 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:26.575511 14553 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:26.582394 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:26.582633 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:26.532840 19156 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.701s	user 0.011s	sys 0.001s
I20260709 13:06:26.622895 13945 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:26.624460 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2843) took 2153 ms. Trace:
I20260709 13:06:26.599565 14555 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:26.619472 14551 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
W20260709 13:06:26.625205 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 83045.5us
W20260709 13:06:26.532109 19123 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.667s	user 0.013s	sys 0.000s
I20260709 13:06:26.627074 13987 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:26.625027 14015 rpcz_store.cc:276] 0709 13:06:24.470640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.622398 (+151758us) service_pool.cc:224] Handling call
0709 13:06:25.709312 (+1086914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.889858 (+180546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.624161 (+734303us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400325ad0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:26.624380 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2498887936}
I20260709 13:06:26.642532 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:26.658843 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:26.661052 13987 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:26.624554 19222 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.672425 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:26.536062 19266 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.668949 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:26.696517 14551 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:06:26.741852 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:26.677425 19237 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.690s	user 0.009s	sys 0.005s
I20260709 13:06:26.760316 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:26.665158 19334 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:26.664242 19149 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:26.638985 19123 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.642266 19153 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 13:06:26.780194 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:26.792905 14553 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:26.680380 19222 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.781904 19335 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:26.833837 19381 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:06:26.834774 19261 raft_consensus.cc:3055] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:26.844303 19261 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:26.842211 19345 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:26.843736 19266 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:26.848237 18874 consensus_meta.cc:287] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.831s	user 0.008s	sys 0.004s
I20260709 13:06:26.774044 19237 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.836599 19123 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:26.804450 19358 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:26.883471 19348 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:26.888824 19241 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.001s
I20260709 13:06:26.904481 19237 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:26.845382 19261 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:26.916801 19261 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:26.924389 19286 consensus_meta.cc:287] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.032s	user 0.012s	sys 0.002s
I20260709 13:06:26.952174 19286 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:26.929134 19328 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.961s	user 0.013s	sys 0.000s
I20260709 13:06:26.953742 19328 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:26.957558 19328 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:26.943977 19143 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.744s	user 0.013s	sys 0.001s
I20260709 13:06:27.143566 13941 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
W20260709 13:06:27.146534 13914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.006s	sys 0.005s
W20260709 13:06:27.147480 13914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.006s	sys 0.005s
W20260709 13:06:27.150411 13915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.010s	sys 0.003s
W20260709 13:06:27.150276 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 489096us
W20260709 13:06:27.151019 13915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.010s	sys 0.003s
I20260709 13:06:27.152611 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:27.154420 13916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.020s	sys 0.011s
W20260709 13:06:27.155067 13916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.020s	sys 0.011s
I20260709 13:06:27.156615 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 764) took 1912 ms. Trace:
I20260709 13:06:27.157183 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:27.166493 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:27.169277 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.176992 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:27.177840 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:27.210709 19282 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.211295 14560 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:06:27.179031 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:27.180076 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:27.214684 14554 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:27.216957 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2532) took 1380 ms. Trace:
I20260709 13:06:27.181401 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:27.218891 14556 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
W20260709 13:06:27.182127 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 912509us
W20260709 13:06:27.157954 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.182919 14558 raft_consensus.cc:2368] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10.
I20260709 13:06:27.183905 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:27.185772 18744 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.188313 18785 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:26.899868 19115 consensus_meta.cc:287] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.646s	user 0.009s	sys 0.004s
I20260709 13:06:27.189597 19124 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:27.191648 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2522) took 1434 ms. Trace:
W20260709 13:06:26.950759 18852 consensus_meta.cc:287] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.006s
I20260709 13:06:27.240986 19124 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.241477 18852 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.192265 19153 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:27.196246 19148 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:26.929895 18874 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.246670 13950 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:27.247170 19041 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:27.247936 19041 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:27.199251 19222 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.252871 19148 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:27.254413 18874 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.200446 19022 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.201771 19084 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:27.262795 19143 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:27.262993 13988 connection.cc:256] RPC call timeout handler was delayed by -1.4813886609999827s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
W20260709 13:06:26.938951 19122 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.015s	user 0.013s	sys 0.002s
W20260709 13:06:27.263811 13988 connection.cc:256] RPC call timeout handler was delayed by -1.3598886610000136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
I20260709 13:06:27.264120 19143 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:27.265188 13988 connection.cc:256] RPC call timeout handler was delayed by -1.2005886609999834s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1978s for a response
I20260709 13:06:27.205629 19299 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.207047 19332 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:27.204171 19149 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:27.266130 19299 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.266465 19149 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.267352 14561 raft_consensus.cc:2368] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16.
I20260709 13:06:27.267421 19332 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.268420 19332 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.239925 13947 rpcz_store.cc:276] 0709 13:06:25.756537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.760639 (+  4102us) service_pool.cc:224] Handling call
0709 13:06:25.760962 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.487031 (+726069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.190852 (+703821us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b54001f4fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:27.191289 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1667506048}
I20260709 13:06:27.157492 13866 rpcz_store.cc:276] 0709 13:06:25.243909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.244124 (+   215us) service_pool.cc:224] Handling call
0709 13:06:25.706144 (+462020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.746718 (+ 40574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.155761 (+1409043us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54000e9f50. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:27.156198 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1061593856}
I20260709 13:06:27.222186 13914 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:27.241472 19115 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.272925 19143 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:27.273125 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 59079.1us
W20260709 13:06:27.274758 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.275096 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:27.234150 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.236032 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:27.264725 19122 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.217545 13941 rpcz_store.cc:276] 0709 13:06:25.836664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.916391 (+ 79727us) service_pool.cc:224] Handling call
0709 13:06:25.917086 (+   695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.145384 (+1228298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.216499 (+ 71115us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400229a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:27.216840 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2817966336}
W20260709 13:06:27.265625 13988 connection.cc:256] RPC call timeout handler was delayed by -1.094249418000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1771s for a response
W20260709 13:06:27.279240 13988 connection.cc:256] RPC call timeout handler was delayed by -1.0610886609999852s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
W20260709 13:06:27.279985 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.269691 19149 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:27.253460 19158 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:27.263001 19022 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.291394 19314 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:27.292054 19314 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.292572 19314 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:27.293967 19299 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:27.295128 19332 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:27.300302 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:27.300529 19379 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.300889 14561 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:27.301400 19379 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.302279 18996 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:27.303367 13948 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:06:27.303122 18996 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.305841 19124 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:27.306818 19379 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:27.284775 18874 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:27.307512 19379 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:27.307973 18874 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.286547 19084 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:27.310252 19122 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.311622 13946 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:27.315893 18852 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.317814 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:27.321563 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.323958 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.325251 13875 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:27.325047 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:27.326239 19222 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:27.326403 14561 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:27.326812 19222 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.327430 19222 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:27.328164 19259 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:27.328393 13914 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:27.329879 19352 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:27.330346 19352 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.330739 19352 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:27.300690 13941 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:27.308015 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 72412.3us
I20260709 13:06:27.310945 19084 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.305768 18996 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:27.341504 13949 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:27.334084 19163 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.342188 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.343359 19163 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.344168 13871 raft_consensus.cc:2393] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 19: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:27.360108 19379 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:27.348543 19149 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:27.372987 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:27.373922 13949 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:27.375572 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.362419 18744 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.378266 19273 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:27.379719 13875 raft_consensus.cc:2410] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:06:27.382300 13945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2942) took 2180 ms (client timeout 2048 ms). Trace:
I20260709 13:06:27.382547 18996 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:27.381462 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:27.382803 13945 rpcz_store.cc:269] 0709 13:06:25.199425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.241179 (+ 41754us) service_pool.cc:224] Handling call
0709 13:06:27.379398 (+2138219us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400085f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:27.379567 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3145204608}
I20260709 13:06:27.288587 19041 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.351110 18785 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.370373 19125 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.396948 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.396759 19125 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.397437 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.398326 18996 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.399787 19125 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.379256 19273 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.402114 19346 raft_consensus.cc:3055] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:27.402689 19346 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:27.406329 18744 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.394869 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:27.389720 19163 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.396970 19041 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:27.394528 18874 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:27.423481 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 57448.1us
I20260709 13:06:27.425103 19122 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:27.425940 19122 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:27.426527 19122 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:27.406423 19115 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:27.433928 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:27.433903 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.440681 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2519) took 1685 ms. Trace:
I20260709 13:06:27.419955 19125 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.423122 19273 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.445245 19084 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:27.446422 13867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 2948) took 3654 ms (client timeout 2517 ms). Trace:
I20260709 13:06:27.441255 13950 rpcz_store.cc:276] 0709 13:06:25.755430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.772826 (+ 17396us) service_pool.cc:224] Handling call
0709 13:06:25.773233 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.247272 (+1474039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.437469 (+190197us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540022a954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:27.440581 (+  3112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3387112832}
W20260709 13:06:27.449966 13867 rpcz_store.cc:269] 0709 13:06:23.786397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.486257 (+699860us) service_pool.cc:224] Handling call
0709 13:06:27.440579 (+2954322us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b54002e77d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:27.440767 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1742900096}
I20260709 13:06:27.452536 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.453470 13871 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:27.453649 19022 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.465108 19321 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:27.466435 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.466866 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2534) took 1527 ms. Trace:
I20260709 13:06:27.467300 13946 rpcz_store.cc:276] 0709 13:06:25.939293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.973421 (+ 34128us) service_pool.cc:224] Handling call
0709 13:06:25.973828 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.312210 (+1338382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.466555 (+154345us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54003309d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:27.466796 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3075397504}
I20260709 13:06:27.469105 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.454241 13949 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:27.471949 19125 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.479751 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:27.480067 13946 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:06:27.480795 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:27.481546 13948 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:27.482595 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:27.483299 13948 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:27.484180 19022 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.484632 13915 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:27.484958 19022 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.485723 19115 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:27.486804 18996 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.488088 13950 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:27.489784 13945 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:06:27.490135 19347 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.575s	user 0.015s	sys 0.026s
W20260709 13:06:27.490504 19347 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.635s	user 0.016s	sys 0.026s
I20260709 13:06:27.492123 19273 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:27.492858 19273 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:27.495572 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 338814us
I20260709 13:06:27.406989 19346 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.520879 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.521405 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:27.528070 19125 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.536762 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.534905 13941 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:27.246930 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:27.530898 18996 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.548213 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:27.550581 19273 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.559571 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:27.560333 19163 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:27.561527 19270 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:27.562024 19163 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:27.562178 19314 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.564793 19314 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:27.562669 18940 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:27.565454 18940 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:27.542972 19352 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.570577 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:27.571313 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46625.7us
I20260709 13:06:27.573944 19352 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:27.577476 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:27.578402 14558 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:27.579874 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:27.582437 18874 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:27.583098 18874 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:27.583488 19115 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:27.583999 19115 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.584375 19115 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:27.584216 18874 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.586328 18996 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.588884 18996 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:27.590400 19395 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:06:27.592368 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:27.570574 18940 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.595939 19022 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.592751 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:27.578552 13950 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:27.581853 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:27.599653 13871 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:27.601615 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:27.602267 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:27.602823 19263 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.428s	user 0.069s	sys 0.129s
W20260709 13:06:27.603252 19263 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.504s	user 0.069s	sys 0.129s
I20260709 13:06:27.603307 13945 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:27.606513 19329 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:27.607371 19328 consensus_queue.cc:1059] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:27.607622 19394 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:27.608948 19328 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:27.609220 19394 raft_consensus.cc:3055] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:27.609344 19328 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:27.609689 19394 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:27.610924 19394 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.602448 14552 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:27.612490 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.617403 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:27.618631 19326 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:27.619132 19326 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.619633 19326 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:27.623778 19354 raft_consensus.cc:3055] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:27.624456 19354 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:27.628199 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:27.629431 18785 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.630389 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:27.631093 13941 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:27.631876 13945 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:27.633572 13942 raft_consensus.cc:2410] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:06:27.634183 19110 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.634789 19110 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.633872 14558 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:27.589452 19222 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.637415 13945 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:06:27.637665 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.639034 19149 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:27.639813 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.644661 13871 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:27.647017 19339 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:27.647846 19314 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.648458 19314 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.649569 19338 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:27.650261 13942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2924) took 2500 ms (client timeout 1858 ms). Trace:
I20260709 13:06:27.650326 19338 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:27.650802 13942 rpcz_store.cc:269] 0709 13:06:25.150027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.155933 (+  5906us) service_pool.cc:224] Handling call
0709 13:06:27.648851 (+2492918us) spinlock_profiling.cc:243] Waited 2.44 s on lock 0x7b5400229054. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:27.650165 (+  1314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5609453312}
I20260709 13:06:27.651592 19314 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.652845 19338 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:27.649690 13944 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:06:27.652928 19354 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.653859 13944 pending_rounds.cc:85] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 8
I20260709 13:06:27.654371 13944 pending_rounds.cc:107] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:06:27.654699 13944 raft_consensus.cc:2889] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:27.655857 13950 raft_consensus.cc:2368] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 19. Current term is 20.
I20260709 13:06:27.657752 19222 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:27.655354 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:27.660082 13875 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 10: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:27.661053 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:27.661464 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:27.661841 13948 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:27.662163 14554 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:27.662858 13838 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:27.664518 19394 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:27.661075 13945 raft_consensus.cc:2410] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:06:27.664723 19273 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:27.666250 19273 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:27.666640 19273 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.666975 19273 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:27.668448 13840 leader_election.cc:336] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.608s (SENT)
I20260709 13:06:27.668879 19346 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:27.669116 13840 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:27.670083 19022 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:27.670491 19022 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:27.671574 19022 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.672528 19339 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:27.673089 19022 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:27.674256 18744 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:27.675215 19328 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.671933 13914 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:27.677024 19328 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:27.677205 13915 connection.cc:256] RPC call timeout handler was delayed by -1.599972643000001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
I20260709 13:06:27.663581 19270 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:27.673185 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2944) took 2227 ms. Trace:
I20260709 13:06:27.679402 19270 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:27.679929 19270 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:27.615200 18940 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:27.681241 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:27.681027 19270 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.679436 13944 rpcz_store.cc:276] 0709 13:06:25.445440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.446857 (+  1417us) service_pool.cc:224] Handling call
0709 13:06:27.649094 (+2202237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.655279 (+  6185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.672889 (+ 17610us) spinlock_profiling.cc:243] Waited 2.2 s on lock 0x7b5400229050. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:27.673084 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5063526784}
W20260709 13:06:27.686462 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.690508 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:27.695549 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:27.696874 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.696547 14522 leader_election.cc:400] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 19. Current term is 20.
I20260709 13:06:27.698632 14522 leader_election.cc:403] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:06:27.699221 14554 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
W20260709 13:06:27.699818 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.703256 13946 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:27.703788 13942 raft_consensus.cc:2410] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 13:06:27.706539 13915 connection.cc:256] RPC call timeout handler was delayed by -1.435272642999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1895s for a response
W20260709 13:06:27.706984 13915 connection.cc:256] RPC call timeout handler was delayed by -1.2534899850001011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
I20260709 13:06:27.707515 13914 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:27.707873 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.708451 13875 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:06:27.713233 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:27.717846 14524 connection.cc:256] RPC call timeout handler was delayed by -1.4780749379999634s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 13:06:27.718575 14524 connection.cc:256] RPC call timeout handler was delayed by -1.2296749380000165s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
I20260709 13:06:27.702848 19379 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.720467 19354 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:27.723526 19341 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:27.750830 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.751765 13866 raft_consensus.cc:2410] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:27.753159 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.754208 19350 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 19. Current term is 20.
I20260709 13:06:27.754575 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.755183 13868 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:27.724669 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.725766 19110 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.697105 19122 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.726089 18744 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:27.757467 13947 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:27.728219 19261 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:27.758855 19261 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:27.759097 19122 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.759222 19261 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:27.760519 19261 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.729538 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:27.757771 18744 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:27.764163 18744 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:27.766311 13947 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:27.766616 19341 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:27.767467 19341 raft_consensus.cc:697] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:27.768985 19341 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:27.734709 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 75372.3us
I20260709 13:06:27.736116 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.737591 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.772266 13871 raft_consensus.cc:2376] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 13. Re-sending same reply.
I20260709 13:06:27.737847 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.738389 19323 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:27.738880 19263 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:27.739382 19123 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:27.748142 19379 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.771157 14552 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:27.729869 13838 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:27.774777 13869 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:27.789080 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:27.793798 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 64030.9us
I20260709 13:06:27.792491 19123 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:27.795538 19124 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:27.795405 19123 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.796044 19124 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.796453 19124 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:27.796680 19261 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:27.797940 18852 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:27.798812 19163 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:06:27.799585 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.799841 13868 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:27.801189 13867 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:27.782752 19273 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.774777 19323 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:27.803056 19323 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.765602 18744 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.807288 18744 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:27.790246 18996 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:27.799466 19041 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:27.792608 19270 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:27.820822 19041 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.822793 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.823839 19041 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:27.818341 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 466265us
I20260709 13:06:27.791885 19149 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:27.834007 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.848896 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.849680 13871 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:27.831373 19321 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:27.854179 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:27.856746 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.832798 18996 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:27.860169 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.860358 18996 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:27.844470 19110 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.863724 19110 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.866483 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:27.868186 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:27.871089 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.872381 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.872936 13873 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:27.875186 19346 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:27.876071 19346 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:27.876596 19346 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:27.853758 18940 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.878538 18940 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.878679 19115 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.878993 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.879779 13872 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:27.880708 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.880784 18940 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.881315 13872 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:06:27.881546 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.882331 13874 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:27.882755 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.883241 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:27.883870 13874 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:27.884717 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:27.857451 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.854444 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:27.832751 19041 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:06:27.885911 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:27.869750 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:27.897858 13874 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:27.856492 19158 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.005s
W20260709 13:06:27.905495 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51149.1us
W20260709 13:06:27.907332 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.877561 19346 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:27.896093 19110 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.910187 19149 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:06:27.930202 13915 leader_election.cc:336] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.543s (SENT)
I20260709 13:06:27.940651 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:27.914526 19263 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.901908 19321 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:27.955647 13840 leader_election.cc:336] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.192s (SENT)
W20260709 13:06:27.964524 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 168304us
I20260709 13:06:27.968024 13840 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:27.940299 19149 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:27.939379 19273 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:27.979041 19239 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:27.960175 19158 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:27.978511 19263 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:27.988862 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 119280us
I20260709 13:06:27.966351 19041 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:27.979629 19239 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.979628 18852 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:27.999454 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 84482.5us
W20260709 13:06:28.010614 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.019049 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:28.024976 13870 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:28.032078 19263 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:28.034883 19326 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.036406 19379 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:28.035815 19149 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.037659 19149 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:28.023722 13987 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:27.998756 19239 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:28.049551 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.049506 18785 raft_consensus.cc:3055] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:28.120174 13872 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:28.126446 13873 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:06:28.037887 19326 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:28.152995 13874 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:28.154026 13943 raft_consensus.cc:2410] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:28.044952 19321 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:28.122560 13838 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:28.157236 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 144461us
W20260709 13:06:28.171960 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.165831 19125 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:28.187862 19125 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.188298 19125 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:28.190518 19408 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 13:06:28.192987 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.115142 18785 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:28.205714 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.210415 13987 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:28.211580 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:28.195920 13838 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:28.213083 13943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 2928) took 3061 ms (client timeout 1843 ms). Trace:
I20260709 13:06:28.216289 19156 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.218241 13872 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:28.218835 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:28.219071 19115 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:28.222211 19338 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:28.230712 13943 rpcz_store.cc:269] 0709 13:06:25.151318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.226240 (+ 74922us) service_pool.cc:224] Handling call
0709 13:06:28.212743 (+2986503us) spinlock_profiling.cc:243] Waited 2.66 s on lock 0x7b5400088254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:28.212977 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6117294848}
I20260709 13:06:28.237247 13867 raft_consensus.cc:2368] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
W20260709 13:06:28.204198 19022 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.002s
W20260709 13:06:28.242604 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 336721us
W20260709 13:06:28.243755 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48048.4us
I20260709 13:06:28.152544 18852 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:06:28.166121 19084 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:28.256875 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 238459us
I20260709 13:06:28.255199 13838 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.249023 19163 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:06:28.253106 19328 consensus_meta.cc:287] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.570s	user 0.017s	sys 0.000s
I20260709 13:06:28.274289 14552 raft_consensus.cc:1813] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Handling vote request from an unknown peer 3779649036364744823fc035dac892c7
I20260709 13:06:28.275717 14552 raft_consensus.cc:2410] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:06:28.272697 19163 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:28.279224 13838 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.276680 13988 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:28.297644 13867 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:06:28.298825 13867 pending_rounds.cc:85] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 7
I20260709 13:06:28.299134 13867 pending_rounds.cc:107] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:06:28.299444 13867 raft_consensus.cc:2889] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:28.236088 19124 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.280833 19163 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:28.280769 19338 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:28.239406 19156 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:28.307425 13915 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.712s (SENT)
I20260709 13:06:28.314803 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:28.281723 19261 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:06:28.334720 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.311025 19338 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:28.350714 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:28.355047 13947 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:28.358655 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:28.320518 19261 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.352974 13942 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:28.361694 13941 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:06:28.365176 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 426001us
W20260709 13:06:28.367062 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 112595us
W20260709 13:06:28.376230 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.332882 19346 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:28.351359 19270 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:28.387349 19270 raft_consensus.cc:697] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:28.388422 19270 consensus_queue.cc:237] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.388525 18785 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.389686 18785 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:28.316293 19084 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:28.382804 19411 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:06:28.383819 19410 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:06:28.384801 19415 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:06:28.385921 19413 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:06:28.355818 18996 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:28.360507 19338 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:28.401252 19416 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
W20260709 13:06:28.421571 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 43865.1us
I20260709 13:06:28.425325 19084 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:28.368659 19261 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:28.365599 19273 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:28.400488 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:28.449107 13915 leader_election.cc:336] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.895s (SENT)
I20260709 13:06:28.439476 13866 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:28.441083 13868 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:28.401757 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:28.449749 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:28.454082 13875 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:28.461184 13838 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.464944 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:28.465718 13869 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
W20260709 13:06:28.470232 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.460268 19084 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.489069 19084 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.491432 19084 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:28.454733 18744 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.006s
W20260709 13:06:28.503255 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 47440.1us
I20260709 13:06:28.511724 19124 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:28.513854 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 73528us
I20260709 13:06:28.493264 19273 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.519474 19149 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.520364 19149 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:28.525085 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:28.525959 13869 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:28.527206 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:28.528539 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:28.529336 13874 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:28.530413 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:28.532286 19417 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.533072 19417 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.470162 19323 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:28.517757 19273 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:28.541108 19409 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:06:28.562971 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 352702us
W20260709 13:06:28.569969 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 150919us
I20260709 13:06:28.562265 19163 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.568748 19409 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.585786 19346 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.598773 19346 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.601502 19346 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:28.604791 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.580541 19417 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:28.609512 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.610903 13871 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:28.590765 19409 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:28.632609 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:28.634933 13871 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:28.635694 19163 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:28.636798 13871 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:06:28.640786 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 55332.5us
I20260709 13:06:28.641881 14558 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:06:28.597486 19125 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:28.646301 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 53902us
W20260709 13:06:28.649150 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 144897us
I20260709 13:06:28.652362 13871 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:28.652915 19323 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:28.653959 13871 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:06:28.642763 19346 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.655406 19346 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.657790 19346 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:28.663846 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:28.663744 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.667089 14556 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:28.673317 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:28.676554 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.672017 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:28.674849 13987 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:28.691227 14558 raft_consensus.cc:2376] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 14. Re-sending same reply.
I20260709 13:06:28.693331 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.694367 14559 raft_consensus.cc:2368] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17.
I20260709 13:06:28.694792 14555 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:28.695182 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.695703 19125 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:28.696023 14553 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:28.698465 14560 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:28.674530 19379 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:06:28.723316 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.658718 19341 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:06:28.719352 19379 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.715606 19346 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:06:28.733181 19239 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.610s	user 0.014s	sys 0.000s
I20260709 13:06:28.740623 19149 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:28.746320 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.748529 19239 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.754933 19239 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:28.755553 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3008) took 1039 ms. Trace:
I20260709 13:06:28.756043 13875 rpcz_store.cc:276] 0709 13:06:27.720964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.721172 (+   208us) service_pool.cc:224] Handling call
0709 13:06:28.755341 (+1034169us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54003acfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:28.755481 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263881088}
W20260709 13:06:28.759857 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 95081.6us
I20260709 13:06:28.735097 19379 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:28.760788 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:28.761897 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 97020.3us
I20260709 13:06:28.770942 19338 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.773603 19338 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:28.774142 19323 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:28.774689 19323 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.775071 19323 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:28.778687 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:28.783471 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.787035 19346 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:28.787565 19346 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.787981 19346 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:28.788022 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.790087 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:28.790449 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:28.790930 14551 raft_consensus.cc:2368] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 9. Current term is 10.
I20260709 13:06:28.791085 13941 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:28.792207 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:28.792887 13947 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:28.790938 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.795497 13875 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:28.795928 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:28.796685 13948 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:28.799468 13838 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.800460 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:28.800665 13838 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.801290 13941 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:28.804339 18785 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:28.804854 18785 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.805233 18785 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:28.806303 19124 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:28.806191 19261 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.806924 19124 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.807446 19124 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:28.808944 19261 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:28.764230 19156 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:28.810202 19328 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.810536 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:28.811185 13948 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:28.811465 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 46248.4us
I20260709 13:06:28.810801 19328 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:28.816553 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.817602 19384 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:28.819556 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:28.821265 19328 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:28.823174 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:28.824077 13875 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
W20260709 13:06:28.830858 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.830901 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:28.831832 13942 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:28.831854 13948 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:28.813618 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.833710 13838 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.842882 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:28.846961 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.847640 14556 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:06:28.850528 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:28.854594 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.855695 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.856992 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:28.857252 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.859064 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.861763 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:28.864840 13947 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:28.858284 14553 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:28.869235 14558 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:28.852738 13950 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:06:28.875339 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:28.842025 18940 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:28.877693 13915 leader_election.cc:400] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:06:28.884369 13915 leader_election.cc:403] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:06:28.884421 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.883253 14560 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:28.886284 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:28.887018 13875 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:28.884831 18996 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:06:28.897161 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:28.901129 14558 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 10: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:28.900167 13941 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:28.902853 13838 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.904781 14551 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:28.869153 19384 raft_consensus.cc:2764] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:06:28.869386 19338 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.876055 18874 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:06:28.901111 13839 leader_election.cc:336] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.544s (SENT)
I20260709 13:06:28.918565 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:28.919550 13875 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:28.916146 14522 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:28.922374 13838 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.928457 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.929464 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:28.896051 19115 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:06:28.884517 19156 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:28.900139 18940 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:28.946875 18940 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.938035 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.948761 19338 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:28.949630 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.950903 19265 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:28.951324 19338 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:28.951957 19338 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:28.951352 19265 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.952958 19265 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:28.936530 19158 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:06:28.935765 13769 catalog_manager.cc:5697] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 9, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 9 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.959756 19417 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:06:28.960067 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 74007.6us
W20260709 13:06:28.961061 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:28.961537 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.960826 13769 catalog_manager.cc:5697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:28.940493 19341 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:06:28.944255 19433 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:06:28.963869 14558 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:28.963940 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.965080 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.965477 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:28.965688 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:28.945706 13771 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57642 (request call id 274) took 32947 ms (client timeout 15000 ms). Trace:
I20260709 13:06:28.935318 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:28.936030 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.966370 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:28.970208 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 84551.8us
I20260709 13:06:28.966373 13948 raft_consensus.cc:2393] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:28.925168 18874 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.973603 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.928756 18852 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:06:28.937057 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.972688 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.972949 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.901736 18996 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:06:28.952320 19338 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.973937 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:28.933375 19430 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:28.978826 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:28.976284 14555 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:28.976317 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:28.985389 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:28.986578 14554 raft_consensus.cc:2368] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16.
I20260709 13:06:28.984160 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:28.970440 13875 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:28.932915 19431 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:28.937721 19156 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.979151 13949 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:06:28.996958 13988 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.834s (SENT) [suppressed 39 similar messages]
I20260709 13:06:29.015172 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:29.023769 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:29.007974 18785 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.028800 18785 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:28.983587 19115 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:29.030435 19115 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.030861 19115 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:29.031738 13872 raft_consensus.cc:2368] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11.
I20260709 13:06:28.976773 18874 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:29.025993 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:29.037735 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 59016.1us
I20260709 13:06:28.980261 19239 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.038753 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:28.981410 19338 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:29.034709 13769 catalog_manager.cc:5697] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 6 to 11, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 11 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:29.139889 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:29.162468 13949 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:29.165230 19441 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:06:29.144788 19239 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.044262 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:29.009395 19156 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:29.130401 13942 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:28.980535 19430 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.153237 13869 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:06:29.155964 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:29.179559 13941 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: []
W20260709 13:06:29.071094 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.008188 19409 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.154353 19222 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:29.030861 19124 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.153961 19424 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:28.983798 18852 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:29.170109 13947 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:29.137523 18744 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:29.171463 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 132817us
I20260709 13:06:29.177922 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:29.186415 13769 catalog_manager.cc:5697] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 7 to 10, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 10 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:29.153959 19431 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:29.196501 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.187907 19430 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:29.230080 19338 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.187722 19321 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:29.265830 19321 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:29.266607 19321 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:29.269241 19379 consensus_meta.cc:287] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.000s
I20260709 13:06:29.271880 19239 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:29.272819 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:29.273648 13950 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:29.274492 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:29.229732 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.228837 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.230710 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.230881 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.236012 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.296933 14024 raft_consensus.cc:2410] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:29.297035 14018 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:29.227626 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 45742.8us
I20260709 13:06:29.235240 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:29.226493 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 56471.4us
I20260709 13:06:29.231637 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.280493 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:29.239006 13875 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:29.214174 13945 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:29.173446 19341 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.301286 19341 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.301996 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.302625 14019 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:29.303303 19338 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:29.303651 19341 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:29.234349 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.235752 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:29.210099 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.309782 14016 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:29.309538 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 131725us
I20260709 13:06:29.327500 14015 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:29.328094 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:29.328871 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:29.328081 13949 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
W20260709 13:06:29.343597 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.234861 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.229023 19222 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:29.355294 14023 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:29.357694 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:29.269759 19379 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.355311 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:29.364980 13946 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:29.304059 19338 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:29.366550 13945 raft_consensus.cc:2368] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11.
I20260709 13:06:29.297798 19321 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:29.376889 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:29.367992 13944 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:29.369133 19379 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:29.370404 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.370476 19424 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.370919 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:29.365581 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 338565us
I20260709 13:06:29.384776 14015 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:29.304208 19341 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.228152 19124 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:29.238946 19334 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:29.378635 14017 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
W20260709 13:06:29.380964 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:29.382305 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 119204us
I20260709 13:06:29.382954 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:29.384109 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:29.389725 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:29.364329 19323 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.589s	user 0.013s	sys 0.001s
W20260709 13:06:29.395038 13839 leader_election.cc:336] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.668s (SENT)
I20260709 13:06:29.405222 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:29.406131 13838 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:29.356882 18996 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:06:29.406881 14022 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:29.372452 19338 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:29.409780 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:29.413194 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 172704us
W20260709 13:06:29.412777 13840 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.500s (SENT)
I20260709 13:06:29.414551 13989 leader_election.cc:400] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11.
I20260709 13:06:29.361321 19384 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:06:29.415179 13769 catalog_manager.cc:5697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 7 to 8, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 8 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:29.418226 13989 leader_election.cc:403] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:29.327214 18744 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:28.971922 13771 rpcz_store.cc:269] 0709 13:05:55.998453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.998742 (+   289us) service_pool.cc:224] Handling call
0709 13:06:28.945416 (+32946674us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:18.796049 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:18.796878 (+   829us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:18.796938 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:06:18.796958 (+    20us) tablet.cc:662] Decoding operations
0709 13:06:19.027066 (+230108us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:19.044497 (+ 17431us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:19.047102 (+  2605us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:06:19.066360 (+ 19258us) tablet.cc:701] Row locks acquired
0709 13:06:19.066422 (+    62us) write_op.cc:260] PREPARE: finished
0709 13:06:19.066674 (+   252us) write_op.cc:270] Start()
0709 13:06:19.066889 (+   215us) write_op.cc:276] Timestamp: P: 1783602379066583 usec, L: 0
0709 13:06:19.066932 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 13:06:19.074826 (+  7894us) log.cc:844] Serialized 7947 byte log entry
0709 13:06:20.052158 (+977332us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b1800360820. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:20.983855 (+931697us) op_driver.cc:464] REPLICATION: finished
0709 13:06:21.217467 (+233612us) write_op.cc:301] APPLY: starting
0709 13:06:21.219938 (+  2471us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:21.450567 (+230629us) spinlock_profiling.cc:243] Waited 9.43 ms on lock 0x7b540012acd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c343032a 00007fb2c33b5f14 00007fb2c33b5ec3 00007fb2c33b5e38 00007fb2c33b5df3 00007fb2c33b50dc 00007fb2c33dee16 00007fb2c33c6cbb 00007fb2c33b139d 00007fb2c33b4d75
0709 13:06:21.477396 (+ 26829us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:21.477455 (+    59us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:21.597233 (+119778us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:21.597836 (+   603us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 13:06:21.597894 (+    58us) write_op.cc:312] APPLY: finished
0709 13:06:21.598455 (+   561us) log.cc:844] Serialized 180 byte log entry
0709 13:06:21.598675 (+   220us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:21.598898 (+   223us) write_op.cc:454] Released schema lock
0709 13:06:21.598981 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:21.601940 (+  2959us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:21.601973 (+    33us) write_op.cc:454] Released schema lock
0709 13:06:22.061007 (+459034us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b1800360820. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
Metrics: {"mutex_wait_us":53187,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":226123,"apply.run_cpu_time_us":37968,"apply.run_wall_time_us":384452,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87607,"num_ops":20,"prepare.queue_time_us":360406,"prepare.run_cpu_time_us":48404,"prepare.run_wall_time_us":1182465,"raft.queue_time_us":458137,"raft.run_cpu_time_us":40281,"raft.run_wall_time_us":626056,"replication_time_us":1784358,"spinlock_wait_cycles":67877248,"thread_start_us":1452585,"threads_started":4,"wal-append.queue_time_us":903671,"wal-append.run_cpu_time_us":24424,"wal-append.run_wall_time_us":2603443}]]}
I20260709 13:06:29.463645 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.187336 19346 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:29.468539 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 63027.5us
I20260709 13:06:29.464469 14024 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
W20260709 13:06:29.471349 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 76360.8us
I20260709 13:06:29.391383 19321 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:06:29.479207 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.412907 19124 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:29.419064 19338 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:29.485549 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 103764us
W20260709 13:06:29.488094 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 68914.3us
I20260709 13:06:29.399296 19341 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.490595 14022 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:29.402280 19323 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.486909 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:29.494386 13838 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:29.491712 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:29.498463 13839 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.694s (SENT)
I20260709 13:06:29.499153 13839 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:29.498430 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.404624 19334 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.472400 19409 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:29.507294 14024 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:29.418638 19265 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.482969 19321 consensus_queue.cc:1243] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:06:29.511307 13950 raft_consensus.cc:2368] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13.
I20260709 13:06:29.517742 13769 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 10 to 11, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 11 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:29.520769 13769 catalog_manager.cc:5697] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 9, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 9 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:29.501201 19270 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:29.500459 19124 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:29.542135 14020 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
W20260709 13:06:29.547006 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 60193.1us
I20260709 13:06:29.549330 14021 raft_consensus.cc:2368] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 14. Current term is 15.
I20260709 13:06:29.549292 14019 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:29.495985 18785 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:06:29.463413 18852 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.497838 19115 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.554617 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:29.561091 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.488668 19222 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.563328 14017 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:06:29.548043 19273 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:29.546381 19350 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.566334 14019 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
W20260709 13:06:29.566759 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 213502us
I20260709 13:06:29.464323 19431 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.552557 19263 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:29.567648 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 69220.2us
W20260709 13:06:29.628309 13839 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.500s (SENT)
I20260709 13:06:29.568714 13941 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:29.549865 19417 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:06:29.610853 19451 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:06:29.565353 19270 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:29.613142 19453 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:06:29.554020 19457 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.625778 14023 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:29.625589 14016 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:29.626559 14018 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:29.628780 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:29.633436 14015 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:29.628762 13769 catalog_manager.cc:5697] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 6 to 8, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 8 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:29.647451 13769 catalog_manager.cc:5697] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 3 to 8, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 8 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:29.648994 13769 catalog_manager.cc:5697] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 4 to 6, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 6 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:29.651197 19239 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:29.653118 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:29.640554 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.635265 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:29.623088 19273 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:29.659878 14024 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:29.663193 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:29.632009 19149 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:29.668262 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.666580 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:29.667364 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 119471us
I20260709 13:06:29.669135 14016 raft_consensus.cc:2368] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17.
I20260709 13:06:29.667527 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.670336 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.671413 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.671677 13947 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:29.672978 14017 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:29.672102 14019 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:29.673574 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.673651 14023 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:29.674993 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:29.675376 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 47540us
I20260709 13:06:29.675537 14018 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:06:29.678804 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:29.681720 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 53442.7us
I20260709 13:06:29.618312 19455 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 13:06:29.692142 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:29.689471 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.693253 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:29.693629 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.693883 13875 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:29.686043 13769 catalog_manager.cc:5697] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:29.666776 19448 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:29.722945 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.704978 19341 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:29.702971 19265 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:29.696147 19424 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:29.683676 19464 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:29.727514 19464 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:29.728475 19464 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.636286 19263 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:29.730605 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.636845 19442 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:06:29.641664 19354 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:06:29.704969 19323 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:29.705936 19115 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:29.709024 18874 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.002s
I20260709 13:06:29.672071 19124 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:29.712486 19445 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:06:29.718075 19334 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:29.719826 19430 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:29.722329 19270 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:29.722744 19156 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.723770 14016 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:29.818400 13840 leader_election.cc:336] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.699s (SENT)
I20260709 13:06:29.704078 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:29.663085 19457 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:29.702816 13769 catalog_manager.cc:5697] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 9 to 10, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 10 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:29.731966 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.684878 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:29.752701 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.763626 13875 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:29.828845 14020 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:29.771835 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.731972 19263 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:29.784057 14017 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:29.752067 19394 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:06:29.792330 14018 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:29.643999 19350 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:29.797560 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:29.779402 19449 log.cc:927] Time spent T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:06:29.813469 14019 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:06:29.842599 14019 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
W20260709 13:06:29.819551 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 125070us
W20260709 13:06:29.823051 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 143939us
I20260709 13:06:29.844357 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.781033 19346 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:29.850559 19460 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
W20260709 13:06:29.851503 13840 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.722s (SENT)
I20260709 13:06:29.854391 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.855136 14020 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:29.855410 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.857367 14015 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: [9.8-12.9]
I20260709 13:06:29.697865 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:29.829053 14024 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:29.836422 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.838044 14018 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:29.840175 14551 raft_consensus.cc:2368] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10.
I20260709 13:06:29.824625 19448 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.844841 13988 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:29.872131 14024 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: [11.6-14.7]
I20260709 13:06:29.874212 14022 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:29.874665 19457 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:29.875037 14020 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:29.876497 14022 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:29.879761 14022 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:29.847276 14019 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:29.847780 13839 leader_election.cc:400] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17.
I20260709 13:06:29.881644 13839 leader_election.cc:403] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:29.812558 19338 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:29.812337 18874 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.887852 14018 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
W20260709 13:06:29.889429 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:29.895995 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:29.896524 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:29.902042 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 57310.5us
I20260709 13:06:29.905165 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:29.907871 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.908708 14022 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:29.909713 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.910320 14022 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:29.910795 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.911170 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.911446 14024 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:29.911891 14022 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:29.912320 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.912642 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.912930 14024 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:29.914391 14022 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:29.915215 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.915769 14022 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:29.916604 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.917236 14022 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:29.917932 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.918447 14022 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:29.919203 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.919879 14022 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:29.920603 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.921689 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.922667 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.923182 14022 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:29.923918 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.924432 14022 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:29.929702 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.930256 14022 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:29.935557 14016 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:29.936115 14024 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:29.937407 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.942144 14017 raft_consensus.cc:1813] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Handling vote request from an unknown peer 3779649036364744823fc035dac892c7
I20260709 13:06:29.942895 14017 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:29.944334 13988 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:29.945182 13942 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:29.898669 19431 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17.
I20260709 13:06:29.946337 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.949656 14024 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:29.946602 14018 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:29.934401 18744 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:29.948175 13840 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.898s (SENT)
I20260709 13:06:29.939150 19346 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:29.953696 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 48885.7us
I20260709 13:06:29.835713 19273 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:29.956774 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:29.946764 19263 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:29.953874 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.958597 14024 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:29.959065 14022 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:29.957738 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.969372 14018 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:29.972067 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.972093 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.973083 14022 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:29.973095 14016 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:29.974695 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.975684 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.976424 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.977183 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.978507 19156 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:29.979965 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.986454 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.987303 14022 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:29.988278 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.988796 14022 raft_consensus.cc:2376] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 14. Re-sending same reply.
I20260709 13:06:29.989653 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.990160 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:29.990979 13872 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:29.993005 19338 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:29.993243 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.993789 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.994606 14016 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:29.995023 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 910) took 1022 ms. Trace:
I20260709 13:06:29.995481 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.996639 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:29.997157 14016 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
W20260709 13:06:29.997597 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 321528us
I20260709 13:06:29.995503 14551 rpcz_store.cc:276] 0709 13:06:28.972667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.972813 (+   146us) service_pool.cc:224] Handling call
0709 13:06:29.994796 (+1021983us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b54006f18d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:29.994953 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1960267008}
I20260709 13:06:29.938423 19448 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:29.999284 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 164200us
I20260709 13:06:29.997926 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:30.000001 14016 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:30.000772 19448 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:29.998760 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:29.993391 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:29.998299 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 59428.8us
I20260709 13:06:30.007570 14022 raft_consensus.cc:2376] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 16. Re-sending same reply.
I20260709 13:06:30.008513 14018 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:30.008306 13988 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:30.011193 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:30.011430 14016 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:30.011816 13875 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:29.998490 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:30.012348 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:30.013077 14016 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:30.014134 14020 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:30.015959 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:30.016829 13872 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:30.030135 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:30.030138 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:30.001736 19466 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:06:30.031371 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 62032.2us
I20260709 13:06:29.956110 19334 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:30.039270 14021 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:30.053062 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:30.053524 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:30.053987 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:30.054490 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:30.055227 13870 raft_consensus.cc:2368] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15.
I20260709 13:06:30.055584 19466 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:30.056005 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:30.056648 14016 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:30.056720 19466 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:30.057581 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:30.058303 18996 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:30.058970 18996 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:30.060141 19163 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.059932 18996 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:30.060657 19163 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:30.062417 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:30.063870 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:30.064445 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:30.065008 13941 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:30.065131 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:30.065886 14022 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:30.063828 19163 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:30.067112 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:30.068229 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:30.068948 14022 raft_consensus.cc:2410] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:06:30.069643 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:30.058174 14016 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:30.069913 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:30.070592 14552 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:30.008514 14024 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:30.053855 14551 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:30.072536 14024 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:30.077576 14022 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
W20260709 13:06:30.078619 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.078779 14015 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:30.073975 14024 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:30.080981 14022 pending_rounds.cc:85] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 7
I20260709 13:06:30.082499 14022 pending_rounds.cc:107] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:06:30.082844 14022 raft_consensus.cc:2889] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:30.053189 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 54778.9us
I20260709 13:06:30.096426 13941 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:06:29.968736 19263 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:30.082772 13914 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:30.103108 14016 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:30.044955 13875 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:30.048756 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:30.107323 14017 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:06:30.110369 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 77716.4us
I20260709 13:06:30.007200 18874 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:30.008278 18852 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:30.123632 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 115434us
I20260709 13:06:30.124492 13988 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:30.125552 13988 leader_election.cc:400] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16.
I20260709 13:06:30.125849 13988 leader_election.cc:403] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:06:30.116958 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:30.116925 13989 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:30.126828 14020 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:30.127223 14021 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:30.127650 14021 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:30.126559 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:29.974316 19022 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.126s
I20260709 13:06:30.159571 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:30.126981 14522 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:30.098610 19431 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.968358 18744 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.130s
I20260709 13:06:30.175029 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:30.196308 13875 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:30.175980 14016 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:30.133944 13872 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
W20260709 13:06:30.185367 13915 connection.cc:256] RPC call timeout handler was delayed by -1.3711703430000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3026s for a response
I20260709 13:06:30.187563 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:30.202201 13871 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:30.114187 19263 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:30.118628 19222 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:30.208683 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:30.211258 13875 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:30.190512 14021 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:30.207537 14018 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:30.117220 19149 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:30.208388 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:30.214718 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:30.215274 13868 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:06:30.210440 19222 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.158293 19341 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:30.104758 19321 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 13:06:30.264495 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:30.265911 13870 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:06:30.267048 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:30.245740 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:30.274592 13942 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:30.263893 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:30.263754 13867 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:30.271767 13944 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:30.276352 13873 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:30.276394 13875 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
W20260709 13:06:30.217314 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 100506us
I20260709 13:06:30.279541 13989 leader_election.cc:400] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15.
I20260709 13:06:30.279937 13989 leader_election.cc:403] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:30.166172 19334 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:30.287444 13868 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:30.292268 14018 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:06:30.300830 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:30.189810 19261 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:30.313580 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:30.315317 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 189866us
I20260709 13:06:30.328400 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:30.265337 19346 raft_consensus.cc:2764] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:06:30.263610 19222 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:30.249398 19328 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:30.264333 19341 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:30.245704 19463 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16.
I20260709 13:06:30.343569 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:30.346042 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.347971 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:30.348778 14015 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:30.350085 13867 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:30.330680 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:30.355517 13873 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:30.320641 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:30.340021 14021 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:30.206384 19431 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:30.331521 19149 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15.
I20260709 13:06:30.392670 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:30.393114 13871 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:30.340168 19459 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.414605 14021 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:06:30.357602 19239 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.706s	user 0.007s	sys 0.006s
W20260709 13:06:30.406406 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.415985 13871 pending_rounds.cc:85] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 13
I20260709 13:06:30.417222 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:06:30.425211 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.429999 13871 pending_rounds.cc:107] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:06:30.431092 13871 raft_consensus.cc:2889] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:30.361423 19328 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:30.395628 19346 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:30.389747 19338 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:30.445616 19430 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:30.451279 19328 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:30.454321 19346 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:30.455646 19346 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:30.417753 19022 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.356990 19261 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:30.460235 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.483587 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 281397us
I20260709 13:06:30.484392 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:30.436856 19239 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11.
W20260709 13:06:30.488396 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.490847 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.489091 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:30.511438 19124 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:30.512144 19124 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.512655 19124 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:30.517833 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:30.519048 14015 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:30.522401 14019 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 9. Current term is 11. Ops: []
I20260709 13:06:30.433735 19459 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:30.539719 13868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:06:30.540786 13868 raft_consensus.cc:606] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:30.541181 13868 raft_consensus.cc:614] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:06:30.551153 14524 leader_election.cc:336] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.510s (SENT)
I20260709 13:06:30.551939 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:30.560170 13915 connection.cc:256] RPC call timeout handler was delayed by -1.091248137999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
W20260709 13:06:30.560889 13915 connection.cc:256] RPC call timeout handler was delayed by -1.07324813799994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15510000000000002s for a response
W20260709 13:06:30.561278 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0345481379998773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
W20260709 13:06:30.541680 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 414714us
I20260709 13:06:30.480829 19022 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:30.523203 19493 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:30.571287 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:30.575234 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.575790 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.611394 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:30.618155 13943 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:06:30.523447 19341 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:30.581575 19493 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:06:30.620611 19481 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:06:30.637044 19022 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:30.644840 14552 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:06:30.646787 19459 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:30.648211 19490 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:06:30.650779 19350 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:30.651154 19494 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:06:30.652508 19495 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:06:30.653174 19431 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:30.656345 14021 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:30.662513 14015 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:30.594244 19384 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
W20260709 13:06:30.776019 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 97014.8us
W20260709 13:06:30.776417 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.620887 19457 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.795727 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:30.809793 14015 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:06:30.817267 19261 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:30.818097 19261 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:30.818790 19261 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:30.824887 19479 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 13:06:30.834702 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2319) took 1126 ms. Trace:
I20260709 13:06:30.834846 19493 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:30.835210 14019 rpcz_store.cc:276] 0709 13:06:29.708597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.881036 (+172439us) service_pool.cc:224] Handling call
0709 13:06:29.881353 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:30.522892 (+641539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.834412 (+311520us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b540023b754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:30.834602 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1473743616}
W20260709 13:06:30.843673 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.855099 19491 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:06:30.858340 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2304) took 1204 ms. Trace:
I20260709 13:06:30.858812 14023 rpcz_store.cc:276] 0709 13:06:29.656211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.845797 (+189586us) service_pool.cc:224] Handling call
0709 13:06:29.884549 (+ 38752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:29.892348 (+  7799us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:30.122579 (+230231us) log.cc:844] Serialized 14 byte log entry
0709 13:06:30.857807 (+735228us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b180010dec0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287
0709 13:06:30.857927 (+   120us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:30.857983 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:30.858045 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.858137 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:30.858274 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6106,"prepare.queue_time_us":253796,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":23279,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":57922,"spinlock_wait_cycles":3104640,"thread_start_us":316040,"threads_started":3,"wal-append.queue_time_us":296149}
I20260709 13:06:30.861150 19466 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:30.816274 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:30.873594 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 57459.8us
I20260709 13:06:30.875263 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:30.885139 14015 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:06:30.894281 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:30.898541 14015 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:06:30.800050 19163 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:06:30.802868 19457 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:30.883211 19464 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:30.919832 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:30.899870 18996 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:06:30.921545 13988 leader_election.cc:336] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.167s (SENT)
I20260709 13:06:30.927094 14015 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:30.929200 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:30.887347 19466 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:06:30.924171 19464 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:30.937716 19492 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:06:30.910706 19379 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:06:30.944303 19467 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:30.947078 19321 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:30.948485 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:30.949384 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:30.991504 13941 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
W20260709 13:06:30.954566 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.955153 19261 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:30.961473 18996 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:30.962191 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 411080us
W20260709 13:06:30.968991 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.917418 19506 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.960868 19350 raft_consensus.cc:2955] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Committing config change with OpId 12.10: config changed from index 3 to 10, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:30.967614 13949 raft_consensus.cc:2955] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 3 to 10, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:30.987619 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:31.013796 14015 raft_consensus.cc:2376] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17. Re-sending same reply.
I20260709 13:06:30.989684 13948 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:31.015317 13839 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:31.019225 13870 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:06:31.020071 13870 raft_consensus.cc:606] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:31.020489 13870 raft_consensus.cc:614] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:06:31.025151 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 103665us
I20260709 13:06:30.988493 19464 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:30.995528 19338 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:31.038638 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:31.043103 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:31.066272 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:31.068302 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:31.026206 19261 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:31.078891 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:31.075191 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 184998us
I20260709 13:06:31.081575 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:31.089919 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:31.002696 19321 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:31.018044 19506 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:31.077775 19346 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:31.097958 14015 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
W20260709 13:06:31.096748 19263 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.843s	user 0.010s	sys 0.011s
I20260709 13:06:31.123868 19517 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:31.131554 19457 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:31.136184 13875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:06:31.157856 13875 raft_consensus.cc:606] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:31.158281 13875 raft_consensus.cc:614] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:31.140538 14023 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:06:31.158892 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 496854us
I20260709 13:06:31.140420 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:31.141363 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:31.160565 14017 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:31.141685 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:31.161612 14016 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:06:31.142755 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:31.143446 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:31.162755 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.144582 18996 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:31.165971 14022 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:31.151417 18874 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:31.154443 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:31.168569 14020 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:31.155802 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:31.164050 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.157385 19517 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:31.170832 19222 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.822s	user 0.010s	sys 0.004s
I20260709 13:06:31.182664 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:31.166942 19346 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:31.183627 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.186528 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:31.172451 18874 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:31.186728 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:31.167407 19457 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:31.188530 14524 leader_election.cc:336] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.880s (SENT)
I20260709 13:06:31.190450 14524 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:31.188814 14019 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:31.187175 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:31.186755 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:31.191636 13942 raft_consensus.cc:2368] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14.
I20260709 13:06:31.192025 13948 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 11: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:31.192111 19457 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:31.194192 19467 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:31.194456 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:31.195088 13942 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:06:31.195842 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:31.196578 13948 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:31.196394 19517 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:31.197855 19517 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:31.205133 14019 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:06:31.194514 19457 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:31.207849 18874 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:31.161651 19263 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:31.216784 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:31.217825 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.237716 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:31.238343 13873 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:31.217725 13942 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:31.247108 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:31.248495 13941 raft_consensus.cc:2368] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15.
I20260709 13:06:31.247290 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:31.249946 13948 raft_consensus.cc:2410] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:06:31.252197 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3051) took 1064 ms. Trace:
W20260709 13:06:31.252854 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.252724 13873 rpcz_store.cc:276] 0709 13:06:30.187449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.392262 (+204813us) service_pool.cc:224] Handling call
0709 13:06:31.251988 (+859726us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b54001aa254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:31.252125 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1942179712}
I20260709 13:06:31.187142 19222 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:31.190558 19346 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:31.271158 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 286316us
I20260709 13:06:31.267223 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:31.279289 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:31.281051 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:31.281992 13942 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:31.286016 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:31.294387 14522 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:31.296765 13945 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:06:31.298383 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 217386us
I20260709 13:06:31.299434 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:31.313630 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 120111us
I20260709 13:06:31.319113 13941 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:06:31.301775 19124 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.789s	user 0.016s	sys 0.001s
I20260709 13:06:31.322793 19321 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:31.296916 19467 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:31.323613 13941 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:06:31.325698 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:31.302443 19354 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:06:31.326805 14524 leader_election.cc:336] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.129s (SENT)
I20260709 13:06:31.288584 19417 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:06:31.338274 19346 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:31.338671 19515 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:06:31.332571 19354 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.385716 19354 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:31.390589 19512 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
W20260709 13:06:31.390777 19514 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.157s	user 0.000s	sys 0.010s
I20260709 13:06:31.362085 19506 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:31.393195 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:31.373790 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:31.385145 19519 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 13:06:31.395576 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 72897.2us
I20260709 13:06:31.361647 19384 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:31.328050 19124 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:31.406633 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:31.406045 13987 connection.cc:256] RPC call timeout handler was delayed by -1.0016025840000111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
I20260709 13:06:31.409049 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:31.404942 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.412343 13838 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:31.416388 13942 raft_consensus.cc:2368] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18.
I20260709 13:06:31.359608 19521 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:06:31.425230 19518 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:06:31.428896 19222 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:31.430176 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 111725us
I20260709 13:06:31.430436 14553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 916) took 1070 ms. Trace:
W20260709 13:06:31.431216 14524 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.156s (SENT)
I20260709 13:06:31.431001 14553 rpcz_store.cc:276] 0709 13:06:30.360456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.360626 (+   170us) service_pool.cc:224] Handling call
0709 13:06:30.400875 (+ 40249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.940102 (+539227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.940136 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:30.940665 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.184684 (+244019us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:31.184735 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.184800 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.430137 (+245337us) raft_consensus.cc:1685] finished
0709 13:06:31.430161 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.430331 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":538683,"raft.queue_time_us":538857,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":243624,"threads_started":1,"wal-append.queue_time_us":361246}
I20260709 13:06:31.392025 19263 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:31.435619 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 69760.3us
I20260709 13:06:31.444289 13838 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:31.448899 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:31.449690 14552 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:31.456714 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.397765 19341 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.451006 19354 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:31.493767 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 49576.5us
I20260709 13:06:31.499504 13866 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:31.410828 19455 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.504515 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:31.506668 13873 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:31.506992 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:31.507655 13867 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:31.409677 19384 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:31.492794 13987 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:31.540185 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:31.569952 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.562376 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:31.542690 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 47732.9us
I20260709 13:06:31.542590 13870 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:31.630427 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 182373us
W20260709 13:06:31.608851 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 66859.6us
I20260709 13:06:31.613979 13874 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:06:31.636260 19261 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:31.637571 19149 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:31.613749 19506 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:06:31.645150 19261 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:31.452608 19448 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:06:31.638662 19350 consensus_meta.cc:287] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 13:06:31.639611 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:31.640600 13870 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:31.618257 14552 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
W20260709 13:06:31.664999 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 223369us
I20260709 13:06:31.665928 13914 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:31.612177 19263 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.511185 19346 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:31.616207 19464 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.570s	user 0.013s	sys 0.000s
W20260709 13:06:31.661178 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 165570us
I20260709 13:06:31.498459 19341 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:31.511409 19521 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:31.668759 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:31.540617 19455 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:31.671584 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:31.658463 19448 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:06:31.675181 19515 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:06:31.698813 13838 leader_election.cc:400] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18.
I20260709 13:06:31.699313 13838 leader_election.cc:403] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:06:31.693454 19448 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:31.676204 19163 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:31.702131 13949 raft_consensus.cc:2410] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:31.667415 19263 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:31.695497 19528 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:06:31.708331 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:31.687517 19503 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:31.690398 19523 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:31.689801 19520 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:06:31.726073 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:31.701570 19507 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:31.676040 19346 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:31.678118 19464 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:31.690567 19489 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 13:06:31.709100 19163 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:31.729038 19149 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:31.726944 19417 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18.
I20260709 13:06:31.727246 19538 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:31.731137 19523 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:31.745319 19163 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:31.667048 19384 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:31.795781 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 184801us
I20260709 13:06:31.780176 19124 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:31.810402 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 141758us
I20260709 13:06:31.812237 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:31.797981 19149 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:31.809373 19384 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:31.820623 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:31.819115 19124 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:31.814167 19379 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:31.840183 13870 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:06:31.847404 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 121485us
I20260709 13:06:31.850802 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:31.764415 19521 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:31.896963 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46255.6us
I20260709 13:06:31.897951 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:31.846630 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:31.844913 19507 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:06:31.908882 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62356.5us
W20260709 13:06:31.911830 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 48503.8us
I20260709 13:06:31.848008 19354 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:31.912715 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:31.914122 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:31.849817 19455 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:31.918304 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:31.907269 19350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49438 (request call id 19) took 3171 ms. Trace:
I20260709 13:06:31.897596 19149 consensus_queue.cc:1059] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:06:31.848737 19517 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:06:31.910089 19534 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:31.911201 19507 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.939520 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:31.909970 19533 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:31.941123 13987 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:31.895411 19379 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:31.919484 19509 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:31.922292 19503 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:31.945729 13840 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:32.095587 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 149948us
I20260709 13:06:31.938150 19547 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:32.013679 19464 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:31.927124 19538 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.945842 19263 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:31.945029 19431 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:31.936299 19350 rpcz_store.cc:276] 0709 13:06:28.734396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.823877 (+ 89481us) service_pool.cc:224] Handling call
0709 13:06:28.941479 (+117602us) log.cc:844] Serialized 501 byte log entry
0709 13:06:29.365249 (+423770us) spinlock_profiling.cc:243] Waited 4.26 ms on lock 0x7b1800198640. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c0348a40 00007fb2c03434e4 00007fb2c0343455 00007fb2c338cb0b 00007fb2c3411b63 00007fb2c3439752 00007fb2c34319fa
0709 13:06:31.900635 (+2535386us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":167708,"raft.queue_time_us":363859,"raft.run_cpu_time_us":16147,"raft.run_wall_time_us":284272,"spinlock_wait_cycles":9787264,"thread_start_us":81142,"threads_started":1,"wal-append.queue_time_us":229548,"wal-append.run_cpu_time_us":2130,"wal-append.run_wall_time_us":1041226}
W20260709 13:06:31.991482 19541 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:06:31.958318 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.934191 19520 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:31.840837 19523 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:31.955847 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:32.067548 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.920946 18874 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:06:32.072491 19124 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:32.100021 19546 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:06:32.073611 19530 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.646s	user 0.004s	sys 0.000s
W20260709 13:06:32.075098 19529 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.640s	user 0.000s	sys 0.003s
I20260709 13:06:32.018309 19346 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:32.027093 19448 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:32.060348 19527 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.636s	user 0.000s	sys 0.002s
I20260709 13:06:32.044608 19163 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:31.920704 19341 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:31.944936 19507 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:32.076368 19531 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.640s	user 0.002s	sys 0.000s
I20260709 13:06:32.087285 19239 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.101172 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:32.094400 19503 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:32.107249 19538 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:32.125093 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 213926us
I20260709 13:06:32.093685 19149 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:32.109823 19547 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:32.104444 19261 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
W20260709 13:06:32.146955 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 205905us
I20260709 13:06:32.147320 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:32.130424 14561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1025) took 1257 ms. Trace:
I20260709 13:06:32.119540 19520 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:32.119985 19546 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:32.148161 14552 raft_consensus.cc:2368] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18.
I20260709 13:06:32.151093 13949 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:06:32.152791 13866 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:32.152781 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:32.199784 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:32.138480 19503 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:32.197798 13949 raft_consensus.cc:606] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:32.144166 19466 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.006s	sys 0.017s
I20260709 13:06:32.132604 19384 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:32.138813 19448 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:32.298717 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 99596.3us
W20260709 13:06:32.268411 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 115004us
W20260709 13:06:32.298848 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 100648us
I20260709 13:06:32.148307 14561 rpcz_store.cc:276] 0709 13:06:30.873212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.873392 (+   180us) service_pool.cc:224] Handling call
0709 13:06:30.873824 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.950121 (+ 76297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.950157 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:30.950681 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.172408 (+221727us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:31.172450 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.172520 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.124155 (+951635us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b24002cf0d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c01cc9c4 00007fb2c01caacb 00007fb2c0385dc9 00007fb2c0385a55 00007fb2c0384fb0 00007fb2c2ffb488 00007fb2c2ffbbe8 00007fb2c2fd3398 00007fb2c2fce76a 00007fb2c2fcdbb6
0709 13:06:32.125447 (+  1292us) raft_consensus.cc:1685] finished
0709 13:06:32.125473 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.130329 (+  4856us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68280,"raft.queue_time_us":69397,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"spinlock_wait_cycles":64016128,"thread_start_us":221344,"threads_started":1,"wal-append.queue_time_us":347245}
I20260709 13:06:32.138625 19489 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:32.303421 14553 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:32.311542 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:32.338951 13758 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 13)
I20260709 13:06:32.313735 19354 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.029s
W20260709 13:06:32.343144 13758 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 144822us
I20260709 13:06:32.345098 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:32.195752 19546 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:32.362911 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 51500.6us
I20260709 13:06:32.372682 14554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 743) took 1305 ms. Trace:
I20260709 13:06:32.373339 14554 rpcz_store.cc:276] 0709 13:06:31.067125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.067282 (+   157us) service_pool.cc:224] Handling call
0709 13:06:31.087023 (+ 19741us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:31.110687 (+ 23664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:31.110742 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:31.123068 (+ 12326us) log.cc:844] Serialized 52 byte log entry
0709 13:06:31.290159 (+167091us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:31.290216 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.290292 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.059402 (+769110us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:32.372211 (+312809us) raft_consensus.cc:1685] finished
0709 13:06:32.372246 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.372589 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13364,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":9511,"spinlock_wait_cycles":69543808,"thread_start_us":166590,"threads_started":1,"wal-append.queue_time_us":300360}
I20260709 13:06:32.145606 19239 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:32.388681 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:32.385367 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 782) took 1344 ms. Trace:
W20260709 13:06:32.393744 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 256384us
I20260709 13:06:32.141072 19417 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.224s
I20260709 13:06:32.394857 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:32.314222 19328 consensus_queue.cc:1059] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:06:32.391193 13869 rpcz_store.cc:276] 0709 13:06:31.040744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.040973 (+   229us) service_pool.cc:224] Handling call
0709 13:06:31.078290 (+ 37317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:31.087184 (+  8894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:31.087228 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:31.087757 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.154042 (+ 66285us) spinlock_profiling.cc:243] Waited 8.99 ms on lock 0x7b1800438be0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:06:31.302160 (+148118us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:31.302238 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.302337 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.074292 (+771955us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:32.385010 (+310718us) raft_consensus.cc:1685] finished
0709 13:06:32.385045 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.385272 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3161,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":4605,"spinlock_wait_cycles":65711232,"thread_start_us":145821,"threads_started":1,"wal-append.queue_time_us":335462}
I20260709 13:06:32.154035 19149 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:32.439399 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 50832.3us
I20260709 13:06:32.312613 19263 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 13:06:32.393905 19354 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:32.448081 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:32.383425 19534 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:32.442379 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 134574us
I20260709 13:06:32.379742 19466 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:06:32.344419 19384 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:32.367347 19350 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:06:32.360110 19539 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:06:32.345305 19462 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:06:32.403965 19489 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:32.393592 19459 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:32.456514 19346 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:32.434770 19379 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:32.439246 19261 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:32.486178 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:32.503841 19263 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:32.504683 19538 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:32.505424 19517 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:32.506989 19557 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:32.507575 19517 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:06:32.508473 19554 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:06:32.508553 19521 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.509197 19521 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:32.507100 19239 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:32.521068 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 214487us
I20260709 13:06:32.522276 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:32.468080 18996 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:06:32.496290 19558 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:06:32.520213 19556 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:06:32.536376 19509 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:32.526500 19328 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:32.537420 19328 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:32.537842 19328 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:32.543414 19455 raft_consensus.cc:3055] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:32.543942 19455 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:06:32.527374 19516 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.136s	user 0.003s	sys 0.003s
W20260709 13:06:32.550783 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 62654.1us
I20260709 13:06:32.495551 19459 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:32.497431 19261 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:32.558140 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:32.563382 19559 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 13:06:32.529986 19163 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:32.577862 19562 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:32.578684 19548 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.384s	user 0.000s	sys 0.001s
I20260709 13:06:32.579005 19261 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:32.581002 19521 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:32.485095 19489 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:32.581754 19525 log.cc:927] Time spent T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.090s	user 0.013s	sys 0.017s
I20260709 13:06:32.465389 19520 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:32.587829 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:32.589411 19384 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:32.590039 19384 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.590651 19384 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:32.552716 19558 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:32.574806 19555 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 13:06:32.575778 19553 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:06:32.576081 19560 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:06:32.597245 19354 raft_consensus.cc:3055] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:32.597503 19559 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:32.597967 19559 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:32.598917 19559 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:32.600289 19559 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:32.605940 19346 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.606674 19346 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:32.607429 19558 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:32.609098 19346 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:32.609678 19346 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.576602 19459 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:32.577208 19149 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:32.597785 19354 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:32.611299 19239 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.611769 19354 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:32.613147 19354 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:32.611806 19239 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:32.617302 19239 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:32.544927 19455 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:32.618554 19507 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:32.620150 13875 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:32.622646 19455 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:32.624006 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 65968.7us
I20260709 13:06:32.625600 19430 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:32.625088 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:32.627660 19459 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:32.638955 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:32.642720 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 177750us
I20260709 13:06:32.650013 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:32.650470 13987 leader_election.cc:400] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11.
I20260709 13:06:32.656397 13987 leader_election.cc:400] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13.
I20260709 13:06:32.656852 13987 leader_election.cc:403] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:06:32.642333 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 149977us
I20260709 13:06:32.664116 13875 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:32.583621 19489 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:32.675261 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.642369 19350 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:32.682204 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:32.692396 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:32.693445 19430 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:32.648723 19506 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:32.682806 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:32.686084 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:32.702426 13941 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:06:32.642333 19431 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:32.684216 13871 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:32.636363 19346 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:32.675557 19547 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:32.659572 19448 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:32.676828 19239 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.708056 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:32.707647 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:32.741593 13871 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:32.742419 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:32.743875 13839 leader_election.cc:400] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18.
W20260709 13:06:32.711853 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 257406us
I20260709 13:06:32.730647 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:32.732604 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:32.738823 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 82633.3us
I20260709 13:06:32.743100 13943 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:32.750288 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:32.754292 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:32.754906 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:32.755218 13874 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:32.706287 19459 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:32.759945 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:32.760833 14553 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:32.761952 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:32.762840 13873 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
W20260709 13:06:32.758104 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 270484us
I20260709 13:06:32.753278 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:32.772506 13987 leader_election.cc:400] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14.
I20260709 13:06:32.773108 13987 leader_election.cc:403] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:32.774434 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:32.776741 13941 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:32.808117 13987 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:32.696580 19328 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:32.679410 19566 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.086s
W20260709 13:06:32.836907 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 165045us
I20260709 13:06:32.723357 19506 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:32.840195 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:32.782589 19493 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14.
I20260709 13:06:32.844265 13987 leader_election.cc:400] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15.
I20260709 13:06:32.846745 13947 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:32.843946 13915 leader_election.cc:336] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.549s (SENT)
I20260709 13:06:32.847520 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:32.785504 19534 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:32.721352 19431 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:32.867215 19431 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:32.850476 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:32.872247 14019 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:32.873613 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:32.874409 14019 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:32.859700 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47575.8us
I20260709 13:06:32.834895 19538 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:32.769059 19338 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:06:32.854202 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:32.753924 19448 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:32.853997 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:32.852584 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:32.913259 14016 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:32.854846 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:32.852401 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:32.815546 19341 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:32.853252 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:32.917299 14017 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:32.914737 14020 raft_consensus.cc:2376] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 11. Re-sending same reply.
I20260709 13:06:32.753022 19239 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:32.851931 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:32.862440 19566 consensus_queue.cc:1243] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:32.879463 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:32.885931 13840 connection.cc:256] RPC call timeout handler was delayed by -1.3058818100000735s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
I20260709 13:06:32.918721 14018 raft_consensus.cc:2368] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:06:32.887045 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:32.889257 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:32.885120 13915 leader_election.cc:336] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.752s (SENT)
I20260709 13:06:32.883277 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:32.852017 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:32.922628 14021 raft_consensus.cc:2368] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:06:32.923810 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:32.924655 14021 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:32.740275 19546 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:32.849022 19507 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:32.918778 13840 connection.cc:256] RPC call timeout handler was delayed by -1.1030573709999771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 13:06:32.943456 13840 connection.cc:256] RPC call timeout handler was delayed by -1.0745818100000406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
I20260709 13:06:32.916388 19338 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:32.921960 13950 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:32.925833 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:32.851554 19328 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:32.947461 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:32.952937 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:32.953758 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 68656us
I20260709 13:06:32.953861 14020 raft_consensus.cc:2376] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 15. Re-sending same reply.
W20260709 13:06:32.954569 13915 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.775s (SENT)
I20260709 13:06:32.955004 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:32.956441 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:32.880539 19506 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:32.877975 19384 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:32.965484 14022 raft_consensus.cc:2376] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 15. Re-sending same reply.
I20260709 13:06:32.880746 19466 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:32.966992 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:32.967772 14018 raft_consensus.cc:2376] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 14. Re-sending same reply.
I20260709 13:06:32.884544 19561 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:32.965601 13947 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:32.966889 13872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f50263f80db844239948e5e67ef3b987"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
W20260709 13:06:32.959455 13987 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:49644 dropped due to backpressure: service queue is full with 50 items
I20260709 13:06:32.969215 13872 raft_consensus.cc:606] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:32.969702 13872 raft_consensus.cc:614] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:32.971323 14024 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:32.973364 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:32.974174 14017 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:32.951788 14021 raft_consensus.cc:2368] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10.
I20260709 13:06:32.979362 13872 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:32.981926 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:32.986768 19466 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:32.988673 19239 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:32.990031 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:32.989923 19466 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:32.995388 19346 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:32.996476 13872 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:32.996948 13871 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:32.997637 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:32.997997 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:32.998934 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:32.999259 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:33.000066 13871 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:33.000315 13872 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: []
W20260709 13:06:33.000507 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 45971.9us
I20260709 13:06:32.996361 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:33.012428 13915 leader_election.cc:336] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.844s (SENT)
I20260709 13:06:33.012475 14023 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:33.004237 13872 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:32.969256 13987 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:49644 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2922)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2380)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2376)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2379)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2372)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2930)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2375)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2935)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2911)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2914)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2936)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2940)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2944)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2945)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2948)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2949)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2377)
W20260709 13:06:33.014215 13915 leader_election.cc:336] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.975s (SENT)
I20260709 13:06:33.014375 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:33.017709 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:33.017532 19457 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:33.003759 13873 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:33.007898 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:33.020639 13867 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:33.023211 19457 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:06:33.025461 19503 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.668s	user 0.000s	sys 0.012s
I20260709 13:06:33.027494 13868 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:33.030583 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.023206 13987 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:49628 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2394)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2380)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2376)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2385)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2379)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2372)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2390)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2930)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2391)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2392)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2375)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2383)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2935)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2384)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2911)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2914)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2936)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2940)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2944)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2945)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2948)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2949)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2377)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2386)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2387)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2388)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2395)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2396)
W20260709 13:06:33.033004 13840 leader_election.cc:336] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.540s (SENT)
I20260709 13:06:33.054482 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.056295 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.057127 14018 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:32.949249 19338 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:33.048578 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.052181 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:33.063616 14024 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:32.959765 19509 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.066629 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.077643 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.077910 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:33.078715 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:33.079558 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:33.065678 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:06:33.082859 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 139849us
I20260709 13:06:33.085079 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:33.079371 13867 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:33.086483 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.088316 13942 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:33.087201 14024 raft_consensus.cc:2376] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 12. Re-sending same reply.
I20260709 13:06:33.095269 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.096123 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.096838 14016 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:33.096392 14024 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:33.109498 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.110728 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.120436 14024 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:33.120775 13872 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:33.103513 19149 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:33.122006 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:33.114292 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.123512 14023 raft_consensus.cc:2376] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 15. Re-sending same reply.
W20260709 13:06:33.125214 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.114941 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.025974 19503 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:33.118660 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.140890 14018 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:33.130121 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.141892 14024 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:33.133636 13872 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:33.135123 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:33.127485 19149 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.116776 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.143390 14016 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:33.144833 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.145732 14018 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:33.147027 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.161948 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:33.169663 14019 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:33.170574 14017 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:33.173511 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.176895 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.178093 13870 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:33.178272 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 125943us
W20260709 13:06:33.179369 13840 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.758s (SENT)
I20260709 13:06:33.179900 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.180495 14018 raft_consensus.cc:2376] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 13. Re-sending same reply.
I20260709 13:06:33.181314 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:33.181600 13840 leader_election.cc:336] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.795s (SENT)
I20260709 13:06:33.087703 19509 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:33.182317 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:33.185120 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.186237 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:33.189119 13840 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.813s (SENT)
W20260709 13:06:33.201159 13840 leader_election.cc:336] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.015s (SENT)
W20260709 13:06:33.202913 13915 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.531s (SENT)
I20260709 13:06:33.205602 14522 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:33.214713 14021 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
W20260709 13:06:33.215893 13840 leader_election.cc:336] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.146s (SENT)
W20260709 13:06:33.220566 13840 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.917s (SENT)
I20260709 13:06:33.236119 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.236958 14016 raft_consensus.cc:2376] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 14. Re-sending same reply.
I20260709 13:06:33.239382 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:33.240295 13840 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.074s (SENT)
W20260709 13:06:33.241664 13840 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.082s (SENT)
I20260709 13:06:33.242830 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.243505 14021 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:33.159253 19149 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:33.244534 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.246076 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.246847 14023 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:33.171645 19570 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:33.253046 14553 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
W20260709 13:06:33.254182 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 90155.3us
I20260709 13:06:33.255551 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.249095 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:33.257129 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.257792 14017 raft_consensus.cc:2376] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 15. Re-sending same reply.
I20260709 13:06:33.259917 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:33.260139 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 57285.3us
I20260709 13:06:33.260654 14021 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
W20260709 13:06:33.260874 13915 leader_election.cc:336] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.551s (SENT)
I20260709 13:06:33.261574 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.262310 14021 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:33.263165 13915 leader_election.cc:336] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.594s (SENT)
I20260709 13:06:33.263628 13915 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:33.181903 14018 raft_consensus.cc:2376] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 14. Re-sending same reply.
I20260709 13:06:33.265354 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.265388 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.266161 14018 raft_consensus.cc:2376] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 11. Re-sending same reply.
I20260709 13:06:33.266260 14021 raft_consensus.cc:2376] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 14. Re-sending same reply.
I20260709 13:06:33.267256 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.268235 14021 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:33.263381 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:33.269872 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.271035 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.271628 14019 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:33.272460 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.273507 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.294507 14021 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:33.281316 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.304279 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.278712 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.340162 14018 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:33.340723 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.341101 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.341362 13947 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:33.341877 14018 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:33.279333 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.342684 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.343883 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.344749 13946 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:33.344002 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.335031 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.347360 13947 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:33.348274 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.349013 13947 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:33.235479 18744 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:33.335031 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.353139 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.353729 14021 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:33.335690 13942 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:06:33.287083 13915 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.608s (SENT)
I20260709 13:06:33.356128 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:33.357144 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 70102.3us
I20260709 13:06:33.357334 13942 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:06:33.286108 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:33.254395 19459 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:33.252127 19354 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:06:33.357875 13915 leader_election.cc:336] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.663s (SENT)
I20260709 13:06:33.359263 14021 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:33.364441 13942 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:33.367606 14018 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:33.345854 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.381628 13948 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:06:33.382130 13948 pending_rounds.cc:85] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 8
I20260709 13:06:33.382365 13948 pending_rounds.cc:107] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:06:33.382599 13948 raft_consensus.cc:2889] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:33.244750 18874 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:33.337299 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 73210.8us
W20260709 13:06:33.385339 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 99378.9us
I20260709 13:06:33.387212 13840 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:33.337688 14016 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:33.260497 19462 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:33.261225 19570 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:33.294565 19558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.012s	sys 0.030s
I20260709 13:06:33.275890 13874 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:33.338827 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.358480 19561 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:33.337181 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.390232 14021 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:06:33.397377 14019 raft_consensus.cc:2410] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:33.158862 19338 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:33.273181 19417 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:33.398208 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.399194 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.417474 14016 raft_consensus.cc:2376] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 12. Re-sending same reply.
I20260709 13:06:33.418481 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.419108 14016 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:33.419919 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.402590 19493 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:33.403385 19261 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:33.395769 18744 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:33.244292 18996 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.140s
I20260709 13:06:33.408679 13947 pending_rounds.cc:85] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 7
I20260709 13:06:33.448556 13947 pending_rounds.cc:107] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:06:33.448638 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.414424 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.449322 14021 raft_consensus.cc:2376] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 15. Re-sending same reply.
I20260709 13:06:33.449525 14019 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:33.414776 14017 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:33.450814 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.451541 14017 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:33.337730 19569 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:33.267292 19350 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.132s
I20260709 13:06:33.383915 19455 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.384454 19321 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:06:33.446414 14524 leader_election.cc:336] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.904s (SENT)
I20260709 13:06:33.445868 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.447867 14553 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:06:33.400169 19539 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:33.411374 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:33.379835 19568 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:33.395726 19534 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.423512 19354 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:33.454384 13987 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:33.423264 19493 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:33.395594 19341 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:33.395555 19462 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:33.448874 13947 raft_consensus.cc:2889] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:33.395999 19570 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:33.415326 19417 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.452821 19561 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:33.460507 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 75014.4us
I20260709 13:06:33.461764 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.422461 19239 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.465037 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.398972 18874 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:33.455554 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.470618 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:33.453424 18744 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:33.471594 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 60721.7us
I20260709 13:06:33.509586 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:33.447600 19520 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:06:33.466506 19448 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:33.516240 13840 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:33.520671 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:33.480496 19417 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:33.473415 19489 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:33.474277 19557 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:33.516076 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 45549.1us
I20260709 13:06:33.526818 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.531683 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:33.531723 14021 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:33.502959 18744 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:33.484684 19585 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:06:33.479113 19462 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:33.521157 19566 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:33.465626 19539 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:33.461493 19455 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:33.539381 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:33.534584 19573 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
I20260709 13:06:33.541742 14019 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:06:33.521562 19263 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:06:33.501755 19341 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:33.534272 19448 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:33.582046 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 68685us
I20260709 13:06:33.578383 14019 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:33.534572 19557 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.583847 13838 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:33.584925 14019 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:33.586539 19603 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:33.586879 14019 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:33.587242 19603 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.587836 19603 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:33.588927 14019 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:06:33.590860 14019 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:33.501389 19239 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:33.590196 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58556.3us
I20260709 13:06:33.600890 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:33.593942 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 54641.8us
I20260709 13:06:33.599081 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:33.538633 19520 raft_consensus.cc:3055] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:33.607115 13838 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:33.608457 14016 raft_consensus.cc:2376] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10. Re-sending same reply.
I20260709 13:06:33.542663 19566 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:33.610044 13941 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.009s	sys 0.005s
W20260709 13:06:33.597824 19547 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.858s	user 0.011s	sys 0.001s
I20260709 13:06:33.595950 19557 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:33.611933 13941 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.010s	sys 0.005s
I20260709 13:06:33.612994 13838 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:33.597592 19503 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:33.602355 19506 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:33.604678 19261 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:33.614449 19566 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:33.611361 19431 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:33.616744 19448 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:33.610038 19520 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:33.621337 19547 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13.
I20260709 13:06:33.643761 19239 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:33.644536 19239 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:33.626984 19546 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:33.627228 19509 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:33.574273 19354 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:33.567116 19493 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:33.631240 19602 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:33.661123 19604 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:33.674402 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:33.637961 19506 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:33.510670 19561 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:33.475385 19534 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:33.675225 14553 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
W20260709 13:06:33.659472 19588 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 13:06:33.745602 13988 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:33.722591 19493 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:33.744335 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 105705us
I20260709 13:06:33.731104 18874 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:33.755482 19578 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:06:33.749472 19239 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:33.741788 19459 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.742079 19261 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:33.766216 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:33.775086 19459 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:33.780239 19583 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.233s	user 0.001s	sys 0.001s
W20260709 13:06:33.780720 19559 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.180s	user 0.013s	sys 0.004s
I20260709 13:06:33.781636 19534 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:33.782891 13915 leader_election.cc:336] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.787s (SENT)
I20260709 13:06:33.816596 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:33.774418 19506 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:33.769318 19455 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:33.786597 19539 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:33.761466 19503 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.768518 19612 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:33.791069 19596 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.190s	user 0.003s	sys 0.000s
I20260709 13:06:33.792375 19341 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:33.755842 19520 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:33.795724 19587 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
W20260709 13:06:33.797150 19590 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.339s	user 0.000s	sys 0.000s
I20260709 13:06:33.801146 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:33.801240 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:33.801862 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 168483us
I20260709 13:06:33.836602 14551 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:06:33.765421 19239 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.761675 19149 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:33.807283 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2639) took 1023 ms. Trace:
W20260709 13:06:33.826002 13915 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.817s (SENT)
W20260709 13:06:33.826761 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 343838us
I20260709 13:06:33.803050 19417 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:33.838414 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:33.844609 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:06:33.846555 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.827867 18996 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:33.841827 13943 rpcz_store.cc:276] 0709 13:06:32.783985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.784515 (+   530us) service_pool.cc:224] Handling call
0709 13:06:32.839656 (+ 55141us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.876697 (+ 37041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.876732 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.577427 (+700695us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b2c000e5d98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:06:33.601039 (+ 23612us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.692496 (+ 91457us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:33.692557 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.692673 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.799070 (+106397us) raft_consensus.cc:1685] finished
0709 13:06:33.799104 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.807196 (+  8092us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36731,"raft.queue_time_us":36025,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"spinlock_wait_cycles":3056896,"thread_start_us":61273,"threads_started":1,"wal-append.queue_time_us":153572}
I20260709 13:06:33.882617 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:33.885449 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:33.888617 19520 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:33.895852 19614 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.851225 18874 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:33.896497 19614 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:33.897190 18874 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:33.902949 19616 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:33.903420 19616 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.903761 19616 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:33.903822 19149 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:33.904444 19149 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:33.909289 19584 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.503s	user 0.000s	sys 0.002s
I20260709 13:06:33.911499 19417 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:33.904969 19149 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:33.911530 19459 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:33.900643 19384 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:33.914209 19384 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:33.914659 19384 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:33.915864 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:33.916733 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2922) took 1058 ms. Trace:
I20260709 13:06:33.917066 14022 rpcz_store.cc:276] 0709 13:06:32.858154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.915547 (+1057393us) service_pool.cc:224] Handling call
0709 13:06:33.916664 (+  1117us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:33.938715 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 53286.3us
I20260709 13:06:33.939821 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:33.848623 19239 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:33.905495 19603 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:33.945873 13915 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:33.952167 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:33.954658 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2629) took 1264 ms. Trace:
I20260709 13:06:33.832511 19503 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:33.955130 13949 rpcz_store.cc:276] 0709 13:06:32.690491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.690627 (+   136us) service_pool.cc:224] Handling call
0709 13:06:32.690991 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.691411 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.691455 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:32.771904 (+ 80449us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b2c000e5d98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:06:32.784577 (+ 12673us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.336729 (+552152us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:33.336783 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.336864 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.908452 (+571588us) spinlock_profiling.cc:243] Waited 9.77 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:33.954393 (+ 45941us) raft_consensus.cc:1685] finished
0709 13:06:33.954418 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.954583 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1689,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"spinlock_wait_cycles":52744832,"thread_start_us":497198,"threads_started":1,"wal-append.queue_time_us":568552}
W20260709 13:06:33.949473 19611 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:06:33.962257 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3074) took 1287 ms. Trace:
I20260709 13:06:33.948094 19562 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:33.966075 19561 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:33.967816 13869 rpcz_store.cc:276] 0709 13:06:32.674848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.674993 (+   145us) service_pool.cc:224] Handling call
0709 13:06:32.699432 (+ 24439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.699777 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.699815 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:32.709744 (+  9929us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.241829 (+532085us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:33.241887 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.246306 (+  4419us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.794937 (+548631us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:33.956576 (+161639us) raft_consensus.cc:1685] finished
0709 13:06:33.956606 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.962142 (+  5536us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255172992,"thread_start_us":499001,"threads_started":1,"wal-append.queue_time_us":772469}
I20260709 13:06:33.833910 19612 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:33.981348 19612 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:33.982597 19614 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:33.984645 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:33.991849 19600 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:33.992446 19600 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:33.950074 19457 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:33.883831 19534 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:06:33.976064 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:33.985491 14552 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:06:33.958706 18996 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:34.008323 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:33.999023 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:34.009616 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 169012us
I20260709 13:06:34.001349 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:34.001960 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:34.005061 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:34.012368 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:33.852855 19261 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.018989 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:33.985461 19570 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:34.028028 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:34.028543 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.026067 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:34.029522 13948 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:34.032419 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:33.989549 19346 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:06:34.032151 19431 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:33.970258 19562 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:34.043918 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:34.045948 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:34.055771 19524 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:34.057690 19503 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:34.018796 18996 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:34.058573 18996 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:34.064582 19462 consensus_meta.cc:287] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.001s
I20260709 13:06:34.065033 13950 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:34.065189 19462 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.066314 19570 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:34.027537 19459 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:34.067534 19459 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:33.993615 19600 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.068662 19459 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.068951 19384 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.070364 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:34.071312 14020 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:34.071790 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2939) took 1201 ms. Trace:
I20260709 13:06:34.072132 14020 rpcz_store.cc:276] 0709 13:06:32.870278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.070029 (+1199751us) service_pool.cc:224] Handling call
0709 13:06:34.071722 (+  1693us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:34.073813 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:34.074242 19562 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:34.074524 14020 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:34.075654 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:34.076268 14020 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:34.016968 19614 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:06:34.077142 19604 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.064675 19566 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.089608 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:34.080539 19462 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:34.080917 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2944) took 1208 ms. Trace:
I20260709 13:06:34.103178 14023 rpcz_store.cc:276] 0709 13:06:32.872547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.079866 (+1207319us) service_pool.cc:224] Handling call
0709 13:06:34.080258 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.080496 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.080534 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.080592 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:34.080640 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.080709 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.080844 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:34.083603 18744 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.002s
I20260709 13:06:34.078377 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3016) took 1116 ms. Trace:
I20260709 13:06:34.030628 19341 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:34.091008 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3069) took 1934 ms. Trace:
I20260709 13:06:34.093828 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:34.030978 19222 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:34.112938 13870 rpcz_store.cc:276] 0709 13:06:32.151638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.151762 (+   124us) service_pool.cc:224] Handling call
0709 13:06:32.152069 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.178582 (+1026513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.085809 (+907227us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002e86d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:34.086010 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2359077760}
I20260709 13:06:34.090620 14020 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:34.128535 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3087) took 1011 ms. Trace:
I20260709 13:06:34.129561 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:34.032429 19534 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.130936 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:34.028678 19457 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:34.131882 14020 raft_consensus.cc:2410] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:06:34.106519 13874 rpcz_store.cc:276] 0709 13:06:32.961963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.962124 (+   161us) service_pool.cc:224] Handling call
0709 13:06:34.078073 (+1115949us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54000a8f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:34.078268 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624920960}
I20260709 13:06:34.120494 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:34.124948 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:34.134323 14553 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:34.134460 14560 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 9.
W20260709 13:06:34.126914 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 83088.7us
I20260709 13:06:34.136971 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:34.129061 13873 rpcz_store.cc:276] 0709 13:06:33.117348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.117480 (+   132us) service_pool.cc:224] Handling call
0709 13:06:33.136162 (+ 18682us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:06:33.176543 (+ 40381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.176594 (+    51us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:06:33.187923 (+ 11329us) log.cc:844] Serialized 104 byte log entry
0709 13:06:33.454340 (+266417us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:33.454484 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.454690 (+   206us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.790214 (+335524us) spinlock_profiling.cc:243] Waited 77.6 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:34.107281 (+317067us) raft_consensus.cc:1685] finished
0709 13:06:34.107307 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.128455 (+ 21148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22498,"raft.queue_time_us":46812,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":71988,"spinlock_wait_cycles":178550656,"thread_start_us":150795,"threads_started":1,"wal-append.queue_time_us":273493}
I20260709 13:06:34.140712 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:34.141340 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:34.141561 14020 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:34.145114 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:34.041859 19431 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.040185 19149 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:34.071799 19384 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:34.154584 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:34.153776 14558 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:34.181758 13988 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:34.103349 19533 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:34.206360 14561 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:34.130498 19467 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:34.087628 19604 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.150435 19570 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:34.130342 19328 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:34.128614 19539 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:34.225688 13875 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:34.196087 19222 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:34.112633 19341 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:34.234490 19533 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:34.234668 19321 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:34.148142 19462 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:34.236784 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:34.236419 19616 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.238654 19566 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:34.231194 19606 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:34.242880 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:34.243156 19600 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:34.244542 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.245332 19603 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:34.104189 19557 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.246898 19467 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.127772 18744 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.161834 19431 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:34.230944 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 85942.7us
I20260709 13:06:34.258500 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:34.261952 19321 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:34.276886 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:34.260154 19539 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.272497 19606 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:34.281636 19606 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:34.283001 19539 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:34.283823 19539 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:34.284315 19539 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.284669 19557 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:34.286505 19467 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:34.289125 18744 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:34.279433 19321 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:34.291584 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:34.295758 13945 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
W20260709 13:06:34.295598 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 310450us
I20260709 13:06:34.198553 19534 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.303907 14553 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:34.273590 19462 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.263903 19328 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.302982 19384 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:34.318497 14023 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:34.308980 18744 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.303771 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:34.312036 19561 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:34.341238 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:34.353070 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:34.310972 19557 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:06:34.337451 19384 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:34.338784 18874 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:34.373597 13838 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:34.277109 19457 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:34.362012 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 314130us
I20260709 13:06:34.356218 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:34.371420 14023 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:06:34.263335 13948 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
W20260709 13:06:34.390769 19627 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:06:34.357714 19604 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:34.394747 19338 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:34.400380 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 44284.4us
W20260709 13:06:34.401266 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.412518 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:34.413919 13949 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
W20260709 13:06:34.359944 19493 consensus_meta.cc:287] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.606s	user 0.006s	sys 0.007s
I20260709 13:06:34.347386 19623 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:34.426813 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.356907 19603 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:34.339818 19562 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.369536 19384 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:34.453259 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:34.454936 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:34.350349 18744 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.417595 19559 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:34.318819 19341 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.338528 19533 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.476999 14020 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:34.475500 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:34.488364 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:34.446735 19346 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:34.289933 19606 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.342585 19222 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.456879 19566 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:34.454460 18874 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:34.454811 19493 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:34.507550 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:34.467422 19559 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.488862 19622 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:34.526785 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:34.533741 19562 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:34.538861 19625 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:06:34.541095 19384 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.556111 19606 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:34.566313 19616 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:34.528187 19539 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:34.489117 19149 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:06:34.471000 19520 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:06:34.523304 19559 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:34.567775 19462 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:34.590142 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2402) took 1504 ms. Trace:
I20260709 13:06:34.604473 14021 rpcz_store.cc:276] 0709 13:06:33.086034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.909162 (+823128us) service_pool.cc:224] Handling call
0709 13:06:33.909489 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.909836 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.909871 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.918471 (+  8600us) log.cc:844] Serialized 26 byte log entry
0709 13:06:34.111188 (+192717us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:34.160897 (+ 49709us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.161458 (+   561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.161535 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:34.589797 (+428262us) raft_consensus.cc:1685] finished
0709 13:06:34.589830 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.590029 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":282,"prepare.queue_time_us":80633,"raft.queue_time_us":56294,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":13230,"spinlock_wait_cycles":12462464,"thread_start_us":192262,"threads_started":1,"wal-append.queue_time_us":371952}
I20260709 13:06:34.597792 19384 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:34.626741 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 184907us
I20260709 13:06:34.529147 19493 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:34.648319 13868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.026s	sys 0.088s
W20260709 13:06:34.648821 13868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.026s	sys 0.088s
I20260709 13:06:34.668290 13949 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:34.670558 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:34.665448 19328 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:34.690522 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 213749us
W20260709 13:06:34.691641 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.648667 19570 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:06:34.700663 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:34.692836 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.701493 13949 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:34.704878 13943 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:34.710426 13948 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:34.677654 19384 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:06:34.680855 19517 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:06:34.715641 13949 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
W20260709 13:06:34.706485 19354 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.024s	sys 0.058s
I20260709 13:06:34.679912 19431 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.717299 14018 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:06:34.716288 13838 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:34.731565 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2940) took 1861 ms. Trace:
I20260709 13:06:34.704092 19534 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:34.732159 14018 rpcz_store.cc:276] 0709 13:06:32.870683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.716634 (+1845951us) service_pool.cc:224] Handling call
0709 13:06:34.717040 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:34.731212 (+ 14172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.731494 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:34.730871 18996 consensus_meta.cc:287] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.672s	user 0.015s	sys 0.000s
I20260709 13:06:34.708635 19602 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:34.754856 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 281229us
W20260709 13:06:34.731056 19354 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.025s	sys 0.058s
I20260709 13:06:34.717803 18744 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:34.734695 19417 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:34.715171 19613 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:34.762221 13874 raft_consensus.cc:2393] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 19: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:34.703392 19222 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:06:34.758615 18996 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.756932 19562 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:06:34.765486 19644 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:34.766407 18996 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:34.728757 19623 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:06:34.767094 19623 consensus_queue.cc:1243] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:34.759717 19507 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:06:34.760372 19565 consensus_queue.cc:1059] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:06:34.768922 19565 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:34.769155 14024 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:34.769330 19565 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:34.771764 13989 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.292s (SENT) [suppressed 72 similar messages]
I20260709 13:06:34.776110 19565 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.767119 18996 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:34.778242 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2411) took 1515 ms. Trace:
I20260709 13:06:34.777859 18996 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:34.780341 19565 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:34.783552 19431 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:34.757980 19602 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.712704 19616 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:06:34.778683 14021 rpcz_store.cc:276] 0709 13:06:33.262377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.631462 (+1369085us) service_pool.cc:224] Handling call
0709 13:06:34.663741 (+ 32279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.678918 (+ 15177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.678982 (+    64us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.679055 (+    73us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:34.679104 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.777949 (+ 98845us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b5400411754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:34.778002 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.778170 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34067328}
I20260709 13:06:34.797634 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:34.797770 19559 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.800724 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3085) took 1714 ms. Trace:
I20260709 13:06:34.801261 19559 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:34.802438 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:34.803702 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:34.806454 19644 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:34.806905 19644 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:34.807857 19644 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.808342 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:34.808897 19644 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:34.809020 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:34.809350 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:34.789374 19638 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:06:34.809985 13875 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:06:34.811586 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2404) took 1707 ms. Trace:
I20260709 13:06:34.811651 14023 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:34.812389 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 639) took 1427 ms. Trace:
I20260709 13:06:34.813130 14023 rpcz_store.cc:276] 0709 13:06:33.384541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.773323 (+1388782us) service_pool.cc:224] Handling call
0709 13:06:34.773814 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.812134 (+ 38320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.812338 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:34.814507 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:34.815141 14023 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:06:34.795553 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:34.809031 13870 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 20.
I20260709 13:06:34.818770 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:34.813109 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:34.820286 13948 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:34.819473 14023 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:34.822239 14524 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:34.823169 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:34.824982 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:34.826089 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.812040 14016 rpcz_store.cc:276] 0709 13:06:33.103972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.247105 (+1143133us) service_pool.cc:224] Handling call
0709 13:06:34.247459 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:34.247861 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:34.247892 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:34.248433 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 13:06:34.454506 (+206073us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:34.454555 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.454644 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:34.811386 (+356742us) raft_consensus.cc:1685] finished
0709 13:06:34.811407 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.811520 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1834,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":205668,"threads_started":1,"wal-append.queue_time_us":439003}
I20260709 13:06:34.827443 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:34.827612 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:34.828440 14023 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:34.801223 13868 rpcz_store.cc:276] 0709 13:06:33.085903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.092728 (+  6825us) service_pool.cc:224] Handling call
0709 13:06:34.086285 (+993557us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:34.086818 (+   533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.086865 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:34.087614 (+   749us) log.cc:844] Serialized 52 byte log entry
0709 13:06:34.649260 (+561646us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:34.649312 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.649399 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:34.799358 (+149959us) raft_consensus.cc:1685] finished
0709 13:06:34.799383 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.800448 (+  1065us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b54000a8f50. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:34.800653 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15337,"raft.queue_time_us":1830,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":8080,"spinlock_wait_cycles":2241968512,"thread_start_us":561124,"threads_started":1,"wal-append.queue_time_us":609239}
I20260709 13:06:34.828138 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:34.830688 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:34.831836 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:34.828188 14016 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:34.840022 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:34.791024 19639 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:06:34.739816 19506 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:06:34.761832 19417 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.848093 14524 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:34.790267 19631 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:06:34.864665 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:34.840845 13943 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:34.866757 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:34.788976 19239 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.830s	user 0.011s	sys 0.001s
I20260709 13:06:34.791606 19628 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.816359 19602 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:34.830454 19523 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:34.870880 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:34.871208 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:34.874709 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:34.874876 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:34.892642 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:34.843043 19567 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:34.894652 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:34.895277 14552 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:34.876121 14016 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
W20260709 13:06:34.883562 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.852129 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:34.896479 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:34.896942 14023 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:34.897150 13943 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:34.858273 13948 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:34.890724 14559 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:34.897856 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:34.897995 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:34.898483 14016 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:34.898619 14561 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:34.899271 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:34.899858 19640 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:06:34.895725 19328 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:34.888059 19646 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.900648 19328 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:34.901185 19328 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:34.901131 19646 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.904062 19565 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:34.863765 19417 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:34.904937 19646 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:34.878763 19239 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.908355 19239 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:34.880586 19628 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:34.782513 18996 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:34.915031 19321 consensus_meta.cc:287] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.602s	user 0.012s	sys 0.001s
I20260709 13:06:34.877228 19636 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:34.916570 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:34.916965 19636 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:34.917424 19636 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:34.864226 19338 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:34.919981 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:34.898581 13838 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:34.864773 19507 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:34.926713 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 261694us
I20260709 13:06:34.927611 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:34.929662 14016 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:06:34.929929 13943 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:34.921329 14524 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:34.846321 19261 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.864593 19520 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.862934 19629 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:34.943573 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 642) took 1545 ms. Trace:
I20260709 13:06:34.881281 19462 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:34.944119 14016 rpcz_store.cc:276] 0709 13:06:33.385449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.928543 (+1543094us) service_pool.cc:224] Handling call
0709 13:06:34.929022 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:34.930374 (+  1352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.930643 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:34.945127 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:34.946151 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:34.931059 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:34.931879 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 160156us
W20260709 13:06:34.924002 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 241059us
W20260709 13:06:34.937040 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.936991 14561 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:06:34.942092 14023 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:34.949277 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:34.969641 14552 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:34.970965 14016 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:34.971647 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 643) took 1586 ms. Trace:
I20260709 13:06:34.974689 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:34.974810 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:34.971993 14016 rpcz_store.cc:276] 0709 13:06:33.385744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.946243 (+1560499us) service_pool.cc:224] Handling call
0709 13:06:34.970770 (+ 24527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.971444 (+   674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.971600 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:34.819480 19613 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:34.981485 13988 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:34.982589 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:34.984015 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 56489.3us
I20260709 13:06:34.946697 19506 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:34.998919 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:34.978698 19600 consensus_meta.cc:287] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.701s	user 0.014s	sys 0.000s
I20260709 13:06:35.004544 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:34.881552 19459 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:06:34.946568 19467 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:34.966971 19462 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:34.915599 19321 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.014591 13867 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:35.016645 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:35.001933 19566 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:35.025013 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2403) took 1928 ms. Trace:
I20260709 13:06:34.948484 19507 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.005308 19489 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:35.029559 14553 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:34.971926 19338 raft_consensus.cc:697] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:34.948359 19493 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:35.017576 19263 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:35.066375 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:35.065312 14022 rpcz_store.cc:276] 0709 13:06:33.096160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.917993 (+821833us) service_pool.cc:224] Handling call
0709 13:06:33.925050 (+  7057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.925568 (+   518us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:34.222386 (+296818us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.688469 (+466083us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:34.689032 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:06:34.689211 (+   179us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:34.698975 (+  9764us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.700229 (+  1254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.700304 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:35.024710 (+324406us) raft_consensus.cc:1685] finished
0709 13:06:35.024731 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.024913 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":945,"prepare.queue_time_us":113764,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":3848,"raft.queue_time_us":67065,"raft.run_cpu_time_us":437,"raft.run_wall_time_us":13464,"thread_start_us":701222,"threads_started":3,"wal-append.queue_time_us":438335}
I20260709 13:06:35.002025 19646 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:35.027820 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:34.919605 19523 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:35.073489 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 56931.5us
I20260709 13:06:35.085668 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:35.088943 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:34.948926 19520 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.000325 19652 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:34.948796 19261 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.027840 19239 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:35.038831 14559 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
W20260709 13:06:35.089785 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 240264us
I20260709 13:06:35.099005 14558 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:06:35.090822 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:35.068147 19602 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.008086 19616 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:35.092417 13950 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:35.070398 19534 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:35.064494 19462 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:35.014873 19467 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:35.147076 14559 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:35.148355 14022 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:35.152236 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2950) took 1765 ms. Trace:
I20260709 13:06:35.152621 14022 rpcz_store.cc:276] 0709 13:06:33.386954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.090478 (+1703524us) service_pool.cc:224] Handling call
0709 13:06:35.147859 (+ 57381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.151990 (+  4131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.152180 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:35.031385 19506 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:35.009914 19457 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:06:35.091826 19646 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.092430 19570 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:35.018803 19629 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.094s
I20260709 13:06:35.078318 19656 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:35.097062 19533 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:35.169412 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:35.172541 14022 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:06:35.172516 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 168089us
I20260709 13:06:35.173714 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2413) took 1753 ms. Trace:
I20260709 13:06:35.174216 14022 rpcz_store.cc:276] 0709 13:06:33.419857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.159454 (+1739597us) service_pool.cc:224] Handling call
0709 13:06:35.172017 (+ 12563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.173329 (+  1312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.173642 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:35.174877 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:35.173295 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 74089.1us
I20260709 13:06:35.154851 19239 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:35.154960 19467 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:35.091034 19263 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.184414 13945 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:35.163801 19646 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:35.187374 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2414) took 1156 ms. Trace:
I20260709 13:06:35.191740 14022 rpcz_store.cc:276] 0709 13:06:34.031368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.175125 (+1143757us) service_pool.cc:224] Handling call
0709 13:06:35.186588 (+ 11463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.186867 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.186907 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:35.186967 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:35.187034 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.187116 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.187296 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:35.188122 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:35.149966 19559 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:06:35.149762 19521 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:06:35.150344 19602 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.166425 19533 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:35.179745 19599 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.562s	user 0.000s	sys 0.009s
I20260709 13:06:35.166500 19656 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:35.198727 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.198518 13987 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:35.200879 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 108881us
I20260709 13:06:35.192508 19534 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:35.164469 19328 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.215782 13945 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:35.218446 14022 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:35.210575 14023 raft_consensus.cc:2955] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 3 to 10, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:35.220412 19664 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:35.197917 19661 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:35.171583 19658 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:06:35.226531 19507 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:35.226229 19239 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.228293 19263 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:35.205436 19656 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:35.204932 19533 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:35.231876 19506 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.234594 19321 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:35.235354 19321 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:35.235632 19328 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:35.236055 19321 consensus_queue.cc:1243] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:06:35.218116 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:35.240300 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:35.242311 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:35.245489 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:35.217096 19493 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:35.264463 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:35.250448 19613 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:35.265776 19520 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:35.243985 19328 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:06:35.266880 19261 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:35.267808 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 641) took 1882 ms. Trace:
I20260709 13:06:35.268213 14021 rpcz_store.cc:276] 0709 13:06:33.385130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.799079 (+1413949us) service_pool.cc:224] Handling call
0709 13:06:34.799476 (+   397us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:34.812794 (+ 13318us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:06:34.843193 (+ 30399us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.088951 (+245758us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.089088 (+   137us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:35.094800 (+  5712us) log.cc:844] Serialized 52 byte log entry
0709 13:06:35.095033 (+   233us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:35.095348 (+   315us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.095733 (+   385us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.095901 (+   168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.095986 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:35.267551 (+171565us) raft_consensus.cc:1685] finished
0709 13:06:35.267575 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.267740 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29,"prepare.queue_time_us":31116,"raft.queue_time_us":26101,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":8849,"spinlock_wait_cycles":7705728,"thread_start_us":178286,"threads_started":1,"wal-append.queue_time_us":301876}
I20260709 13:06:35.242714 19665 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:35.270630 19665 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:35.271064 19665 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:35.275521 13948 raft_consensus.cc:1217] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Deduplicated request from leader. Original: 8.7->[9.8-13.10]   Dedup: 11.9->[13.10-13.10]
I20260709 13:06:35.276239 13948 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: [9.8-13.10]
I20260709 13:06:35.276759 19636 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.280308 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1043) took 1129 ms. Trace:
I20260709 13:06:35.280781 14551 rpcz_store.cc:276] 0709 13:06:34.150435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.152397 (+  1962us) service_pool.cc:224] Handling call
0709 13:06:34.152727 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.285943 (+133216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.285979 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.286019 (+    40us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:06:34.290433 (+  4414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.280041 (+989608us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54005622d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:35.280098 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.280244 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":305902976}
W20260709 13:06:35.253438 19657 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:06:35.254750 19602 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:35.189756 19659 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:06:35.255710 19417 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.260056 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:35.295959 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2660) took 1278 ms. Trace:
I20260709 13:06:35.296523 13946 rpcz_store.cc:276] 0709 13:06:34.004526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.093429 (+ 88903us) service_pool.cc:224] Handling call
0709 13:06:35.283152 (+1189723us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b540022a954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:35.283309 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1410381696}
W20260709 13:06:35.297375 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 51983.1us
I20260709 13:06:35.297775 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:35.298206 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:35.261752 19597 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.670s	user 0.008s	sys 0.000s
I20260709 13:06:35.263190 13950 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:35.304025 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:35.252321 19507 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:35.239447 19523 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:35.190528 19338 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.246986 19263 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:35.219787 19616 raft_consensus.cc:697] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:06:35.261343 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:35.334905 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.299763 13840 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:35.301770 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:35.304423 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:35.316355 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:35.305959 19384 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.331998 13950 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:06:35.315510 19652 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:35.341588 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 644) took 1955 ms. Trace:
I20260709 13:06:35.322736 19613 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:35.316704 19261 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:35.324612 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 645) took 1938 ms. Trace:
I20260709 13:06:35.340534 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:35.345608 14023 rpcz_store.cc:276] 0709 13:06:33.385987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.974151 (+1588164us) service_pool.cc:224] Handling call
0709 13:06:34.974743 (+   592us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:34.975149 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.975191 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:34.976119 (+   928us) log.cc:844] Serialized 501 byte log entry
0709 13:06:35.210403 (+234284us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:35.321791 (+111388us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.341243 (+ 19452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.341337 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:35.341379 (+    42us) raft_consensus.cc:1685] finished
0709 13:06:35.341391 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.341529 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11206,"raft.queue_time_us":22000,"thread_start_us":139981,"threads_started":1,"wal-append.queue_time_us":208739}
I20260709 13:06:35.344170 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:35.347187 14016 rpcz_store.cc:276] 0709 13:06:33.386246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.976538 (+1590292us) service_pool.cc:224] Handling call
0709 13:06:34.988203 (+ 11665us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:35.000958 (+ 12755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.001007 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:35.014862 (+ 13855us) log.cc:844] Serialized 52 byte log entry
0709 13:06:35.183691 (+168829us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:35.298960 (+115269us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.299357 (+   397us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.324130 (+ 24773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.324249 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:35.324312 (+    63us) raft_consensus.cc:1685] finished
0709 13:06:35.324331 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.324533 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35558,"raft.queue_time_us":44391,"raft.run_cpu_time_us":572,"raft.run_wall_time_us":44793,"thread_start_us":206711,"threads_started":2,"wal-append.queue_time_us":139235}
I20260709 13:06:35.351671 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3089) took 1918 ms. Trace:
I20260709 13:06:35.308885 19503 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:06:35.331382 19263 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:35.354022 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:35.346573 19431 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.357106 19431 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.359555 19431 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:35.352157 13872 rpcz_store.cc:276] 0709 13:06:33.433170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.444505 (+ 11335us) service_pool.cc:224] Handling call
0709 13:06:33.445010 (+   505us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:33.466663 (+ 21653us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.466711 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:33.480216 (+ 13505us) log.cc:844] Serialized 52 byte log entry
0709 13:06:34.631329 (+1151113us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b180023ea20. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:06:34.996915 (+365586us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:34.996956 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.997038 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:35.351368 (+354330us) raft_consensus.cc:1685] finished
0709 13:06:35.351396 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.351609 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25218,"raft.queue_time_us":21863,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":32983,"spinlock_wait_cycles":7040384,"thread_start_us":365178,"threads_started":1,"wal-append.queue_time_us":1595949}
I20260709 13:06:35.338732 19507 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:35.367728 13943 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: [14.8-15.9]
I20260709 13:06:35.371584 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:35.374439 14016 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:35.333988 19658 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:06:35.377468 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:35.382169 19431 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:35.390663 19431 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:35.391170 19431 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:35.383020 13943 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:35.384505 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:35.385171 14556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1048) took 1233 ms. Trace:
I20260709 13:06:35.389205 19417 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:35.389461 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2634) took 2664 ms. Trace:
I20260709 13:06:35.396865 14556 rpcz_store.cc:276] 0709 13:06:34.152130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.155221 (+  3091us) service_pool.cc:224] Handling call
0709 13:06:34.155571 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.352222 (+196651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.352255 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.352300 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:34.352357 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.384907 (+1032550us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54005609d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:35.384974 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.385113 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":451759488}
I20260709 13:06:35.364573 19263 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:35.397920 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 81949.6us
I20260709 13:06:35.397276 13943 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:06:35.400136 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:35.365685 19613 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:35.398380 13944 rpcz_store.cc:276] 0709 13:06:32.724643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.724792 (+   149us) service_pool.cc:224] Handling call
0709 13:06:32.755485 (+ 30693us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.783347 (+ 27862us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:33.252059 (+468712us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.347102 (+ 95043us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.347642 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.347789 (+   147us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:33.347822 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.347895 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:35.154475 (+1806580us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:35.389136 (+234661us) raft_consensus.cc:1685] finished
0709 13:06:35.389247 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.389394 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20658,"prepare.queue_time_us":646798,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":16866,"raft.run_cpu_time_us":428,"raft.run_wall_time_us":44936,"spinlock_wait_cycles":11075328,"thread_start_us":398956,"threads_started":2,"wal-append.queue_time_us":270907}
I20260709 13:06:35.348598 19384 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.418807 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:35.423913 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:35.422192 19616 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.427556 14016 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:35.393730 19507 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:35.431821 19636 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:35.438529 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.412117 19623 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:35.440214 19384 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:35.439951 19623 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:35.457737 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.467841 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:35.468986 19661 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:35.442262 19384 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:35.442655 19613 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.443487 19507 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:35.410274 19598 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.815s	user 0.010s	sys 0.001s
I20260709 13:06:35.444931 19646 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.422964 19493 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:35.492035 19493 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.410022 19547 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:35.494007 19671 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:06:35.494442 19547 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.495746 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:35.351619 19652 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.446846 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:35.430837 19644 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.621s	user 0.014s	sys 0.003s
I20260709 13:06:35.484213 14015 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:35.486477 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:35.513360 19547 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:35.512035 13942 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:35.490852 19239 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:35.496416 13868 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:35.517274 19658 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:35.517741 19658 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:35.505733 14021 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:35.458833 18996 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:35.531075 14022 raft_consensus.cc:2410] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:06:35.533881 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:35.534971 14021 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:35.537111 14016 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:35.446870 19467 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.541085 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:35.498957 19506 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:35.550423 19623 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:35.552246 19646 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:35.462697 19665 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.485317 19533 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.530997 19534 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:35.554868 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 214434us
I20260709 13:06:35.485385 19659 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:06:35.566612 13868 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:06:35.571759 13840 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:35.531088 19616 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:06:35.573678 14524 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:35.569518 13942 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:35.553661 19658 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.578930 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:35.580267 13838 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:35.579442 13944 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:35.530869 19239 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.583990 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:35.554103 19656 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.585991 14021 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:06:35.581888 19646 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:35.596143 19646 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:35.597105 19646 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.584774 19559 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:35.598438 19646 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:35.599079 19559 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:35.600093 19559 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.587918 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:35.572799 19534 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.581575 19459 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:35.581733 19507 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:35.584980 18744 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:35.590832 19652 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:35.591859 19431 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.583204 19661 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:35.594937 19263 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.604619 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:35.624576 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 221388us
I20260709 13:06:35.587385 19659 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:35.620486 13945 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:35.565845 19506 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.628209 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:35.630543 13942 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:06:35.634683 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.658545 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:35.618692 19616 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.617077 19507 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:35.632174 14021 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:06:35.631923 18744 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:35.665947 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 50847.3us
W20260709 13:06:35.665944 19462 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.002s
W20260709 13:06:35.667773 13915 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.532s (SENT)
I20260709 13:06:35.684894 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:35.688253 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:35.689036 14021 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
W20260709 13:06:35.691479 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.698258 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:35.699616 19666 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:06:35.699960 19682 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 13:06:35.700774 19678 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 13:06:35.702569 19677 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:06:35.702806 19533 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:35.685885 19462 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.707679 13948 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:06:35.705955 19462 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:35.671862 19616 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.712496 19462 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:35.713028 19462 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.714295 19616 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:35.715260 19431 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:35.718144 19507 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.719839 19507 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:35.723100 19665 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:35.731037 18744 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.677436 19459 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:35.743013 14021 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:35.743422 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:35.743978 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 45821.4us
W20260709 13:06:35.760581 19601 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 2.128s	user 0.010s	sys 0.004s
I20260709 13:06:35.763113 18874 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:35.686098 19613 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:06:35.793170 18874 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:35.767838 19658 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:35.768606 19567 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:35.770519 19665 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:35.795811 19665 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:35.796674 19567 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:35.796847 19665 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.771818 19239 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:35.772399 19661 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:35.800290 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 2989) took 2520 ms. Trace:
I20260709 13:06:35.800742 13947 rpcz_store.cc:276] 0709 13:06:33.280036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.360716 (+ 80680us) service_pool.cc:224] Handling call
0709 13:06:33.361046 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.481032 (+119986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.481068 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.481553 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.531126 (+ 49573us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:33.551320 (+ 20194us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.551602 (+   282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.910895 (+359293us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b18003a8040. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419
0709 13:06:33.911846 (+   951us) spinlock_profiling.cc:243] Waited 47.4 ms on lock 0x7b54000f5354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:33.911902 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:35.686579 (+1774677us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:35.800036 (+113457us) raft_consensus.cc:1685] finished
0709 13:06:35.800062 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.800214 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19611,"prepare.queue_time_us":19814,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":25254,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":14700,"spinlock_wait_cycles":236778496,"thread_start_us":49108,"threads_started":1,"wal-append.queue_time_us":124381}
I20260709 13:06:35.803117 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:35.773576 19506 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:35.805539 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:35.806602 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:35.806902 13872 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
W20260709 13:06:35.777344 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 79829.2us
I20260709 13:06:35.809602 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:35.810416 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:35.810444 13870 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:06:35.811957 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:35.812567 13875 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:35.813613 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:35.775712 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:35.815968 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:35.820910 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:35.821523 13866 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:06:35.821877 13870 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:35.823719 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:35.825551 13947 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:35.697029 19664 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:35.830782 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:35.831435 13867 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:35.833669 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:35.835238 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:35.816668 13872 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:35.837363 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:35.829550 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:35.829763 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:35.841392 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:35.841828 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:35.842723 14556 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 20: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:35.843158 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:35.843336 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:35.843818 14561 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:35.844281 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:35.844897 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:35.767864 19534 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:35.744401 19652 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:35.854650 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:35.848445 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:35.786744 19604 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.856037 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:35.856698 14560 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:35.852908 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:35.745386 19666 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:35.858335 14558 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:06:35.859193 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.825390 19417 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:35.858394 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 114584us
W20260709 13:06:35.869740 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 51012.2us
W20260709 13:06:35.871974 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.873759 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:35.839818 19665 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:35.965219 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:35.968080 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:35.969106 13869 raft_consensus.cc:2410] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:06:35.970657 19417 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:35.978909 19696 raft_consensus.cc:3055] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:35.979525 19696 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:35.982110 13875 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:35.848562 19462 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:35.856810 19636 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:36.008912 19636 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:36.009523 19636 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:36.009961 19692 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:36.011045 19680 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.220s
I20260709 13:06:36.012071 19569 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:36.012435 19569 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:35.852064 19489 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:36.013381 19569 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.013724 19489 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:36.014814 19686 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:06:36.015161 19569 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:36.017012 19489 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:35.857028 19533 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:35.878859 19467 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:36.013340 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:35.995277 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:36.005445 14024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2954) took 2617 ms (client timeout 1532 ms). Trace:
W20260709 13:06:36.022442 14024 rpcz_store.cc:269] 0709 13:06:33.387902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.397877 (+  9975us) service_pool.cc:224] Handling call
0709 13:06:36.005157 (+2607280us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540023f854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:36.005315 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3111918464}
I20260709 13:06:36.023947 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:36.007831 19493 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:36.008857 19462 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:36.027006 19462 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.029698 19462 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:36.026118 19506 consensus_queue.cc:1175] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" responder_term: 16 status { last_received { term: 16 index: 14 } last_committed_idx: 13 last_received_current_leader { term: 16 index: 14 } }
I20260709 13:06:36.025413 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:36.021862 13875 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:36.001791 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:35.980800 19455 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:36.042371 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:36.049033 13870 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:06:35.865808 19604 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:36.051229 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:36.051591 13839 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:36.052305 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:36.098428 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:36.101743 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:36.102546 14024 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:36.103343 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:36.104173 13949 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
W20260709 13:06:36.053917 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:36.110613 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:36.111416 19685 log.cc:927] Time spent T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:06:36.100841 19694 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:36.112052 19694 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:36.112618 19694 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:36.039966 19696 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.115118 19696 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:36.120682 19628 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:36.121588 19604 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:36.030639 19459 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.102258 19656 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:36.066184 19602 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:06:36.134158 19263 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:36.067309 19680 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:36.060488 19533 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:36.136485 13869 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:36.141434 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:36.141793 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.147284 13871 raft_consensus.cc:2410] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:06:36.135811 19659 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:36.066637 19455 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:36.133595 19636 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.160635 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:36.144356 19680 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:36.164690 19680 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.153996 19659 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:36.166734 19659 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:36.154493 18996 consensus_meta.cc:287] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.003s
W20260709 13:06:36.175904 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:36.177603 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.178926 13949 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:36.181624 14024 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:36.182871 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:36.183612 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:36.184079 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:36.184621 13869 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:36.185554 14024 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 17: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:36.188436 19462 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.189065 19462 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.190917 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:36.191696 19455 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:36.193571 13946 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:36.194779 19417 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:36.198326 19533 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:36.097599 19666 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:36.204114 19646 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.605s	user 0.009s	sys 0.003s
I20260709 13:06:36.207151 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2687) took 1194 ms. Trace:
I20260709 13:06:36.207844 13943 rpcz_store.cc:276] 0709 13:06:35.012739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.411548 (+398809us) service_pool.cc:224] Handling call
0709 13:06:35.412695 (+  1147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.537917 (+125222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.537950 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:35.537997 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:35.538038 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.206785 (+668747us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54001478d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:36.206860 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.207064 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287444224}
I20260709 13:06:36.209754 13875 raft_consensus.cc:2410] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:06:36.211937 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:36.212715 13875 raft_consensus.cc:2376] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20. Re-sending same reply.
I20260709 13:06:36.216950 19417 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:36.218925 18874 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:36.219796 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:36.220726 13948 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:36.222201 18874 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:36.224838 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:36.235287 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:36.235935 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:36.236523 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:36.220752 18996 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:36.226600 19562 consensus_queue.cc:1059] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:36.227118 13948 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:36.263670 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:36.264653 13948 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:36.227638 19636 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:36.229481 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:36.272570 13946 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:36.231549 19653 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:36.232059 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:36.233220 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:36.273339 19653 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:36.276813 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:36.279040 13943 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:36.280507 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:36.237185 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:36.242357 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:36.258808 13867 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 21.
I20260709 13:06:36.283397 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:36.283587 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:36.284521 13875 raft_consensus.cc:2410] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:06:36.285310 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:36.285583 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:36.286366 13875 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:36.287020 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:36.287431 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.288583 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:36.289402 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.289613 13867 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:36.268040 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:36.287529 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:36.290738 19668 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:36.291232 19668 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.291368 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:36.291687 19668 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:36.292943 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:36.291813 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.294510 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:36.260372 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:36.297189 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:36.297688 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:36.298525 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:36.299454 19569 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:36.299845 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:36.300539 14024 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:36.301501 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:36.301651 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:36.303762 19569 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:36.311090 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:36.312685 13942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.009s	sys 0.010s
I20260709 13:06:36.312516 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:36.313609 13942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.009s	sys 0.010s
W20260709 13:06:36.304834 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 88155.3us
I20260709 13:06:36.318967 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2395) took 3288 ms. Trace:
I20260709 13:06:36.320061 13867 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:06:36.320493 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 88500.8us
W20260709 13:06:36.325539 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.319541 14019 rpcz_store.cc:276] 0709 13:06:33.021315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.610936 (+589621us) service_pool.cc:224] Handling call
0709 13:06:33.621231 (+ 10295us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:33.802350 (+181119us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:34.009498 (+207148us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.452040 (+442542us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:34.462364 (+ 10324us) log.cc:844] Serialized 52 byte log entry
0709 13:06:34.462615 (+   251us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:34.462961 (+   346us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.473425 (+ 10464us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.483704 (+ 10279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.483833 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:35.257859 (+774026us) spinlock_profiling.cc:243] Waited 4.62 ms on lock 0x7b18002d0520. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:35.493556 (+235697us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b18002d0520. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:36.012722 (+519166us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b18002d0520. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1
0709 13:06:36.195232 (+182510us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b18002d0520. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:36.309313 (+114081us) raft_consensus.cc:1685] finished
0709 13:06:36.309347 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.309563 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117731,"prepare.queue_time_us":255074,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":153,"raft.queue_time_us":204750,"raft.run_cpu_time_us":423,"raft.run_wall_time_us":13277,"spinlock_wait_cycles":53343232,"thread_start_us":681446,"threads_started":3,"wal-append.queue_time_us":446578}
I20260709 13:06:36.337363 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:36.338297 14019 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:36.367637 14024 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:36.262722 18996 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:36.311152 19559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.007s	sys 0.016s
I20260709 13:06:36.393923 13872 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:36.307224 19665 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:36.435662 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 53481.7us
I20260709 13:06:36.305452 19489 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:36.357102 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:36.492398 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 135458us
I20260709 13:06:36.494670 14019 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
W20260709 13:06:36.482514 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 430993us
I20260709 13:06:36.498616 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:36.500533 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:36.384253 19569 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:36.513248 19569 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.513770 19569 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:36.433594 19559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.007s	sys 0.016s
I20260709 13:06:36.506860 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:36.508006 19658 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:36.508854 19431 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:36.511123 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:36.275244 19684 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.201s
I20260709 13:06:36.384253 19656 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:36.431201 18996 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:36.522147 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:36.526580 13949 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:36.539621 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:36.476779 19534 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:36.505592 19678 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:06:36.530320 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:36.507085 19613 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:36.549543 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:36.543931 13943 raft_consensus.cc:3055] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:36.553958 13943 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:36.487545 19646 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:36.498410 19604 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:36.539340 19645 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:06:36.559163 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 440785us
I20260709 13:06:36.497519 19665 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:36.555533 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:36.547238 19431 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:06:36.568524 13947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.003s	sys 0.015s
W20260709 13:06:36.562345 19507 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.838s	user 0.009s	sys 0.005s
I20260709 13:06:36.569430 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:36.430215 19653 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:36.559036 19674 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:36.554232 19636 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:36.551054 19656 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:36.552354 19613 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:36.571756 13943 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.565376 19645 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:36.573132 14019 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:36.550508 19705 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:36.592085 13947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.004s	sys 0.015s
I20260709 13:06:36.594992 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:36.596231 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.553229 19658 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:36.569177 19321 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:36.551267 19567 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:06:36.623922 13943 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:36.570988 19431 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:36.553525 19534 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.564256 19646 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.647174 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:36.656508 14556 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:36.567489 19604 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.616662 19613 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:36.667898 14024 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
W20260709 13:06:36.659672 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 64841.7us
I20260709 13:06:36.672421 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:36.626324 19645 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:36.679105 19709 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:06:36.682842 19700 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.683463 19700 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.625181 19636 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.688037 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2963) took 1448 ms. Trace:
I20260709 13:06:36.690446 14021 rpcz_store.cc:276] 0709 13:06:35.239461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.782409 (+542948us) service_pool.cc:224] Handling call
0709 13:06:35.782774 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:35.787314 (+  4540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.787347 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:35.806924 (+ 19577us) log.cc:844] Serialized 26 byte log entry
0709 13:06:36.279074 (+472150us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:36.279953 (+   879us) log.cc:844] Serialized 14 byte log entry
0709 13:06:36.281397 (+  1444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.281515 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.687814 (+406299us) raft_consensus.cc:1685] finished
0709 13:06:36.687834 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.687971 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4029,"prepare.queue_time_us":234,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":4732,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":34104,"spinlock_wait_cycles":68966144,"thread_start_us":412656,"threads_started":1,"wal-append.queue_time_us":709661}
I20260709 13:06:36.693928 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:36.689355 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:36.691347 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:36.696406 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.703691 14019 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:36.704743 14021 raft_consensus.cc:2410] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 13:06:36.707654 14015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2907) took 3853 ms (client timeout 2123 ms). Trace:
W20260709 13:06:36.708295 14015 rpcz_store.cc:269] 0709 13:06:32.853832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.853953 (+   121us) service_pool.cc:224] Handling call
0709 13:06:36.702812 (+3848859us) spinlock_profiling.cc:243] Waited 2.49 s on lock 0x7b54002981d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:36.707563 (+  4751us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5727003392}
W20260709 13:06:36.711498 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 142163us
W20260709 13:06:36.711858 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.712352 14524 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:36.712553 14015 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:36.713171 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2973) took 1460 ms. Trace:
I20260709 13:06:36.713326 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:36.716440 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:36.717607 13941 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
I20260709 13:06:36.718626 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:36.713551 14015 rpcz_store.cc:276] 0709 13:06:35.252543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.711852 (+1459309us) service_pool.cc:224] Handling call
0709 13:06:36.712212 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.712887 (+   675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.713097 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:36.686033 19696 consensus_meta.cc:287] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.000s
I20260709 13:06:36.667593 19462 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:36.673624 19674 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:36.730582 14558 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:36.729943 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:36.722183 14015 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:36.753481 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2974) took 1500 ms. Trace:
I20260709 13:06:36.725656 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:36.757339 14021 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:36.753983 14015 rpcz_store.cc:276] 0709 13:06:35.252833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.721516 (+1468683us) service_pool.cc:224] Handling call
0709 13:06:36.721897 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.753083 (+ 31186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.753377 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:36.760337 14015 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [11.6-15.8]
I20260709 13:06:36.761216 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2977) took 1129 ms. Trace:
I20260709 13:06:36.740021 13943 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:36.728543 14017 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:36.747154 19708 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:36.765100 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 651) took 1294 ms. Trace:
I20260709 13:06:36.688481 19636 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:36.719949 19459 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:36.731315 19567 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.643993 19705 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:36.790328 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.795382 14021 rpcz_store.cc:276] 0709 13:06:35.469835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.756412 (+1286577us) service_pool.cc:224] Handling call
0709 13:06:36.756863 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.757859 (+   996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.763935 (+  6076us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:36.798411 14021 pending_rounds.cc:85] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 9
I20260709 13:06:36.798893 14021 pending_rounds.cc:107] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:06:36.799268 14021 raft_consensus.cc:2889] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:36.761649 14015 rpcz_store.cc:276] 0709 13:06:35.632351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.759371 (+1127020us) service_pool.cc:224] Handling call
0709 13:06:36.759942 (+   571us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:36.760820 (+   878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.761141 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:36.801329 14015 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:06:36.812188 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2979) took 1137 ms. Trace:
I20260709 13:06:36.814486 14015 rpcz_store.cc:276] 0709 13:06:35.674567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.800544 (+1125977us) service_pool.cc:224] Handling call
0709 13:06:36.801007 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:36.805586 (+  4579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.811910 (+  6324us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:36.786815 19694 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.674s	user 0.015s	sys 0.000s
I20260709 13:06:36.784613 19696 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:06:36.786659 19533 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.018s	sys 0.031s
I20260709 13:06:36.788316 19462 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.826517 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:36.786418 19674 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:36.832901 14015 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
I20260709 13:06:36.836176 13839 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:36.803045 19459 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:36.815670 19700 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:36.782226 19431 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:36.750350 19658 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:36.841394 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2980) took 1166 ms. Trace:
I20260709 13:06:36.856329 14015 rpcz_store.cc:276] 0709 13:06:35.674892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.834479 (+1159587us) service_pool.cc:224] Handling call
0709 13:06:36.834805 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.840843 (+  6038us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.840895 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:36.840967 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:36.841027 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.841116 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.841312 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:36.859918 19706 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 13:06:36.862792 19712 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:06:36.784897 19656 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:36.864512 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:36.865118 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.864696 19459 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:36.866858 13874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 793) took 2627 ms (client timeout 2192 ms). Trace:
I20260709 13:06:36.867035 13875 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 19 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 13:06:36.867338 13874 rpcz_store.cc:269] 0709 13:06:34.239357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.242578 (+  3221us) service_pool.cc:224] Handling call
0709 13:06:36.866575 (+2623997us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54001aa254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:36.866765 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1122252672}
I20260709 13:06:36.867820 19674 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:36.868698 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3091) took 1036 ms. Trace:
I20260709 13:06:36.868842 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:36.869078 13875 rpcz_store.cc:276] 0709 13:06:35.832551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.295040 (+462489us) service_pool.cc:224] Handling call
0709 13:06:36.866739 (+571699us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:36.867531 (+   792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.868485 (+   954us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b54001aa250. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:36.868638 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1313664256}
I20260709 13:06:36.871177 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3094) took 1037 ms. Trace:
I20260709 13:06:36.871097 13914 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:36.871695 13875 rpcz_store.cc:276] 0709 13:06:35.833548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.870300 (+1036752us) service_pool.cc:224] Handling call
0709 13:06:36.870644 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.870830 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.870855 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:36.870900 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:36.870945 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.871008 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.871122 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:36.872182 19457 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:36.873081 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2698) took 1400 ms. Trace:
I20260709 13:06:36.877357 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:06:36.878774 19717 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:06:36.878858 19570 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:36.880625 19570 raft_consensus.cc:3055] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:06:36.880815 19716 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:06:36.881048 14558 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:36.881421 13914 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:36.883282 13914 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:36.873457 13942 rpcz_store.cc:276] 0709 13:06:35.472673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.659984 (+187311us) service_pool.cc:224] Handling call
0709 13:06:35.660323 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:35.661763 (+  1440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.661796 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:35.662347 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:06:36.314205 (+651858us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:36.314257 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.314341 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.872878 (+558537us) raft_consensus.cc:1685] finished
0709 13:06:36.872901 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.873023 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4815,"raft.queue_time_us":3669,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":2784,"thread_start_us":651478,"threads_started":1,"wal-append.queue_time_us":960792}
W20260709 13:06:36.871373 19659 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.702s	user 0.011s	sys 0.000s
W20260709 13:06:36.885865 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.885596 19659 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:36.886891 19701 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:06:36.887773 19703 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:36.888506 19707 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:06:36.889019 13869 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:36.890673 19653 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:36.802568 19567 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.888342 19659 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:36.827718 19694 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.929116 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:36.853226 19722 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.838613 19462 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.856559 19708 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:36.881044 19570 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:36.853762 19665 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:36.965430 19462 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:36.965646 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2972) took 1715 ms. Trace:
I20260709 13:06:36.966131 14023 rpcz_store.cc:276] 0709 13:06:35.250388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.228012 (+977624us) service_pool.cc:224] Handling call
0709 13:06:36.228381 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:36.228869 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:36.229210 (+   341us) log.cc:844] Serialized 14 byte log entry
0709 13:06:36.623877 (+394667us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:36.627898 (+  4021us) log.cc:844] Serialized 26 byte log entry
0709 13:06:36.628066 (+   168us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:36.628105 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.628186 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.965403 (+337217us) raft_consensus.cc:1685] finished
0709 13:06:36.965432 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.965575 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9310,"raft.queue_time_us":33729,"raft.run_cpu_time_us":422,"raft.run_wall_time_us":62112,"spinlock_wait_cycles":23032576,"thread_start_us":390018,"threads_started":1,"wal-append.queue_time_us":680366}
I20260709 13:06:36.968432 14023 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:36.969041 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2982) took 1293 ms. Trace:
I20260709 13:06:36.969329 14023 rpcz_store.cc:276] 0709 13:06:35.675511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.967889 (+1292378us) service_pool.cc:224] Handling call
0709 13:06:36.968225 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.968802 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.968987 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:36.914513 19680 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:36.953732 19562 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:36.970450 19680 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:36.970860 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 653) took 1294 ms. Trace:
I20260709 13:06:36.958875 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:36.959173 19534 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:36.912626 19703 consensus_queue.cc:1059] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:06:36.962693 19570 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.970808 19562 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:36.918815 19431 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:36.958762 19719 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:06:36.975862 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2970) took 1729 ms. Trace:
I20260709 13:06:36.872445 19645 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:36.972231 14023 rpcz_store.cc:276] 0709 13:06:35.676175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.970076 (+1293901us) service_pool.cc:224] Handling call
0709 13:06:36.970389 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.970565 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.970590 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:36.970627 (+    37us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:36.970656 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.970708 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.970801 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:36.995730 14023 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:36.996520 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2984) took 1151 ms. Trace:
I20260709 13:06:36.996915 14023 rpcz_store.cc:276] 0709 13:06:35.845497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.995067 (+1149570us) service_pool.cc:224] Handling call
0709 13:06:36.995483 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.996218 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.996447 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:36.921202 18744 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:36.982905 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 982) took 1111 ms. Trace:
I20260709 13:06:36.950646 19692 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:36.988407 13943 raft_consensus.cc:3055] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:37.020726 13943 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:37.021644 13943 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.022636 13943 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:36.912706 19506 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
W20260709 13:06:36.991716 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.991055 14016 rpcz_store.cc:276] 0709 13:06:35.246998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.215702 (+968704us) service_pool.cc:224] Handling call
0709 13:06:36.216095 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:36.216516 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.216549 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:36.236608 (+ 20059us) log.cc:844] Serialized 26 byte log entry
0709 13:06:36.658939 (+422331us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:36.714763 (+ 55824us) log.cc:844] Serialized 14 byte log entry
0709 13:06:36.715143 (+   380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.715256 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.975569 (+260313us) raft_consensus.cc:1685] finished
0709 13:06:36.975600 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.975780 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48074,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":14782,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":15657,"thread_start_us":182780,"threads_started":1,"wal-append.queue_time_us":505176}
W20260709 13:06:36.981968 19668 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.690s	user 0.007s	sys 0.007s
I20260709 13:06:36.978030 19533 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:36.989660 19646 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:37.031163 14560 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:37.020892 13947 rpcz_store.cc:276] 0709 13:06:35.871685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.871870 (+   185us) service_pool.cc:224] Handling call
0709 13:06:35.872259 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:35.872672 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.872700 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:35.968420 (+ 95720us) log.cc:844] Serialized 26 byte log entry
0709 13:06:36.644374 (+675954us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:36.973848 (+329474us) log.cc:844] Serialized 14 byte log entry
0709 13:06:36.976828 (+  2980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.982570 (+  5742us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.982639 (+    69us) raft_consensus.cc:1685] finished
0709 13:06:36.982657 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.982821 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4361,"prepare.queue_time_us":309955,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":4164,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":8050,"thread_start_us":916081,"threads_started":2,"wal-append.queue_time_us":672952}
W20260709 13:06:37.027396 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 191306us
I20260709 13:06:37.032567 13914 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:36.988016 19659 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:37.034210 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:37.035120 14560 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:37.030678 14016 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 13:06:37.044495 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 64323.7us
I20260709 13:06:36.972162 19708 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.024034 19570 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:36.957540 19722 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:37.040468 19703 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:37.070398 13943 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:06:37.071086 13943 pending_rounds.cc:85] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 7
I20260709 13:06:37.071395 13943 pending_rounds.cc:107] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:06:37.071712 13943 raft_consensus.cc:2889] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:37.072997 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:37.073700 13943 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:06:37.047382 19507 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:37.031417 19668 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.079655 19703 raft_consensus.cc:697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:37.046200 19659 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:37.164058 14554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1044) took 2939 ms. Trace:
I20260709 13:06:37.164752 14024 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:37.164608 14554 rpcz_store.cc:276] 0709 13:06:34.150783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.152907 (+  2124us) service_pool.cc:224] Handling call
0709 13:06:34.202216 (+ 49309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.350982 (+148766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.351031 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.351092 (+    61us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:34.351142 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.090099 (+2738957us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b540040a6d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:37.090155 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.090307 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341454208}
I20260709 13:06:37.028967 19692 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:37.196369 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.197001 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.199412 19720 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:06:37.206282 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:37.081080 19507 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:37.223524 18996 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.672s	user 0.007s	sys 0.006s
I20260709 13:06:37.210011 19659 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:37.238381 18996 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.212833 19694 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.240860 18996 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.240935 19703 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:37.241671 13945 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
W20260709 13:06:37.223408 19613 consensus_meta.cc:287] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.000s
I20260709 13:06:37.208777 19562 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.203615 19567 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.208819 19708 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.220750 19668 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:37.281635 19567 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:37.282362 19567 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:37.298296 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.317615 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:37.318828 14558 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:37.332052 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:37.333101 13949 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
W20260709 13:06:37.339057 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.343806 13948 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:37.344238 13948 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:37.345091 13948 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:37.341656 14522 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:37.346953 13948 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:37.289209 19606 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:37.360782 19539 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:37.361263 19539 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.360994 19606 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:37.361464 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.361642 19539 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:37.348090 13943 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:06:37.350867 19346 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.362218 13943 pending_rounds.cc:85] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 12
I20260709 13:06:37.362521 13943 pending_rounds.cc:107] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:06:37.362437 19346 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.362910 13943 raft_consensus.cc:2889] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:37.365034 19346 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:37.365536 19507 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.351325 19628 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:06:37.368760 19741 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:06:37.369575 19569 consensus_meta.cc:287] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.849s	user 0.011s	sys 0.000s
I20260709 13:06:37.373163 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2433) took 1143 ms. Trace:
I20260709 13:06:37.362960 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:37.373889 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3044) took 1550 ms. Trace:
I20260709 13:06:37.375732 13868 rpcz_store.cc:276] 0709 13:06:35.823331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.827319 (+  3988us) service_pool.cc:224] Handling call
0709 13:06:35.827678 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:35.972955 (+145277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.972995 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:35.973577 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:06:36.393466 (+419889us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:36.393534 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.373438 (+979904us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54003abbd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:37.373524 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:37.373586 (+    62us) raft_consensus.cc:1685] finished
0709 13:06:37.373605 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.373829 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":272,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":132,"spinlock_wait_cycles":333071104,"thread_start_us":419140,"threads_started":1,"wal-append.queue_time_us":656502}
I20260709 13:06:37.373656 14016 rpcz_store.cc:276] 0709 13:06:36.233847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.038449 (+804602us) service_pool.cc:224] Handling call
0709 13:06:37.042436 (+  3987us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:37.043097 (+   661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.043142 (+    45us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:37.049825 (+  6683us) log.cc:844] Serialized 78 byte log entry
0709 13:06:37.171164 (+121339us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:37.197820 (+ 26656us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.201924 (+  4104us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.207355 (+  5431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.212333 (+  4978us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:37.372890 (+160557us) raft_consensus.cc:1685] finished
0709 13:06:37.372912 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.373094 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26,"prepare.queue_time_us":26476,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":66,"raft.queue_time_us":2036,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":5637,"spinlock_wait_cycles":6384640,"thread_start_us":94229,"threads_started":1,"wal-append.queue_time_us":179132}
W20260709 13:06:37.350564 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.351938 14522 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:37.381702 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.352880 19705 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.353691 13942 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:06:37.359050 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.360085 19659 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:37.268826 19613 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:37.402664 13870 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:37.402387 19606 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:37.406055 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 650) took 1936 ms. Trace:
I20260709 13:06:37.406834 19567 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.406469 14019 rpcz_store.cc:276] 0709 13:06:35.469522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.738119 (+1268597us) service_pool.cc:224] Handling call
0709 13:06:36.738624 (+   505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:36.740635 (+  2011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.740684 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:36.741318 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 13:06:36.980918 (+239600us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:37.001355 (+ 20437us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.020001 (+ 18646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.020110 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:37.324391 (+304281us) spinlock_profiling.cc:243] Waited 4.96 ms on lock 0x7b18003d3ba0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:37.405711 (+ 81320us) raft_consensus.cc:1685] finished
0709 13:06:37.405842 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.405980 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22309,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":2671,"raft.queue_time_us":1652,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":22720,"spinlock_wait_cycles":59465472,"thread_start_us":170354,"threads_started":1,"wal-append.queue_time_us":254233}
I20260709 13:06:37.408973 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:37.410682 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:37.411419 13948 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:06:37.393743 13872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3097) took 1551 ms (client timeout 1980 ms). Trace:
W20260709 13:06:37.419866 13872 rpcz_store.cc:269] 0709 13:06:35.842384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.843096 (+   712us) service_pool.cc:224] Handling call
0709 13:06:37.393544 (+1550448us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54000a8054. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:37.393677 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":984393088}
I20260709 13:06:37.387481 19263 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.395641 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:37.422214 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:37.422312 13867 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:37.423398 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:37.423924 19263 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.398101 19507 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.425096 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3046) took 1576 ms. Trace:
I20260709 13:06:37.426507 13867 rpcz_store.cc:276] 0709 13:06:35.849187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.424320 (+1575133us) service_pool.cc:224] Handling call
0709 13:06:37.424598 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.424768 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.424807 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:37.424855 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:37.424899 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.424949 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.425041 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:37.428135 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.384333 19569 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:37.403729 19613 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:37.411521 14019 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:37.431743 19523 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:37.432188 19523 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:37.433127 19523 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:37.365911 13943 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:37.398342 19694 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:37.435063 19694 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.414408 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:37.435462 19694 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:37.394534 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:37.413149 19659 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.438257 19606 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.425486 19507 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.438887 19606 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.440397 19507 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.440757 19539 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:37.391920 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:37.438412 19659 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.442883 13872 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:37.444677 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3048) took 1577 ms. Trace:
I20260709 13:06:37.303299 19645 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:37.468078 13867 rpcz_store.cc:276] 0709 13:06:35.867726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.434540 (+1566814us) service_pool.cc:224] Handling call
0709 13:06:37.444011 (+  9471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.444248 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.444285 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:37.444346 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:37.444393 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.444469 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.444599 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:37.477519 13867 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:37.478211 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3102) took 1509 ms. Trace:
I20260709 13:06:37.478521 13867 rpcz_store.cc:276] 0709 13:06:35.969693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.477005 (+1507312us) service_pool.cc:224] Handling call
0709 13:06:37.477340 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.477929 (+   589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.478151 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:37.444012 19695 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.571s	user 0.000s	sys 0.019s
I20260709 13:06:37.488747 14558 raft_consensus.cc:2368] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17.
I20260709 13:06:37.469547 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3098) took 1622 ms. Trace:
I20260709 13:06:37.463122 19462 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:06:37.470660 19523 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:37.474308 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:37.490403 13872 rpcz_store.cc:276] 0709 13:06:35.847270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.442437 (+1595167us) service_pool.cc:224] Handling call
0709 13:06:37.442711 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.469248 (+ 26537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.469483 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:37.441167 19606 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:37.472066 19569 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:37.461251 19696 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
W20260709 13:06:37.519248 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.520025 13872 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:37.520819 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3104) took 1548 ms. Trace:
I20260709 13:06:37.520903 14019 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:37.521229 13872 rpcz_store.cc:276] 0709 13:06:35.972245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.501383 (+1529138us) service_pool.cc:224] Handling call
0709 13:06:37.501732 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.520442 (+ 18710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.520700 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:37.524431 14015 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.027s	sys 0.068s
W20260709 13:06:37.524816 14015 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.027s	sys 0.068s
I20260709 13:06:37.487711 19239 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:37.528272 19263 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.527984 19239 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.523427 13868 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:37.525521 13872 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:37.529633 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3105) took 1557 ms. Trace:
I20260709 13:06:37.529999 13872 rpcz_store.cc:276] 0709 13:06:35.972805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.524774 (+1551969us) service_pool.cc:224] Handling call
0709 13:06:37.525132 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.529366 (+  4234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.529574 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:37.532596 13872 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:37.533195 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3109) took 1557 ms. Trace:
I20260709 13:06:37.533543 13872 rpcz_store.cc:276] 0709 13:06:35.975540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.532017 (+1556477us) service_pool.cc:224] Handling call
0709 13:06:37.532411 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.532960 (+   549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.533136 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:37.535606 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.535492 13872 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:37.536893 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:37.539525 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.523648 13874 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:37.540355 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3110) took 1564 ms. Trace:
I20260709 13:06:37.527293 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:37.541026 13872 rpcz_store.cc:276] 0709 13:06:35.976390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.534605 (+1558215us) service_pool.cc:224] Handling call
0709 13:06:37.535009 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.536844 (+  1835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.540282 (+  3438us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:37.542819 14019 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:37.541659 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:37.543177 13872 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:37.507360 19604 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:37.551680 19595 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:37.544404 19570 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:37.552918 19570 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:37.551028 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3111) took 1574 ms. Trace:
I20260709 13:06:37.553558 13872 rpcz_store.cc:276] 0709 13:06:35.977181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.542648 (+1565467us) service_pool.cc:224] Handling call
0709 13:06:37.542994 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.550721 (+  7727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.550963 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:37.556353 19653 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:37.553388 19570 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:37.556993 19653 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:37.563625 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3092) took 1730 ms. Trace:
I20260709 13:06:37.564332 13869 rpcz_store.cc:276] 0709 13:06:35.832888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.499110 (+666222us) service_pool.cc:224] Handling call
0709 13:06:36.499656 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.889489 (+389833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.563215 (+673726us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b540026bbd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:37.563529 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894825216}
I20260709 13:06:37.566013 13869 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:37.628273 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 804) took 1595 ms. Trace:
I20260709 13:06:37.629498 13869 rpcz_store.cc:276] 0709 13:06:36.032998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.565496 (+1532498us) service_pool.cc:224] Handling call
0709 13:06:37.565833 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.605109 (+ 39276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.628018 (+ 22909us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:37.659667 13868 raft_consensus.cc:2376] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 15. Re-sending same reply.
I20260709 13:06:37.544142 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:37.661257 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 110134us
I20260709 13:06:37.662561 14019 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:37.675685 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.676069 13868 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 16 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:37.702222 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.712309 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.741016 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.740993 14019 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:37.743957 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:37.744731 14019 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:37.745716 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:37.753330 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:37.754772 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:37.757851 19737 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.559s	user 0.000s	sys 0.001s
W20260709 13:06:37.759857 19744 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 13:06:37.761216 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2981) took 2086 ms. Trace:
I20260709 13:06:37.761638 14015 rpcz_store.cc:276] 0709 13:06:35.675204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.858507 (+1183303us) service_pool.cc:224] Handling call
0709 13:06:36.858994 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:36.910931 (+ 51937us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:36.910987 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:36.979261 (+ 68274us) log.cc:844] Serialized 26 byte log entry
0709 13:06:37.525556 (+546295us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:37.525611 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.525680 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:37.760972 (+235292us) raft_consensus.cc:1685] finished
0709 13:06:37.760997 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.761151 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1961,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":13986,"thread_start_us":507366,"threads_started":1,"wal-append.queue_time_us":542491}
I20260709 13:06:37.763521 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:37.764145 14015 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:37.757870 19341 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:37.765622 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:37.767333 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:37.768414 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:37.769032 14015 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:37.769517 19722 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.769814 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:37.760708 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:37.772799 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:37.773629 14019 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:37.662302 19676 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:37.658128 19239 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:37.784099 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 120470us
I20260709 13:06:37.666779 19569 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.825907 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:37.826628 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3045) took 1987 ms. Trace:
I20260709 13:06:37.829454 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:37.800946 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.817847 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 220021us
I20260709 13:06:37.733980 19604 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:37.833228 14019 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: [12.8-14.9]
I20260709 13:06:37.834826 19656 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:06:37.828922 13875 rpcz_store.cc:276] 0709 13:06:35.839080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.872838 (+1033758us) service_pool.cc:224] Handling call
0709 13:06:36.873189 (+   351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:36.873560 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.873590 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:36.874284 (+   694us) log.cc:844] Serialized 52 byte log entry
0709 13:06:37.001889 (+127605us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:37.001931 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.002003 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:37.757046 (+755043us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:37.826183 (+ 69137us) raft_consensus.cc:1685] finished
0709 13:06:37.826217 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.826539 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3244,"raft.queue_time_us":68743,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":4985,"spinlock_wait_cycles":7205760,"thread_start_us":127027,"threads_started":1,"wal-append.queue_time_us":323541}
I20260709 13:06:37.834671 19457 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:06:37.833788 14015 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:37.852974 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.817436 19748 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:06:37.855566 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2440) took 1619 ms. Trace:
I20260709 13:06:37.838948 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2441) took 1602 ms. Trace:
I20260709 13:06:37.856813 14015 rpcz_store.cc:276] 0709 13:06:36.236013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.780177 (+1544164us) service_pool.cc:224] Handling call
0709 13:06:37.828169 (+ 47992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.855206 (+ 27037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.855490 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:37.856990 14019 rpcz_store.cc:276] 0709 13:06:36.236584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.832383 (+1595799us) service_pool.cc:224] Handling call
0709 13:06:37.832926 (+   543us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:37.836664 (+  3738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.838866 (+  2202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:37.864188 14019 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:37.865123 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2445) took 1604 ms. Trace:
W20260709 13:06:37.831565 19703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.036s	sys 0.029s
I20260709 13:06:37.784893 19539 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:37.868673 19703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.037s	sys 0.030s
I20260709 13:06:37.831816 19676 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.872238 19676 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.857578 19457 consensus_queue.cc:1243] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:37.867545 14023 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:37.848567 19604 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.836644 19569 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:37.865545 14019 rpcz_store.cc:276] 0709 13:06:36.260264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.863501 (+1603237us) service_pool.cc:224] Handling call
0709 13:06:37.863898 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.864785 (+   887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.865044 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:37.784754 19743 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:06:37.809180 19694 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:37.785128 19653 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:37.890156 19694 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:37.890271 19569 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:37.890642 19570 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:37.955082 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.956471 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 126906us
W20260709 13:06:37.959290 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.961716 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.963881 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.966370 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2436) took 1731 ms. Trace:
I20260709 13:06:37.968271 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2407) took 4835 ms. Trace:
I20260709 13:06:37.966970 14016 rpcz_store.cc:276] 0709 13:06:36.234741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.436572 (+1201831us) service_pool.cc:224] Handling call
0709 13:06:37.437041 (+   469us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:37.437601 (+   560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.437643 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:37.443326 (+  5683us) log.cc:844] Serialized 78 byte log entry
0709 13:06:37.771385 (+328059us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:37.818988 (+ 47603us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.823211 (+  4223us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.823727 (+   516us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.823917 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.824025 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:37.965475 (+141450us) raft_consensus.cc:1685] finished
0709 13:06:37.965534 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.966007 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":98124,"prepare.run_cpu_time_us":174,"prepare.run_wall_time_us":176,"raft.queue_time_us":7862,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":6045,"spinlock_wait_cycles":44396160,"thread_start_us":327628,"threads_started":1,"wal-append.queue_time_us":449543}
I20260709 13:06:37.968730 14020 rpcz_store.cc:276] 0709 13:06:33.132396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.635631 (+1503235us) service_pool.cc:224] Handling call
0709 13:06:34.636178 (+   547us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:34.636761 (+   583us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.636804 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:34.651197 (+ 14393us) log.cc:844] Serialized 52 byte log entry
0709 13:06:37.782965 (+3131768us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b1800245d40. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:06:37.958438 (+175473us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:37.966534 (+  8096us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.967227 (+   693us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.967460 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.967622 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:37.967807 (+   185us) raft_consensus.cc:1685] finished
0709 13:06:37.967821 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.968018 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7294,"raft.queue_time_us":1023,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":1897,"spinlock_wait_cycles":36415872,"thread_start_us":174742,"threads_started":1,"wal-append.queue_time_us":3244593}
I20260709 13:06:37.972131 14020 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:37.973053 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2988) took 1384 ms. Trace:
I20260709 13:06:37.974517 14016 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:37.983258 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2450) took 1721 ms. Trace:
I20260709 13:06:37.983731 14016 rpcz_store.cc:276] 0709 13:06:36.261662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.970023 (+1708361us) service_pool.cc:224] Handling call
0709 13:06:37.973920 (+  3897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.982931 (+  9011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.983169 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:37.985150 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:37.981540 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:37.989398 14016 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 13:06:37.999221 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.003080 13948 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:38.003250 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:38.005824 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2989) took 1109 ms. Trace:
I20260709 13:06:38.006395 14016 rpcz_store.cc:276] 0709 13:06:36.895985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.988192 (+1092207us) service_pool.cc:224] Handling call
0709 13:06:37.988827 (+   635us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:38.005414 (+ 16587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.005724 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:38.008175 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:38.008682 19431 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:38.012326 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3093) took 2179 ms. Trace:
I20260709 13:06:38.014863 19457 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:38.015070 19521 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:38.015822 13871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3039) took 2205 ms (client timeout 1932 ms). Trace:
I20260709 13:06:38.016929 13869 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
W20260709 13:06:38.018893 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.020828 19604 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:38.020491 19703 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:38.021481 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2715) took 1096 ms. Trace:
I20260709 13:06:38.022373 19523 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.973460 14020 rpcz_store.cc:276] 0709 13:06:36.589003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.971112 (+1382109us) service_pool.cc:224] Handling call
0709 13:06:37.971644 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.972758 (+  1114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.972967 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:38.018884 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:38.044200 13869 pending_rounds.cc:85] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 8
I20260709 13:06:38.026847 13870 rpcz_store.cc:276] 0709 13:06:35.833199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.748628 (+915429us) service_pool.cc:224] Handling call
0709 13:06:36.748991 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.403218 (+654227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.011571 (+608353us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b54003ac0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:38.012036 (+   465us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1502678400}
W20260709 13:06:38.043732 13871 rpcz_store.cc:269] 0709 13:06:35.809973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.813365 (+  3392us) service_pool.cc:224] Handling call
0709 13:06:38.015235 (+2201870us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54002e59d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:38.015546 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765259520}
I20260709 13:06:38.058622 13869 pending_rounds.cc:107] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:06:38.022413 19656 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:38.066789 19656 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.015872 19493 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:38.070631 13873 raft_consensus.cc:2376] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 16. Re-sending same reply.
W20260709 13:06:38.071702 13873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3100) took 2205 ms (client timeout 1744 ms). Trace:
I20260709 13:06:38.071393 19493 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:38.072180 13873 rpcz_store.cc:269] 0709 13:06:35.866582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.869135 (+  2553us) service_pool.cc:224] Handling call
0709 13:06:38.018423 (+2149288us) spinlock_profiling.cc:243] Waited 6.96 ms on lock 0x7b5800003698. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:06:38.071593 (+ 53170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15998848}
I20260709 13:06:38.073940 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.077529 13873 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:38.070041 19656 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:38.078447 19523 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.047930 13945 rpcz_store.cc:276] 0709 13:06:36.925128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.925421 (+   293us) service_pool.cc:224] Handling call
0709 13:06:36.925859 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.242229 (+316370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.020779 (+778550us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54001478d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:38.021225 (+   446us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725795072}
I20260709 13:06:38.064296 13869 raft_consensus.cc:2889] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:38.043346 19521 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:38.085817 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.086727 19570 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:38.087672 19521 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:38.089385 13871 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:38.089871 13871 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:38.091423 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 83700.2us
I20260709 13:06:38.091626 13871 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.093354 13871 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:38.097437 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:38.103292 14016 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:38.098577 19694 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:38.099510 19457 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:38.111279 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:38.100919 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:38.112597 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.114137 19570 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:38.115247 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.115958 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:38.117890 14016 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:06:38.118307 14016 pending_rounds.cc:85] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 8
I20260709 13:06:38.118587 14016 pending_rounds.cc:107] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:06:38.118824 14016 raft_consensus.cc:2889] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:38.119095 14020 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:38.119762 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 660) took 1146 ms. Trace:
I20260709 13:06:38.120162 14020 rpcz_store.cc:276] 0709 13:06:36.973242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.062302 (+1089060us) service_pool.cc:224] Handling call
0709 13:06:38.118788 (+ 56486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:38.119516 (+   728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.119694 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:38.121258 19570 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:38.121639 14020 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:38.109380 19457 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.124277 19703 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:38.125439 19321 consensus_queue.cc:1059] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:38.129586 14556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1055) took 1945 ms. Trace:
I20260709 13:06:38.122862 19570 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.130856 19656 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:38.142239 14556 rpcz_store.cc:276] 0709 13:06:36.184061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.185817 (+  1756us) service_pool.cc:224] Handling call
0709 13:06:36.186303 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.673844 (+487541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.128955 (+1455111us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b5400530f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:38.129327 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080849664}
I20260709 13:06:38.148342 19261 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:38.148875 19261 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:38.151844 19431 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:34.264508 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:38.160209 19569 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:38.176960 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:38.201329 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 77749.2us
I20260709 13:06:38.185365 19523 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:38.204365 14016 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:38.217489 19261 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.219801 19261 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:38.220957 19503 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:38.166112 19521 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:06:38.215670 13945 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:38.241904 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:38.274308 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 68741us
W20260709 13:06:38.279759 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.282438 19341 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:38.283440 19341 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.286550 19694 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:38.288208 19761 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:38.288769 19761 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:38.291354 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.291640 14558 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:38.292934 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.295002 14554 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:38.281769 19697 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:38.294643 19761 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.296398 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:38.295797 19756 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:06:38.310909 14553 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:38.312273 19761 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:38.296900 14556 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:38.318488 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3118) took 1398 ms. Trace:
I20260709 13:06:38.319089 13875 rpcz_store.cc:276] 0709 13:06:36.834579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.949698 (+1115119us) service_pool.cc:224] Handling call
0709 13:06:37.950603 (+   905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.951227 (+   624us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:38.065746 (+114519us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.212703 (+146957us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.217045 (+  4342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:38.217129 (+    84us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:38.217193 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.224037 (+  6844us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.224239 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":134800,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":89,"raft.queue_time_us":18511,"raft.run_wall_time_us":39302,"spinlock_wait_cycles":44018176,"thread_start_us":181552,"threads_started":2,"wal-append.queue_time_us":147552}
I20260709 13:06:38.323405 19703 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:38.324313 19703 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.324671 19680 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:38.325259 19503 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:38.325735 14016 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:38.334313 13875 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:06:38.307816 14020 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:38.348729 19672 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:06:38.349835 14016 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:38.349610 19430 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:06:38.350620 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.350839 19430 raft_consensus.cc:3055] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:38.351312 19430 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:38.352821 19430 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.299515 19341 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:38.326733 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2443) took 2066 ms. Trace:
I20260709 13:06:38.354525 19430 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:38.348842 14020 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:38.365757 19764 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:38.366235 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3117) took 1532 ms. Trace:
I20260709 13:06:38.366353 19764 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:38.367734 19764 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.366772 13868 rpcz_store.cc:276] 0709 13:06:36.834153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.677515 (+843362us) service_pool.cc:224] Handling call
0709 13:06:37.677794 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.678003 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:37.949505 (+271502us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.270396 (+320891us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.345722 (+ 75326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:38.363744 (+ 18022us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:38.365681 (+  1937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.365812 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.365976 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115365,"prepare.queue_time_us":294605,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":118,"raft.queue_time_us":57950,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":15737,"thread_start_us":343153,"threads_started":3,"wal-append.queue_time_us":236148}
I20260709 13:06:38.369304 19764 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:38.370818 19561 consensus_queue.cc:1059] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:38.370990 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.371984 19561 consensus_queue.cc:1243] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Peer 4dd265d61fc141a48ca9d3d5c8f52921 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:06:38.373319 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.374272 13871 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:38.304353 19521 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:06:38.380858 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 265006us
I20260709 13:06:38.366407 19703 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:38.388700 19715 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.562s	user 0.003s	sys 0.000s
I20260709 13:06:38.405104 19728 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:38.408459 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.396725 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:38.398046 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 138599us
I20260709 13:06:38.390630 19521 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:38.418725 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.354847 14015 rpcz_store.cc:276] 0709 13:06:36.259736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.863322 (+1603586us) service_pool.cc:224] Handling call
0709 13:06:37.863615 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.863856 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:37.959488 (+ 95632us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.200161 (+240673us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.203405 (+  3244us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:38.207554 (+  4149us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:38.245378 (+ 37824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.259669 (+ 14291us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.326491 (+ 66822us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12710,"prepare.queue_time_us":154334,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":125,"raft.queue_time_us":1031,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":4030,"thread_start_us":100439,"threads_started":1,"wal-append.queue_time_us":144283}
W20260709 13:06:38.430794 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 79066.5us
I20260709 13:06:38.430950 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.432610 13947 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:38.297955 19697 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.421495 19703 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:06:38.435967 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.409119 19728 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.434367 13914 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:38.441468 13840 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:38.442380 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:38.443372 14022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 2965) took 3203 ms (client timeout 1888 ms). Trace:
W20260709 13:06:38.443878 14022 rpcz_store.cc:269] 0709 13:06:35.239993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.397288 (+157295us) service_pool.cc:224] Handling call
0709 13:06:38.442881 (+3045593us) spinlock_profiling.cc:243] Waited 99.3 ms on lock 0x7b5400239e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:38.443155 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228402560}
I20260709 13:06:38.455227 19703 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.456432 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3062) took 2229 ms. Trace:
I20260709 13:06:38.458153 13869 rpcz_store.cc:276] 0709 13:06:36.226608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.657617 (+1431009us) service_pool.cc:224] Handling call
0709 13:06:38.015917 (+358300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:38.106606 (+ 90689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.456190 (+349584us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b54002e59d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:38.456363 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":822772480}
I20260709 13:06:38.459467 19703 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:38.460721 19521 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.461544 19768 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.470660 19768 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.473137 19417 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:06:38.475124 13874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3116) took 1646 ms (client timeout 1812 ms). Trace:
W20260709 13:06:38.475636 13874 rpcz_store.cc:269] 0709 13:06:36.829066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.868506 (+ 39440us) service_pool.cc:224] Handling call
0709 13:06:38.474611 (+1606105us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b540026d9d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:38.474904 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1232119552}
I20260709 13:06:38.476229 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.461177 19767 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:38.465190 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:38.477100 13945 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:38.477536 19517 raft_consensus.cc:3055] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:38.477661 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.478080 19517 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:38.478621 13948 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
W20260709 13:06:38.468500 13989 connection.cc:256] RPC call timeout handler was delayed by -2.1578884259999995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2514s for a response
I20260709 13:06:38.478516 19775 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:38.469205 19728 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:38.479954 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.462385 19521 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:38.478541 19636 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:38.492086 19636 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:38.492674 14556 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:38.492838 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2948) took 5618 ms. Trace:
I20260709 13:06:38.482915 19775 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.493305 14017 rpcz_store.cc:276] 0709 13:06:32.874304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.539163 (+664859us) service_pool.cc:224] Handling call
0709 13:06:36.707577 (+3168414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.766063 (+ 58486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.492335 (+1726272us) spinlock_profiling.cc:243] Waited 3.17 s on lock 0x7b54002981d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:38.492601 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7285630336}
I20260709 13:06:38.492690 19636 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:38.491430 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2688) took 3210 ms. Trace:
I20260709 13:06:38.494740 13944 rpcz_store.cc:276] 0709 13:06:35.281001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.414114 (+133113us) service_pool.cc:224] Handling call
0709 13:06:35.451889 (+ 37775us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.582736 (+130847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.490873 (+2908137us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400053a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:38.491209 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":292686720}
I20260709 13:06:38.495615 19775 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:38.479527 19517 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.500144 19776 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.499318 13944 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:38.500187 19517 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:38.502239 19779 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:38.502354 19703 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:38.502985 19779 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:38.503508 19779 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:38.503286 19703 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.499303 19664 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:38.505854 19628 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.505695 19664 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.506556 19628 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:38.507920 19738 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.269s	user 0.003s	sys 0.000s
I20260709 13:06:38.508347 19664 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:38.506563 19692 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:38.510787 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 2978) took 2836 ms. Trace:
I20260709 13:06:38.511221 14021 rpcz_store.cc:276] 0709 13:06:35.674229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.797550 (+1123321us) service_pool.cc:224] Handling call
0709 13:06:36.798125 (+   575us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:36.823575 (+ 25450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.823654 (+    79us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:36.824561 (+   907us) log.cc:844] Serialized 52 byte log entry
0709 13:06:37.053453 (+228892us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:37.053498 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.053579 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:38.506925 (+1453346us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:38.510179 (+  3254us) raft_consensus.cc:1685] finished
0709 13:06:38.510204 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.510546 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2073,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":11209,"spinlock_wait_cycles":9758464,"thread_start_us":201075,"threads_started":1,"wal-append.queue_time_us":344114}
I20260709 13:06:38.509217 19628 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:38.473826 19768 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:38.514075 13989 connection.cc:256] RPC call timeout handler was delayed by -1.9919314250000753s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1605s for a response
W20260709 13:06:38.514726 13989 connection.cc:256] RPC call timeout handler was delayed by -1.9550314250000156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
W20260709 13:06:38.515162 13989 connection.cc:256] RPC call timeout handler was delayed by -1.895942069000057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
W20260709 13:06:38.517194 19770 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.223s	user 0.002s	sys 0.000s
I20260709 13:06:38.518421 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:38.501055 19776 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:38.515564 19758 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
W20260709 13:06:38.517117 13989 connection.cc:256] RPC call timeout handler was delayed by -1.3797549949999848s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
I20260709 13:06:38.521821 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2690) took 3240 ms. Trace:
W20260709 13:06:38.536113 13989 connection.cc:256] RPC call timeout handler was delayed by -1.3486075399999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 13:06:38.520853 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 90063.8us
W20260709 13:06:38.521457 19765 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.219s	user 0.003s	sys 0.000s
I20260709 13:06:38.456454 19697 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:38.528816 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:38.531363 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2431) took 2301 ms. Trace:
W20260709 13:06:38.540571 13989 connection.cc:256] RPC call timeout handler was delayed by -1.3001420690000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2594s for a response
I20260709 13:06:38.534315 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2714) took 1613 ms. Trace:
I20260709 13:06:38.534943 19566 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:38.503227 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2412) took 5226 ms. Trace:
I20260709 13:06:38.539594 14023 rpcz_store.cc:276] 0709 13:06:36.229620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.000080 (+770460us) service_pool.cc:224] Handling call
0709 13:06:37.000412 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.878719 (+878307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.530791 (+652072us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54002ea754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:38.531156 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993039104}
I20260709 13:06:38.536216 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:38.542143 13989 connection.cc:256] RPC call timeout handler was delayed by -1.2974474090000285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 13:06:38.532747 19776 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:38.545066 13989 connection.cc:256] RPC call timeout handler was delayed by -1.231081482000036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
I20260709 13:06:38.541720 13942 rpcz_store.cc:276] 0709 13:06:36.921112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.921266 (+   154us) service_pool.cc:224] Handling call
0709 13:06:36.921591 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.390258 (+468667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.533789 (+1143531us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b5400092754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:38.534116 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":992867712}
W20260709 13:06:38.548642 13989 connection.cc:256] RPC call timeout handler was delayed by -1.2205429939999703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
W20260709 13:06:38.549201 13989 connection.cc:256] RPC call timeout handler was delayed by -1.1847471290000158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2292s for a response
W20260709 13:06:38.549738 13989 connection.cc:256] RPC call timeout handler was delayed by -1.161184128000059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 13:06:38.550217 13989 connection.cc:256] RPC call timeout handler was delayed by -1.0968215539999164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
W20260709 13:06:38.550680 13989 connection.cc:256] RPC call timeout handler was delayed by -1.0757107229999292s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19690000000000002s for a response
W20260709 13:06:38.551115 13989 connection.cc:256] RPC call timeout handler was delayed by -1.05467287700003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2097s for a response
W20260709 13:06:38.542130 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.554703 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.552358 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.555197 19566 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.559793 19763 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:38.573935 13944 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:38.564677 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2448) took 2303 ms. Trace:
I20260709 13:06:38.566468 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2720) took 1605 ms. Trace:
I20260709 13:06:38.567046 14560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 775) took 1480 ms. Trace:
I20260709 13:06:38.567333 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:38.567588 19775 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:38.568346 13945 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:38.571300 19768 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:38.562587 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.582384 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:38.582746 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:38.583565 13867 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:38.589937 13949 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:38.543033 14018 rpcz_store.cc:276] 0709 13:06:33.276400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.757645 (+1481245us) service_pool.cc:224] Handling call
0709 13:06:34.758113 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:37.203904 (+2445791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.203939 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:37.204432 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:06:37.269689 (+ 65257us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:37.306466 (+ 36777us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.306705 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.502565 (+1195860us) spinlock_profiling.cc:243] Waited 2.45 s on lock 0x7b540029a4d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:38.502640 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:38.502711 (+    71us) raft_consensus.cc:1685] finished
0709 13:06:38.502818 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.503020 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27733,"prepare.queue_time_us":89690,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":7021,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":34957,"spinlock_wait_cycles":5623741824,"thread_start_us":78171,"threads_started":2,"wal-append.queue_time_us":311268,"wal-append.run_cpu_time_us":2301,"wal-append.run_wall_time_us":1002256}
I20260709 13:06:38.584348 19559 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.591605 19559 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.592913 19566 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:38.593657 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 48295.4us
I20260709 13:06:38.576917 13941 rpcz_store.cc:276] 0709 13:06:36.961148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.974991 (+ 13843us) service_pool.cc:224] Handling call
0709 13:06:36.975332 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.066402 (+ 91070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.066438 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:37.066484 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:37.066523 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.565909 (+1499386us) spinlock_profiling.cc:243] Waited 90.8 ms on lock 0x7b5400087854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:38.565983 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.566247 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208810880}
I20260709 13:06:38.586560 19603 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:38.595470 19603 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.596467 19653 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:38.577265 14560 rpcz_store.cc:276] 0709 13:06:37.085163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.085493 (+   330us) service_pool.cc:224] Handling call
0709 13:06:37.194225 (+108732us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:37.204913 (+ 10688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.204962 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:37.205548 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:06:38.139680 (+934132us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b18000e4400. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:06:38.280823 (+141143us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:38.280937 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.281054 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:38.520455 (+239401us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:38.565686 (+ 45231us) raft_consensus.cc:1685] finished
0709 13:06:38.565711 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.565893 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":14484,"spinlock_wait_cycles":47236096,"thread_start_us":98711,"threads_started":1,"wal-append.queue_time_us":1086589}
I20260709 13:06:38.584733 19763 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:38.598032 19603 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:38.594426 13839 leader_election.cc:400] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17.
I20260709 13:06:38.602792 13839 leader_election.cc:403] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:06:38.604262 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:38.607120 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:38.607678 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:38.608162 14561 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:38.608495 14551 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:38.610996 19489 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:38.614301 14554 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:38.614820 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47749.6us
I20260709 13:06:38.615804 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:38.616142 19517 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:38.598515 19703 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:38.616873 19517 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:38.598961 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 43697us
I20260709 13:06:38.536389 13950 rpcz_store.cc:276] 0709 13:06:35.281502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.460515 (+179013us) service_pool.cc:224] Handling call
0709 13:06:35.460819 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.860170 (+1399351us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:36.994319 (+134149us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.180992 (+186673us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:37.181052 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:37.181101 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.521330 (+1340229us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54000f4e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:38.521411 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.521622 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1681,"prepare.queue_time_us":111346,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":9539,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":77143,"spinlock_wait_cycles":3217543040,"thread_start_us":317974,"threads_started":3,"wal-append.queue_time_us":258351,"wal-append.run_cpu_time_us":2042,"wal-append.run_wall_time_us":1106410}
I20260709 13:06:38.601955 14560 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:38.616290 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:38.600262 19467 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:06:38.647421 19780 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.648242 19780 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.635424 19603 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.650295 14553 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:06:38.650542 19603 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.650847 19768 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.653101 19603 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:38.650538 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:38.651332 19736 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.414s	user 0.009s	sys 0.002s
I20260709 13:06:38.656920 13914 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:38.655851 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:38.658545 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:38.659195 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.665371 19603 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.666740 13875 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:38.659823 13870 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:06:38.576853 14019 rpcz_store.cc:276] 0709 13:06:36.261069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.880569 (+1619500us) service_pool.cc:224] Handling call
0709 13:06:37.881783 (+  1214us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:37.954781 (+ 72998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.954895 (+   114us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:37.956735 (+  1840us) log.cc:844] Serialized 52 byte log entry
0709 13:06:38.203744 (+247009us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:38.216666 (+ 12922us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.243865 (+ 27199us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.244157 (+   292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.244287 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:38.299757 (+ 55470us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b48002115d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:06:38.514714 (+214957us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:38.564194 (+ 49480us) raft_consensus.cc:1685] finished
0709 13:06:38.564219 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.564450 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13269,"prepare.queue_time_us":47592,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":3464,"raft.queue_time_us":7000,"raft.run_cpu_time_us":3218,"raft.run_wall_time_us":123389,"spinlock_wait_cycles":101758336,"thread_start_us":242408,"threads_started":1,"wal-append.queue_time_us":307077}
I20260709 13:06:38.669343 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.653170 19768 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:38.656158 19523 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.669019 19561 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:38.665936 19780 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:38.635895 19517 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.686664 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.665962 19603 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.684892 19561 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.662822 19636 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:38.693650 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 251546us
W20260709 13:06:38.694921 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.697270 19668 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.697162 18874 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 13:06:38.698010 19668 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.698714 18874 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:38.699237 18874 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:38.700578 19668 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:38.701709 19354 raft_consensus.cc:3055] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:38.702158 19354 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:38.703506 19354 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.704736 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:38.705029 19354 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:38.705390 14551 raft_consensus.cc:1813] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Handling vote request from an unknown peer e1649107c781428692458f10597c5742
I20260709 13:06:38.705971 14551 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:38.706310 13866 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:38.697468 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.706835 19778 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:38.694579 19566 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.708138 19566 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.711695 19489 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17.
I20260709 13:06:38.711544 19523 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.688840 19559 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:38.713670 13874 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:38.715559 13875 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:38.717090 14558 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:38.717980 19606 consensus_queue.cc:1059] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:06:38.719926 19341 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:38.713348 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:38.726177 19493 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:38.727370 19674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.632s	user 0.009s	sys 0.024s
W20260709 13:06:38.727830 19674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.687s	user 0.009s	sys 0.024s
I20260709 13:06:38.728248 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.728793 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:38.729691 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.730625 13870 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 20.
I20260709 13:06:38.732044 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.734494 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:38.736034 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:38.740890 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 124728us
I20260709 13:06:38.741681 19321 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.741900 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:38.742375 19321 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.742967 19674 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:38.743438 19674 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:38.744012 19674 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:38.744853 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.746178 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.729749 13868 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:38.695043 19561 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:38.748512 19341 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.755370 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:38.753430 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:38.760676 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.762724 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.763427 13871 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:38.765185 14019 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: [12.10-13.11]
I20260709 13:06:38.766526 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:38.767182 14556 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:38.768966 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:38.770116 19430 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:38.770731 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:38.770556 19430 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.771003 14551 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:38.774940 14558 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:38.776193 19430 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:38.777853 19726 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:38.757516 19493 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:38.776515 19776 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
W20260709 13:06:38.779937 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 96055.6us
I20260709 13:06:38.780483 19523 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:38.782325 19595 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:38.782878 19595 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:38.783322 19595 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:38.782450 19726 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.790679 19726 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.791929 19694 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:38.792652 19694 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:38.749827 19341 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.796872 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:38.797596 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:38.799495 19523 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:38.762815 19668 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.811831 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.816596 14019 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:38.796890 19566 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:38.818495 13947 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
W20260709 13:06:38.821877 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51281.8us
I20260709 13:06:38.822021 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:38.811903 19493 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:38.816978 19778 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:06:38.802043 19763 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:38.833449 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51424.9us
I20260709 13:06:38.815150 19668 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.826532 19726 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:38.826368 19606 consensus_queue.cc:1059] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:06:38.830237 19603 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:38.840847 19341 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:38.838855 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 49436.5us
I20260709 13:06:38.851588 18874 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:38.852648 14024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2437) took 2617 ms (client timeout 2877 ms). Trace:
W20260709 13:06:38.856247 14024 rpcz_store.cc:269] 0709 13:06:36.235004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.691038 (+456034us) service_pool.cc:224] Handling call
0709 13:06:38.852227 (+2161189us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b540023f854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:38.852453 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1042959744}
I20260709 13:06:38.853446 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:38.853787 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.854230 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:38.857723 19779 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.858376 14558 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:38.860102 19779 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:38.860720 19763 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:38.860980 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3090) took 3030 ms. Trace:
I20260709 13:06:38.861282 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:38.868541 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 185652us
I20260709 13:06:38.855304 19430 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:38.868266 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.880484 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.872377 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:06:38.881311 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 118024us
I20260709 13:06:38.881779 14556 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:38.869954 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 155064us
I20260709 13:06:38.883118 13868 raft_consensus.cc:1217] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Deduplicated request from leader. Original: 14.7->[15.8-15.8]   Dedup: 15.8->[]
I20260709 13:06:38.883312 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:38.883701 13868 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:06:38.861418 13866 rpcz_store.cc:276] 0709 13:06:35.832080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.842086 (+ 10006us) service_pool.cc:224] Handling call
0709 13:06:35.842732 (+   646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:38.706778 (+2864046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.860677 (+153899us) spinlock_profiling.cc:243] Waited 2.85 s on lock 0x7b54002e4fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:38.860881 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6545812352}
I20260709 13:06:38.865602 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3114) took 2865 ms. Trace:
I20260709 13:06:38.870687 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:38.855564 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:38.885975 13871 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:38.835716 19517 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:06:38.890153 18874 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:38.890753 19321 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:38.891360 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.892544 19606 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:38.893704 19534 raft_consensus.cc:3055] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:38.895004 19781 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:38.896526 13947 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:38.896817 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:38.897126 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.897569 19725 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:38.904886 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:38.881498 19636 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:38.900838 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.903527 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:38.847921 19603 raft_consensus.cc:3055] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:38.913839 19603 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:38.884605 13872 rpcz_store.cc:276] 0709 13:06:36.000188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.559465 (+1559277us) service_pool.cc:224] Handling call
0709 13:06:37.559799 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.561330 (+  1531us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:06:38.184095 (+622765us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.436617 (+252522us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.638390 (+201773us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.654122 (+ 15732us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.864696 (+210574us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b18002f8ae0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c0348a40 00007fb2c03434e4 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287 00007fb2c344e208
0709 13:06:38.864917 (+   221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:38.864993 (+    76us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:38.865061 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.865240 (+   179us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.865407 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":315662,"prepare.queue_time_us":435549,"prepare.run_cpu_time_us":177,"prepare.run_wall_time_us":180,"raft.queue_time_us":359802,"raft.run_cpu_time_us":577,"raft.run_wall_time_us":2450,"spinlock_wait_cycles":11044096,"thread_start_us":668414,"threads_started":3,"wal-append.queue_time_us":198371}
I20260709 13:06:38.914824 19603 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.915956 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:38.916247 19603 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:38.917788 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:38.918437 19559 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:38.919014 19559 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:38.919497 19781 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:38.920043 19559 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.921519 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:38.925312 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:38.926523 13872 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:38.884541 19779 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:06:38.931641 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:38.932438 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.907717 13943 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:38.908255 19783 consensus_queue.cc:1059] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:38.908947 19503 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:38.948459 13949 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:38.910624 19561 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:38.911001 19668 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:38.901314 19665 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:06:38.904204 19534 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:38.906876 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:38.889182 19595 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:38.943749 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3103) took 2972 ms. Trace:
I20260709 13:06:38.962563 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:38.963958 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 116536us
I20260709 13:06:38.943006 19743 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:38.962093 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.952795 19503 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:38.963985 13867 rpcz_store.cc:276] 0709 13:06:35.971619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.479295 (+1507676us) service_pool.cc:224] Handling call
0709 13:06:37.479549 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:38.648040 (+1168491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.943395 (+295355us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003ab6d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:38.943658 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2537685888}
I20260709 13:06:38.944103 19489 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:38.951050 19779 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.962554 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:38.973338 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.969215 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:38.982481 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:38.986903 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:38.993048 19606 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:38.993275 13767 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 7 to 13, term changed from 14 to 18, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to e1649107c781428692458f10597c5742 (127.12.109.65), VOTER 3779649036364744823fc035dac892c7 (127.12.109.66) evicted. New cstate: current_term: 18 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:38.995875 13767 catalog_manager.cc:5697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:38.998387 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:39.000473 19789 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 13:06:39.002626 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 105878us
I20260709 13:06:39.002818 19430 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.004252 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:39.005036 14561 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:39.005168 19341 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:38.973484 19489 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.968089 19668 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:39.006181 19489 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:38.972124 19743 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:39.005739 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:38.978550 19779 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:38.986151 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:39.016934 14556 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:38.974376 19503 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:39.020781 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 200808us
I20260709 13:06:39.038895 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:39.039249 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.039736 19595 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.040004 13874 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:06:39.040383 19595 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.023281 19520 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:39.040881 19595 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.041304 19520 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:39.041724 19520 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:39.042359 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.046096 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:39.046512 19668 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:39.046550 19493 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:39.047219 19493 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.049522 19493 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:39.050194 19694 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.053066 19679 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:39.053552 19679 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:39.054522 19679 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.055158 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:39.055789 14554 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:39.055938 19679 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:39.056723 19520 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.058408 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:39.059628 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47384.9us
I20260709 13:06:39.059980 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:39.060788 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:39.061489 13871 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:39.062016 19566 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:39.062744 19603 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:39.026405 19534 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.070838 19534 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:39.020336 14553 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:38.975594 19674 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.031529 19656 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:39.072888 19656 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.074311 19674 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.075178 19656 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.031927 19561 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:39.032281 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:39.034320 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 81554.2us
I20260709 13:06:39.076764 13838 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:39.034794 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:39.035364 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.078722 14522 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:39.079746 19430 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:39.066043 14551 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:39.080374 19636 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:06:39.078698 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.082976 19595 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.084216 19595 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.084420 13947 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:39.084708 19595 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.023707 19743 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.086550 13947 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:39.086961 19743 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:39.088255 13947 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:39.089311 19489 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.067834 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.090400 13914 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:39.090634 13874 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:39.091248 19517 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:39.091689 19517 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.092238 19517 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:39.068509 13839 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:39.069418 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:39.095278 19783 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:39.097429 14554 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
W20260709 13:06:39.098661 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 93016.9us
I20260709 13:06:39.099332 19781 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:39.099155 13767 catalog_manager.cc:5697] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:39.099783 19783 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:39.098378 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:39.101082 13839 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:39.101639 19766 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:39.103199 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:39.111316 13914 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:39.112223 19694 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.113126 13943 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
W20260709 13:06:39.114194 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 206359us
I20260709 13:06:39.115926 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:39.119015 13941 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
W20260709 13:06:39.119498 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 57493.6us
I20260709 13:06:39.130818 13950 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:39.132069 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:39.149860 14522 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:39.143535 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:39.150458 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:39.150846 13949 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:39.130738 19341 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:39.116153 19493 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:06:39.135226 19606 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:39.167933 19774 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:39.141299 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:39.174003 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:39.174391 19779 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.175226 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:39.140290 13871 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 21.
I20260709 13:06:39.153384 19636 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:39.155246 19782 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:39.155359 19780 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:39.156985 13945 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:06:39.158237 19666 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:06:39.159211 19674 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.159781 19570 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:39.160041 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 61884.9us
I20260709 13:06:39.186008 19725 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:39.160236 19763 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:39.186610 19725 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:39.186859 19763 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.187700 19725 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.189221 19763 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.189865 19570 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:39.190994 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45719.1us
I20260709 13:06:39.191883 13989 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:39.192268 19570 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.160782 19561 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:39.161134 19595 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:39.162376 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 51184.7us
I20260709 13:06:39.194231 19604 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:06:39.194525 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.162825 19321 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:39.195199 13914 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:39.195199 19321 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:39.155514 19668 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:39.196460 19321 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.197022 19523 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:39.197583 19523 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:39.162936 19628 raft_consensus.cc:3055] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:39.198518 19523 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.199371 19674 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.199708 19503 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.201485 19674 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.201745 19503 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.149015 19766 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:39.154043 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 53052.3us
I20260709 13:06:39.233381 19606 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:39.236608 19603 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:39.239862 13942 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:39.179453 13949 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:06:39.180751 19341 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:39.184719 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:39.244127 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 59500.7us
I20260709 13:06:39.198935 19628 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:39.248698 19628 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.250020 19628 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:39.251030 19781 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:39.252560 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.252988 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.253268 13871 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:39.153291 19768 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:39.202744 19790 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.092s	user 0.001s	sys 0.003s
I20260709 13:06:39.254169 19768 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:39.254416 19503 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:39.254665 19768 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:39.203397 19780 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:39.255142 13872 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:39.254951 19503 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.256083 19780 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.203986 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:39.256596 19341 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:39.204561 19665 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:39.205044 13767 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:39.204005 19779 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:39.206434 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:39.180860 19782 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:39.164131 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 60991.6us
I20260709 13:06:39.211961 19520 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:39.215171 19794 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:06:39.218636 19561 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:06:39.262704 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58786.6us
I20260709 13:06:39.224030 19517 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.225639 19489 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.226642 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:39.280896 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 54347.3us
W20260709 13:06:39.229012 13769 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57648 (request call id 280) took 44116 ms (client timeout 15000 ms). Trace:
I20260709 13:06:39.193993 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:39.241410 19606 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:39.247571 19354 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:39.237058 19766 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:39.284709 19354 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.285353 19354 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:39.286289 19603 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:39.286741 19603 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.287071 19603 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:39.256173 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:39.259707 13867 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:39.289162 13870 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:39.259914 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.261385 14553 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:39.266255 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:39.258376 19665 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:39.277748 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.263301 19782 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:39.253856 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.256608 19780 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.293356 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.312939 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:39.315133 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:39.315656 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.317057 14524 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:39.333597 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.334987 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.342592 13867 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:39.239635 19559 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
W20260709 13:06:39.358587 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 64105.1us
I20260709 13:06:39.359987 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.360966 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.288965 19606 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:39.295047 19665 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.361687 13873 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:39.365245 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:39.367740 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.340299 19668 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:39.379000 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.377691 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:39.380802 13949 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:39.385156 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:39.385814 13869 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:39.359923 19726 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:39.353726 19774 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.358243 19520 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.392866 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:39.394418 13872 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:39.392593 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 100699us
W20260709 13:06:39.394507 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 200601us
I20260709 13:06:39.398196 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:39.372629 19665 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:39.401640 13948 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:39.393736 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:39.414767 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55335.4us
I20260709 13:06:39.415933 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:39.417714 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.422132 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:39.372956 19694 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:06:39.439883 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:39.426201 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:39.440959 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:39.343586 19341 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.419402 19493 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.485893 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:39.505329 13949 raft_consensus.cc:2410] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:39.487135 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.513314 13873 raft_consensus.cc:2368] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17.
I20260709 13:06:39.515216 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:39.517400 19430 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:39.517647 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:39.519053 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.519836 13866 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:06:39.526393 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 51224.7us
I20260709 13:06:39.528223 19517 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:39.532043 19803 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:06:39.533249 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 108687us
I20260709 13:06:39.514530 13767 catalog_manager.cc:5697] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:39.534070 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:39.535202 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:39.422610 19679 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:06:39.542157 19679 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.542348 19725 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:39.537555 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:39.538607 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:39.543183 14019 raft_consensus.cc:1813] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Handling vote request from an unknown peer e1649107c781428692458f10597c5742
I20260709 13:06:39.544215 14019 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:06:39.419335 19520 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:39.546530 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.543776 14023 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:39.551205 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:39.551990 14016 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:39.571936 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:39.506543 19566 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:39.570847 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 96491.1us
I20260709 13:06:39.475863 19559 consensus_queue.cc:1059] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:06:39.438724 19489 consensus_queue.cc:1059] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.118s
I20260709 13:06:39.589308 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:39.472831 19774 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.542836 19725 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:39.600664 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 180052us
W20260709 13:06:39.602502 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 174003us
I20260709 13:06:39.533015 19503 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.588095 14018 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:39.588558 14024 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:39.588646 13871 raft_consensus.cc:2376] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 16. Re-sending same reply.
W20260709 13:06:39.590377 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.602461 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:39.282931 13769 rpcz_store.cc:269] 0709 13:05:55.113095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.114190 (+  1095us) service_pool.cc:224] Handling call
0709 13:06:39.228726 (+44114536us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:30.451697 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:30.452409 (+   712us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:30.452462 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:06:30.452481 (+    19us) tablet.cc:662] Decoding operations
0709 13:06:30.453712 (+  1231us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:30.453816 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:30.453845 (+    29us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:06:30.826152 (+372307us) tablet.cc:701] Row locks acquired
0709 13:06:30.826217 (+    65us) write_op.cc:260] PREPARE: finished
0709 13:06:30.826425 (+   208us) write_op.cc:270] Start()
0709 13:06:30.826586 (+   161us) write_op.cc:276] Timestamp: P: 1783602390826356 usec, L: 0
0709 13:06:30.826621 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:06:31.024996 (+198375us) log.cc:844] Serialized 4795 byte log entry
0709 13:06:32.072085 (+1047089us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:32.592533 (+520448us) op_driver.cc:464] REPLICATION: finished
0709 13:06:32.994524 (+401991us) write_op.cc:301] APPLY: starting
0709 13:06:32.994642 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:33.572818 (+578176us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:33.572875 (+    57us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:33.791280 (+218405us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:33.791675 (+   395us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:06:33.791729 (+    54us) write_op.cc:312] APPLY: finished
0709 13:06:33.792184 (+   455us) log.cc:844] Serialized 113 byte log entry
0709 13:06:33.956877 (+164693us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:33.967602 (+ 10725us) write_op.cc:454] Released schema lock
0709 13:06:34.006271 (+ 38669us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:34.068845 (+ 62574us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:34.068886 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16017,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":401078,"apply.run_cpu_time_us":48334,"apply.run_wall_time_us":1049487,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54215,"num_ops":12,"prepare.queue_time_us":467784,"prepare.run_cpu_time_us":35015,"prepare.run_wall_time_us":865744,"raft.queue_time_us":155673,"raft.run_cpu_time_us":44591,"raft.run_wall_time_us":282857,"replication_time_us":1731062,"spinlock_wait_cycles":35948800,"thread_start_us":989940,"threads_started":5,"wal-append.queue_time_us":618869,"wal-append.run_cpu_time_us":28099,"wal-append.run_wall_time_us":3113794}]]}
I20260709 13:06:39.612358 13869 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:06:39.615403 14019 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
W20260709 13:06:39.618093 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 76108.1us
I20260709 13:06:39.537335 19493 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.542815 19679 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.622463 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:39.619253 13875 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:39.571230 19781 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:06:39.595463 19595 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.605356 19725 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:39.632652 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.631145 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:39.583817 13870 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:39.649570 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:39.652940 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:39.654551 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 50915.6us
I20260709 13:06:39.630136 19780 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:39.660416 14553 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:06:39.635766 19559 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:39.647035 19561 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:06:39.664645 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 137466us
I20260709 13:06:39.673738 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:39.674458 14018 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:39.668548 13838 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:39.631683 19656 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:06:39.635890 19595 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:39.685491 19804 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:06:39.690264 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:06:39.690882 19800 log.cc:927] Time spent T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
W20260709 13:06:39.692361 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 70016.3us
I20260709 13:06:39.699693 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:39.693918 14552 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:39.663630 19559 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:39.682859 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:39.701783 14017 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:39.670302 19561 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.670044 19726 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:39.689407 19354 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.677507 19321 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:39.713256 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:39.716081 14015 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:06:39.677307 19780 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:06:39.724694 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:39.701534 19656 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:39.723042 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:39.711125 19561 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:39.730110 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 100519us
I20260709 13:06:39.713600 19636 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:06:39.732527 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.755678 14023 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:06:39.727644 19668 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:39.757843 19780 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:39.740612 19636 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:39.758884 19812 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:39.758445 19559 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.761590 19766 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:39.762923 19809 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:06:39.763830 19813 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:06:39.725138 19781 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:39.770804 14017 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
W20260709 13:06:39.786093 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 44519.3us
I20260709 13:06:39.788830 19595 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:39.842840 19816 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:06:39.794322 19493 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:39.795578 19810 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:06:39.796763 19768 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.002s
I20260709 13:06:39.730145 19656 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:39.798890 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 130442us
I20260709 13:06:39.899133 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:39.899631 13838 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:39.765373 19520 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.808817 19636 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:39.811739 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:39.811848 19812 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:06:39.815488 19603 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.002s
W20260709 13:06:39.818985 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 96053us
I20260709 13:06:39.820881 14018 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:39.866998 19679 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:39.838174 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.841725 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:39.837284 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.849290 14552 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:39.832798 19815 raft_consensus.cc:3055] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:39.834594 19503 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:39.856696 13874 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:39.837214 19774 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.789283 19354 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:39.743599 19628 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.108s
I20260709 13:06:39.873374 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:39.941990 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 100362us
I20260709 13:06:39.833112 19782 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:39.945327 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 72060.4us
I20260709 13:06:39.912489 19818 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:39.839607 19783 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:39.909523 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:39.912578 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:39.951260 13873 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:39.951411 14020 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:39.910643 14022 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
W20260709 13:06:39.951733 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 146915us
I20260709 13:06:39.800024 19606 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.918116 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:39.830736 13872 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:39.680366 13767 catalog_manager.cc:5697] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 11 to 13, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 13 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:39.921993 13866 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:39.938756 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:39.958665 14018 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:39.829967 19665 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.890690 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:39.950404 13914 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:39.947767 19694 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:39.925125 19815 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:39.906816 19779 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:06:39.909808 19666 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:06:39.971349 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:39.911051 19768 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:39.943416 19559 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:39.972297 14023 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:39.948203 19814 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.981822 13869 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:39.955057 19818 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.954519 19595 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:39.981848 14018 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 13:06:39.954298 19783 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:39.925980 19503 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.921218 19603 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:39.988247 13871 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:06:39.967239 19493 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.975131 19321 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:06:39.985432 19822 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:06:39.991770 13989 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:40.006003 14021 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:40.008805 13873 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:06:39.990471 19818 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:40.027602 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.029657 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.026777 19725 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.031849 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.036692 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.042042 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.042996 14017 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:40.044377 19725 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:40.004853 19595 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:40.049968 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:39.962116 19656 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:40.014830 19321 raft_consensus.cc:697] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:40.005513 19493 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.049513 19628 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:39.992789 19814 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } attrs { replace: true } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:40.073657 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 114107us
W20260709 13:06:40.078608 19819 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.321s	user 0.005s	sys 0.000s
I20260709 13:06:40.081066 19603 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:40.083755 14020 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:06:40.084182 19321 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.068665 19656 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:40.090278 13767 catalog_manager.cc:5697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 11 to 15, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 15 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:40.092389 13767 catalog_manager.cc:5697] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 5 to 8, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 8 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:40.095506 19493 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:40.100611 19561 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:40.094863 19595 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.112962 14015 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
W20260709 13:06:40.116416 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 170046us
I20260709 13:06:40.118528 14553 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:40.120922 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:40.135169 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.139264 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:40.139472 13989 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:40.140048 14554 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:06:40.142488 14552 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:40.144212 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:40.145463 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:40.149323 13942 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:06:40.012177 19812 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.164219 14556 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:40.173230 14553 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:40.151618 19566 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:40.187175 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 73255.8us
I20260709 13:06:40.187422 14015 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:40.188825 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:40.189610 14015 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:40.194347 19783 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.195266 19561 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:40.195596 19783 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:40.195915 19779 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:40.182299 14023 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:40.198671 19815 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:40.200476 19815 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:40.201205 19503 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:40.204279 19606 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:40.205891 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:40.207212 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.208312 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.210332 13875 raft_consensus.cc:3055] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:40.210749 13875 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:40.211613 13875 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.212744 13875 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:40.198678 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.215258 14023 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:40.216251 13945 raft_consensus.cc:2410] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:06:40.231967 14015 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:40.232834 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.234899 14023 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:40.239207 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.239838 13870 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:40.129859 19725 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:06:40.240551 14015 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:40.241531 14021 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:06:40.176167 19781 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:40.249048 13941 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:40.250348 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:40.251200 13947 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:40.253271 14015 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:40.254506 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:40.255422 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.256183 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:40.256574 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:40.257459 14553 raft_consensus.cc:2410] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:06:40.257331 13989 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:40.258710 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.259377 14017 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:40.260939 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:40.261605 14021 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:40.289343 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:40.290904 13947 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:06:40.302244 19766 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:40.285135 19636 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.306823 19636 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:40.307790 13943 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:06:40.308844 14552 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []
W20260709 13:06:40.311033 19833 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:06:40.275712 19606 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:40.320380 19534 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:40.320852 19534 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:40.322510 19837 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:40.323031 19341 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.323570 19341 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:40.325253 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.326171 19835 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.326751 19835 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.328295 14554 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:06:40.328697 14554 pending_rounds.cc:85] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting all ops after (but not including) 10
I20260709 13:06:40.328984 14554 pending_rounds.cc:107] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:06:40.329018 19835 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:40.329360 14554 raft_consensus.cc:2889] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:40.330657 13944 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:40.331708 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:40.333212 13948 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:40.294924 19628 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:40.334569 19628 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:40.266853 19668 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:06:40.336790 19628 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:40.330544 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:40.338872 13949 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:40.320132 19836 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 13:06:40.341804 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:40.344702 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:40.325774 19341 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:40.347744 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:40.352103 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:40.352600 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:40.352818 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:40.353252 13948 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:40.354100 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:40.354589 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:40.357086 19489 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:40.358108 19489 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:40.264460 13767 catalog_manager.cc:5697] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 5 to 9, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 9 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:40.358785 19489 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:40.359181 19489 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:40.358134 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3067) took 1105 ms. Trace:
I20260709 13:06:40.360863 13874 rpcz_store.cc:276] 0709 13:06:39.252811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.254134 (+  1323us) service_pool.cc:224] Handling call
0709 13:06:39.254461 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:39.938908 (+684447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:40.357853 (+418945us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b540026c0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:40.358067 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1384525056}
I20260709 13:06:40.362217 19520 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:40.362689 19814 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:40.362874 19430 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:40.363487 19430 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:40.362663 19520 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.364178 19520 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:40.364915 19817 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.364492 19430 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.365432 19817 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:40.356686 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:40.366484 14017 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:40.360754 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:40.359743 13767 catalog_manager.cc:5697] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 8 to 10, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 10 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:40.372792 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.374202 14015 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:40.354645 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:40.388775 14553 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:06:40.402992 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:40.403683 14553 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:40.375654 19636 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:06:40.405509 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 50974.7us
I20260709 13:06:40.414350 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:40.415391 13943 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:40.415589 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:40.376339 19559 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:40.416498 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:40.416751 14556 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:06:40.417092 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:40.426877 13949 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:06:40.433885 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:40.444370 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 187081us
I20260709 13:06:40.445217 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:40.450898 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.360066 19489 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.456540 13948 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:40.457881 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:40.397013 19666 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:06:40.420188 19559 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:40.438258 19603 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:40.389247 19818 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:40.467823 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:40.468694 19503 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:40.468817 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:40.469329 19503 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:40.443543 19595 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:40.470012 19595 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.470404 19503 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:40.472466 19595 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:40.473649 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:40.474545 14551 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 11: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:06:40.475616 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.476922 19821 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:40.477496 19821 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:40.478577 19821 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:40.481815 19595 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.482473 19595 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:40.472381 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:40.483074 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:40.501353 13943 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
W20260709 13:06:40.503654 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45868us
I20260709 13:06:40.504642 13989 leader_election.cc:400] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17.
I20260709 13:06:40.504958 13989 leader_election.cc:403] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:40.438515 19776 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:06:40.465569 19603 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:40.464653 19559 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:40.464895 19837 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:40.458305 19838 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:40.463801 19848 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:40.515986 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:40.516803 14551 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:40.490074 19782 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:40.490383 19781 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 13:06:40.521230 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:40.518409 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46114.3us
I20260709 13:06:40.517477 19824 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:40.512957 19595 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:40.465070 19815 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:06:40.532387 19321 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:40.533725 19818 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:40.538722 19840 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:06:40.513223 19603 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:40.547531 13989 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:40.544451 14021 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:40.550448 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:40.554368 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.558447 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:40.552986 19328 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:06:40.559533 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:40.559434 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:40.524945 19837 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17.
I20260709 13:06:40.563285 14551 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:06:40.588033 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:40.588799 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:40.545158 19844 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:40.458701 13767 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 6 to 9, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 9 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:40.596405 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:40.795118 13944 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:40.728171 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 240961us
W20260709 13:06:40.795663 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 248216us
W20260709 13:06:40.796198 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 207464us
I20260709 13:06:40.797561 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:40.763330 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.783326 13947 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:40.800364 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:40.800844 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:40.801272 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.802631 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:40.663885 19851 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:06:40.561977 19782 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:40.662586 19842 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
W20260709 13:06:40.580742 19846 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:06:40.805272 13872 raft_consensus.cc:2368] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18.
W20260709 13:06:40.790356 19858 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:06:40.698113 19847 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.689967 19838 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:40.812192 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.679216 19776 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:40.628032 19817 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:40.814818 14015 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:40.815369 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.817373 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:40.788236 19763 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:40.756388 19824 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:40.791450 19636 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:40.818935 14024 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:40.819443 19665 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:40.792462 19321 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:40.795344 19628 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:40.808724 19566 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:40.826625 13767 catalog_manager.cc:5697] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 11 to 13, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 13 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:40.834739 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:40.904001 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:40.836236 19520 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.842131 19768 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:40.685626 19559 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:40.792465 19341 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:40.869812 19844 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.852288 13947 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:40.745175 19606 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:40.853447 14018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.011s	sys 0.008s
I20260709 13:06:40.860962 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:40.864208 14524 leader_election.cc:400] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18.
I20260709 13:06:40.922506 14524 leader_election.cc:403] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:06:40.873297 19763 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:40.604758 19354 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.244s
I20260709 13:06:40.871045 14016 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
W20260709 13:06:40.898680 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 49740.2us
I20260709 13:06:40.904876 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:40.961030 14023 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 17: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:40.913122 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.684757 19534 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.915243 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:40.963425 14015 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:40.846624 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:40.904736 13950 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:40.866451 19837 consensus_queue.cc:1059] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
W20260709 13:06:40.865155 13942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.015s	sys 0.018s
W20260709 13:06:40.969784 13942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.016s	sys 0.019s
W20260709 13:06:40.922194 14018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.011s	sys 0.008s
I20260709 13:06:40.919835 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:40.927413 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:40.967836 19354 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:40.971823 14020 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:40.972015 19354 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:40.866118 19847 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:40.973804 19354 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.974661 14023 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:40.975404 19354 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:40.975914 19847 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:40.977681 19534 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:40.968174 13950 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:40.862025 19782 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.981801 13767 catalog_manager.cc:5697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 9 to 16, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 16 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:40.921841 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:40.923041 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 60215.2us
I20260709 13:06:40.875200 19776 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.993515 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:40.998463 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:40.998850 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:41.004928 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:40.928357 19430 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18.
I20260709 13:06:40.956312 19559 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:41.009653 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 49283.6us
I20260709 13:06:41.010705 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:40.918462 19866 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.877070 19838 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:40.990442 19782 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.025712 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:40.928174 19606 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.007732 19328 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:41.050319 19559 consensus_queue.cc:1059] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:40.929080 19844 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.051918 19559 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:41.052840 19867 raft_consensus.cc:3055] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:41.052654 19559 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.053319 19867 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:41.054095 19844 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.054767 19844 raft_consensus.cc:3055] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:41.056990 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.012378 19636 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.044392 13872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:06:41.045670 19603 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.037869 19763 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.046687 19520 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.048960 19838 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.057904 13875 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:41.059906 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.061532 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.064283 14020 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:06:41.066854 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 829) took 1197 ms. Trace:
I20260709 13:06:41.111132 13869 rpcz_store.cc:276] 0709 13:06:39.869368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.870585 (+  1217us) service_pool.cc:224] Handling call
0709 13:06:39.871039 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:39.982324 (+111285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.066563 (+1084239us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54002e6dd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:41.066786 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":254296832}
I20260709 13:06:41.067289 14015 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
W20260709 13:06:41.074065 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 198402us
W20260709 13:06:41.112826 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.034960 19874 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:41.103849 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.058915 19817 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:41.104702 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:41.057477 19844 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:41.097729 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.121958 14556 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:41.113363 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.114400 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.119053 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:41.123261 13867 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:41.119244 14019 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:41.119765 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.104202 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.119845 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:41.017035 19780 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 13:06:41.127714 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:41.121565 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 123185us
W20260709 13:06:41.113260 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.109951 19328 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:41.131072 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.137359 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.117350 19763 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:41.141597 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:41.142323 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 1.2446e+06us
I20260709 13:06:41.142338 13869 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:41.143316 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:41.144158 13875 raft_consensus.cc:2410] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:06:41.144032 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:41.144702 14552 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:41.121754 19817 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:41.131161 19774 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:41.184446 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.025377 19824 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.124454 19566 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:41.144680 19853 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:06:41.192893 19881 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:41.202452 14556 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
W20260709 13:06:41.190943 19872 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:06:41.024495 19866 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:41.146884 19864 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:06:41.212589 19603 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:41.193228 19606 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:41.214977 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3010) took 1176 ms. Trace:
I20260709 13:06:41.217132 13871 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:41.221297 19559 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.193571 19817 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:41.114915 19867 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.222658 19665 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.261327 19665 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.262408 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:41.222404 19566 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.263212 13868 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 17: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:41.263636 19665 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.264561 13868 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:06:41.210657 19783 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:41.224861 19853 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:41.265712 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:41.230473 19321 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.230477 19776 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.205750 19774 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:41.243281 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 100026us
I20260709 13:06:41.245756 13839 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:41.311606 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 65890.2us
I20260709 13:06:41.249747 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:41.250552 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.125643 19874 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.250856 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.313338 13875 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:41.247668 19628 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:41.251909 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:41.267539 19489 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:41.252619 19866 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.247097 19768 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.254705 19503 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:41.220120 19881 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.244922 14022 rpcz_store.cc:276] 0709 13:06:40.039053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.039543 (+   490us) service_pool.cc:224] Handling call
0709 13:06:40.039925 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:40.094465 (+ 54540us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:40.094840 (+   375us) log.cc:844] Serialized 14 byte log entry
0709 13:06:40.847266 (+752426us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b18003b0200. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287
0709 13:06:40.847594 (+   328us) log.cc:844] Serialized 14 byte log entry
0709 13:06:40.867001 (+ 19407us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:40.867055 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:40.867113 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.214726 (+347613us) spinlock_profiling.cc:243] Waited 54.2 ms on lock 0x7b54000f23d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:41.214778 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:41.214914 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28230,"prepare.queue_time_us":1745,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":91,"raft.queue_time_us":68941,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":129889408,"thread_start_us":311827,"threads_started":2,"wal-append.queue_time_us":318652}
I20260709 13:06:41.315268 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.317909 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.231366 19328 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.316113 13839 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:41.313655 13869 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:41.313696 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:41.325788 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:41.328661 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:41.329636 14553 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
W20260709 13:06:41.334462 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.320322 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:41.347707 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:41.350880 14556 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 10.
I20260709 13:06:41.283241 19853 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:41.356177 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:41.364882 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:41.364794 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:41.311201 19821 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:41.361801 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.318295 19780 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:41.315276 19774 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:41.285851 19811 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.069s
I20260709 13:06:41.232049 19844 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.355545 19866 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:41.365343 19666 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:41.285756 19783 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.346390 19665 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:41.344826 19818 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:41.374625 19766 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:41.378579 19520 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:41.321928 19881 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:41.342734 19768 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.414942 19866 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 22 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:41.401908 19666 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.433550 13874 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:41.374750 19559 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
W20260709 13:06:41.440303 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 82288.7us
I20260709 13:06:41.430212 19844 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:41.431142 19520 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.427779 19878 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:41.443156 19666 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:41.418529 19665 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:41.481015 19891 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 13:06:41.511229 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.484138 19520 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:41.484197 19878 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.500890 19888 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
W20260709 13:06:41.514525 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 62784us
W20260709 13:06:41.524566 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.531297 19595 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.533195 19888 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:41.533349 19595 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.534044 19817 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.426654 19853 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.536679 13870 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:41.533725 19888 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:41.536034 19595 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.525929 19878 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:41.541391 13873 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:06:41.512948 19763 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:06:41.543699 19888 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.549468 19817 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:41.568346 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 195081us
I20260709 13:06:41.554611 19853 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:41.598762 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.542863 19866 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.587133 19766 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.589058 19888 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:41.608661 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 74152.3us
I20260709 13:06:41.590322 19874 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.602874 19783 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.608192 19780 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:41.635322 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:41.659375 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.627064 19824 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.659350 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:41.652863 19328 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.675662 19665 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.676326 19903 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:06:41.679872 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.682034 13871 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:41.681885 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:41.682440 19768 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.683017 19321 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.685081 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:41.685635 19595 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:06:41.712577 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 53347.3us
I20260709 13:06:41.710392 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:41.686399 19780 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.688151 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.688416 19824 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.690492 13875 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:41.692414 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:41.694521 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.694576 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:41.619216 19766 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:41.704727 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.714249 13838 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:41.717850 13944 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:41.721113 13946 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:06:41.721448 13915 connection.cc:256] RPC call timeout handler was delayed by -1.4541769899999508s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 13:06:41.721946 13915 connection.cc:256] RPC call timeout handler was delayed by -1.4253769900000179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
I20260709 13:06:41.725251 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:41.725888 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:41.722294 13915 connection.cc:256] RPC call timeout handler was delayed by -1.4028769899999816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
W20260709 13:06:41.726912 13915 connection.cc:256] RPC call timeout handler was delayed by -1.36867699000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260709 13:06:41.727389 13915 connection.cc:256] RPC call timeout handler was delayed by -1.337176989999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
I20260709 13:06:41.718619 19595 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.738488 19666 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.739969 19824 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.719172 19910 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.741837 19910 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.744712 13871 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:06:41.745674 13867 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:41.747777 13868 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:41.719041 19896 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.718493 19899 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:06:41.766587 13915 connection.cc:256] RPC call timeout handler was delayed by -1.16617698999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18330000000000002s for a response
I20260709 13:06:41.731539 19768 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:41.767293 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0947335009999506s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
I20260709 13:06:41.747659 19595 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.754168 19783 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:41.769304 13947 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:41.715606 19321 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.750192 19665 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.748736 19910 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.768527 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:41.722326 19520 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.711481 19328 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.764775 19896 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.791420 19328 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.791908 19896 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.793325 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:41.799703 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:41.770493 19774 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.780963 19783 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:41.820153 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2489) took 1021 ms. Trace:
I20260709 13:06:41.801976 19763 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:41.782984 19665 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:41.802744 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 669868us
I20260709 13:06:41.802812 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:41.784737 19666 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.774569 19768 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.787158 19595 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:41.817303 19901 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:41.801131 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:41.802225 19838 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:41.798159 19910 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:41.820943 14021 rpcz_store.cc:276] 0709 13:06:40.798991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.799103 (+   112us) service_pool.cc:224] Handling call
0709 13:06:40.805203 (+  6100us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:40.858461 (+ 53258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:40.858511 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:40.866832 (+  8321us) log.cc:844] Serialized 26 byte log entry
0709 13:06:41.065467 (+198635us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:41.108741 (+ 43274us) log.cc:844] Serialized 14 byte log entry
0709 13:06:41.801873 (+693132us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b18003ce3e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c0348a40 00007fb2c03434e4 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287 00007fb2c344e208
0709 13:06:41.806404 (+  4531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.806466 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:41.806500 (+    34us) raft_consensus.cc:1685] finished
0709 13:06:41.813499 (+  6999us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:41.820088 (+  6589us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":799605,"prepare.queue_time_us":46463,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":4405,"raft.queue_time_us":727725,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":768186,"spinlock_wait_cycles":5735296,"thread_start_us":144616,"threads_started":1,"wal-append.queue_time_us":225155}
I20260709 13:06:41.841774 19768 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.847460 19628 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:41.848026 19628 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.849677 19883 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.850443 19883 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.852407 19603 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:41.852998 19603 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.853345 19883 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.839264 19838 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.854822 19520 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:41.836018 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:41.858421 19881 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.860661 19881 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.862856 19665 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.863051 19321 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.864171 19628 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:41.866958 13944 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
I20260709 13:06:41.866580 19783 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:41.867816 19886 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.466s	user 0.004s	sys 0.000s
I20260709 13:06:41.871313 19603 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:41.871615 19912 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:06:41.873327 19766 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.823335 19877 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:06:41.823516 19895 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:41.825312 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 79264.6us
I20260709 13:06:41.876891 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:41.877144 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:41.829559 13866 raft_consensus.cc:2368] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16.
I20260709 13:06:41.877776 14553 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:41.832927 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:41.845223 13868 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:41.840386 19901 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:41.797633 19835 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:06:41.882578 19910 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:41.882750 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 81732.9us
I20260709 13:06:41.869729 19774 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:41.887387 13873 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:41.887786 19694 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:41.888550 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:41.883803 19811 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.892103 19901 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:41.897133 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 70128.3us
I20260709 13:06:41.897377 13943 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:06:41.898861 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 62941.4us
I20260709 13:06:41.899211 19668 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:06:41.900310 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 67518.4us
I20260709 13:06:41.942909 13989 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:41.900734 19606 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:41.950129 13867 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:41.902861 19865 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:41.953692 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:41.955111 13989 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:41.837512 19595 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:41.909515 13915 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:41.956190 19595 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:41.958478 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:41.960319 19341 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:41.961467 19341 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:41.914278 14553 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:41.961880 19341 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.962319 19341 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:41.947552 19636 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:41.963617 19844 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.964205 19844 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.923151 13838 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:41.936484 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:41.939657 13945 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:41.967079 13948 raft_consensus.cc:2368] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16.
I20260709 13:06:41.968003 19877 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:41.969107 13989 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:41.947980 19811 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.969964 19606 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:41.884230 19838 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:41.930994 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:41.902206 13947 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:06:41.933625 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:41.953475 19865 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.980314 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:41.960278 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:41.963301 19636 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.903523 19818 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.975710 13950 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:41.823047 19763 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:41.975785 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:41.977538 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:41.978603 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:41.940027 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:41.981053 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71620.3us
I20260709 13:06:41.991175 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:41.992556 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:41.989960 13946 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:41.994438 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:41.995095 14020 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 10.
I20260709 13:06:41.996696 14015 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:41.993193 14017 raft_consensus.cc:1813] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Handling vote request from an unknown peer 3779649036364744823fc035dac892c7
I20260709 13:06:41.999913 14017 raft_consensus.cc:2368] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 19. Current term is 21.
W20260709 13:06:41.999965 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 76887.2us
I20260709 13:06:42.001390 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:42.003232 13989 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:42.008500 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:42.010607 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:42.013849 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:42.015853 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:42.016207 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.016335 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:42.019011 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 88171.4us
I20260709 13:06:42.005873 19668 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:42.007588 19628 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:42.026290 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 48840.3us
I20260709 13:06:41.979715 19534 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:42.027748 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:41.988513 19895 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:42.006847 19888 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:41.988875 19910 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:42.029848 19882 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:42.042991 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:42.043601 14017 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:42.045991 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.031803 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:41.986179 19835 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:06:41.995473 19606 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:42.014750 19877 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:42.035668 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:42.036083 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:41.968238 19878 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.011224 19503 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:42.038201 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:42.037212 19489 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.046860 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.027755 19566 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:42.071233 19566 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.071373 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.071522 14024 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:06:42.071750 19566 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:42.071983 14024 pending_rounds.cc:85] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 15
I20260709 13:06:42.072273 14024 pending_rounds.cc:107] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:06:42.072515 14024 raft_consensus.cc:2889] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:42.072893 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:42.028461 19814 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:42.033560 19821 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:42.011471 19328 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:06:41.988298 19818 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:42.061014 13989 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:42.074210 19920 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:42.086565 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:42.051209 19884 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:42.092238 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.038178 19666 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:06:42.068416 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.075491 19766 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:42.073817 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.074281 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:42.076830 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.080741 14020 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:42.080958 19888 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:42.030328 19844 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:42.057117 19606 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.062891 14023 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:42.107535 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.108479 13873 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:42.033957 19321 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:06:42.068096 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:42.052583 19881 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:42.096441 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:42.096751 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:42.047274 19534 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.131248 19534 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:42.098551 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.052024 19595 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.133451 19595 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:42.092029 19821 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:42.077188 19901 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:42.101092 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:42.134703 19821 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:42.135591 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:42.135821 19901 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:42.136404 19901 raft_consensus.cc:697] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:42.104799 14022 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:42.138949 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:42.137365 19901 consensus_queue.cc:237] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.139829 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:42.140491 14021 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:42.140529 14022 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:06:42.084172 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.080260 19489 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:42.142071 19903 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:42.143553 19903 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:42.144461 19489 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:42.112154 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.100067 19628 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:42.074753 19603 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:06:42.145648 19628 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.148461 19636 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:42.051506 19895 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:42.149644 19628 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:42.150789 19895 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:42.097766 19848 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:42.154565 19848 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.154999 19848 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:42.117151 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:42.159812 19606 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:42.105525 19814 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:42.111203 19888 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:42.162760 19888 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:42.162642 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:42.165068 13875 pending_rounds.cc:85] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 12
I20260709 13:06:42.165490 13875 pending_rounds.cc:107] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:06:42.165750 13875 raft_consensus.cc:2889] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:42.166846 19881 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:42.167335 19878 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:42.168267 13950 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:42.117594 14023 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:42.170430 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.171581 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.105109 19668 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:06:42.172428 13915 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:42.184129 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:42.122260 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:42.185499 14015 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:42.090222 19877 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:42.186466 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:42.186542 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:42.185617 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:42.128695 19818 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:42.129168 19923 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:06:42.121579 19865 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:42.172628 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:42.204650 13868 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:42.197088 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:42.205952 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:42.208230 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:42.175060 13869 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:06:42.180680 14023 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:42.184962 19814 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:42.189421 13871 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:42.197924 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2770) took 1647 ms. Trace:
I20260709 13:06:42.139755 19779 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:42.212204 13949 rpcz_store.cc:276] 0709 13:06:40.550382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.550875 (+   493us) service_pool.cc:224] Handling call
0709 13:06:40.551150 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:40.922787 (+371637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:40.922819 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:40.922859 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:40.922892 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.197652 (+1274760us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b5400087854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:42.197709 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.197854 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854151424}
I20260709 13:06:42.229148 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:42.212430 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:42.187480 19910 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:06:42.240850 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:42.218902 14015 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:42.241739 14023 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:42.252926 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:42.256946 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:42.263388 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3031) took 1964 ms. Trace:
I20260709 13:06:42.263988 14018 rpcz_store.cc:276] 0709 13:06:40.299508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.299675 (+   167us) service_pool.cc:224] Handling call
0709 13:06:40.300055 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:40.336880 (+ 36825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:40.336918 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:40.337446 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:06:40.970497 (+633051us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:40.970536 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.235538 (+1265002us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b540023cb54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:42.235598 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:42.235648 (+    50us) raft_consensus.cc:1685] finished
0709 13:06:42.235663 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.263299 (+ 27636us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84073728,"thread_start_us":632689,"threads_started":1,"wal-append.queue_time_us":637389,"wal-append.run_cpu_time_us":1990,"wal-append.run_wall_time_us":1004960}
I20260709 13:06:42.227020 19321 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:42.266955 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:42.234513 19595 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:42.279500 14556 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:06:42.270601 13873 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:42.269033 19908 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:42.231952 19878 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:06:42.298465 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.292376 14021 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:42.301223 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:42.302173 14021 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:42.302189 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:42.293433 14023 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:42.305579 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.307631 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.308669 14015 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:06:42.266209 19903 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:06:42.297578 19908 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.284127 19595 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:42.307006 14023 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:42.312959 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.317157 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.317951 14015 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:42.319512 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.320565 14023 raft_consensus.cc:1813] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Handling vote request from an unknown peer e1649107c781428692458f10597c5742
I20260709 13:06:42.321959 14023 raft_consensus.cc:2410] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:06:42.322700 13915 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:42.314206 19908 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:42.328858 13915 leader_election.cc:400] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 19. Current term is 21.
I20260709 13:06:42.331624 13987 leader_election.cc:400] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16.
I20260709 13:06:42.332055 13987 leader_election.cc:403] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:06:42.333787 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:42.334185 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:42.321003 19668 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:42.359092 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:42.349102 14023 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:06:42.317631 19341 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:42.337828 19903 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:42.373466 14023 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:06:42.364437 19835 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:42.344599 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:42.358964 19901 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:06:42.376443 19595 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:42.386816 19934 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 13:06:42.388221 19935 log.cc:927] Time spent T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:06:42.389380 19937 log.cc:927] Time spent T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:06:42.391865 19341 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:42.364462 19668 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.394593 19838 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:42.398168 14019 raft_consensus.cc:2368] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12.
I20260709 13:06:42.397298 19838 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:42.358320 19328 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:06:42.399955 19328 raft_consensus.cc:3055] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:42.400350 19328 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:42.401408 19328 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:42.333542 19910 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:42.379164 14024 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:42.351516 14021 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:42.379895 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:42.365971 19843 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 13:06:42.430368 19341 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:42.335012 19603 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 13:06:42.414163 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 69691us
I20260709 13:06:42.432222 19603 consensus_queue.cc:1243] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:42.424585 19848 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.378701 19946 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:42.426384 19838 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.418257 19910 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:42.434342 19910 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:42.429698 14021 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:42.419301 19884 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16.
I20260709 13:06:42.436620 19843 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:42.408476 19668 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:42.437448 19843 raft_consensus.cc:697] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:42.437760 19328 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:42.438588 19843 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:42.443653 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 63837.1us
I20260709 13:06:42.406154 19566 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.458302 19835 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:42.468082 19881 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:42.467639 19838 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:42.471243 19628 raft_consensus.cc:3055] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:42.471752 19628 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:42.472760 19628 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:42.478720 19914 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.620s	user 0.000s	sys 0.003s
I20260709 13:06:42.483727 19882 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:42.484385 19882 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.404228 19867 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:42.486017 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.487811 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:42.489135 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.490495 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.492412 13942 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:06:42.492815 13942 pending_rounds.cc:85] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 9
I20260709 13:06:42.493052 13942 pending_rounds.cc:107] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:06:42.493297 13942 raft_consensus.cc:2889] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:42.468619 19881 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:42.536346 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.579239 13945 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:42.589107 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:42.590158 13944 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:42.603706 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.461560 19946 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.613948 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:42.614712 14021 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:42.574834 19835 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 13:06:42.613195 19884 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:42.613291 19946 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:42.667873 19848 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:42.717149 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:42.717810 14022 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:42.722579 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:42.723023 19913 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.894s	user 0.000s	sys 0.006s
I20260709 13:06:42.724090 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.725260 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:42.725710 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.726461 13945 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:42.729489 13945 pending_rounds.cc:85] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 7
I20260709 13:06:42.729792 13945 pending_rounds.cc:107] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:06:42.713725 19566 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:42.730432 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.731962 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.732990 19882 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:42.734342 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.734588 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:42.735013 14018 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:42.737145 13945 raft_consensus.cc:2889] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:42.725840 13948 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:06:42.707988 19888 consensus_meta.cc:287] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.003s
I20260709 13:06:42.744854 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.739298 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.747843 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.707688 19636 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.712510 14021 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:42.753912 14022 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:42.755266 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.755399 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.758896 14022 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 17: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:42.757840 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:42.759795 14018 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:42.761533 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:42.762938 13948 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:42.738013 19811 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:42.696784 19884 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.629971 19881 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.764904 14016 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:42.739106 19866 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:42.753608 19888 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.795437 14016 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:06:42.763203 19636 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:42.784112 19887 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.787328 19838 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:42.811591 14021 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: [15.11-16.12]
I20260709 13:06:42.806136 19889 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.784557 19866 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.828094 19321 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:42.807237 19887 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:42.892634 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.802640 19908 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.884349 19321 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:42.909773 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:42.912990 19951 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:06:42.913859 19953 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:42.920876 13866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3097) took 1669 ms (client timeout 1720 ms). Trace:
W20260709 13:06:42.925921 13866 rpcz_store.cc:269] 0709 13:06:41.251532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.251657 (+   125us) service_pool.cc:224] Handling call
0709 13:06:42.920619 (+1668962us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b54000da054. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:42.920793 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1178238720}
I20260709 13:06:42.883561 19889 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:42.928876 13944 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:42.941035 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.932760 19953 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:42.917951 19954 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:42.948473 13950 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 12: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:42.953003 19916 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.088s	user 0.002s	sys 0.004s
I20260709 13:06:42.956670 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:42.957279 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:42.957482 13948 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:06:42.960531 13866 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:42.970805 13872 raft_consensus.cc:606] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:42.971524 13872 raft_consensus.cc:614] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:42.972318 13874 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:42.953094 19953 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:42.988281 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:42.989007 13873 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:42.958936 19954 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:43.005805 19954 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:43.007258 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:43.016255 19944 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.394s	user 0.002s	sys 0.000s
I20260709 13:06:43.019098 19897 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:43.019645 19897 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:43.020589 19897 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:43.021699 19897 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:43.024592 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:43.029352 14522 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:43.030292 19811 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:43.030709 19811 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.031057 19811 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:43.031204 13989 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.720s (SENT) [suppressed 15 similar messages]
I20260709 13:06:43.033278 19867 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:43.036310 19889 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:43.037418 19866 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:43.038211 19949 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:06:43.038774 19908 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:43.039880 19887 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:42.980301 19884 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:43.047283 19948 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 13:06:43.048662 19628 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:42.985503 19636 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:43.059250 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.066872 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:43.075467 13873 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:43.093101 13950 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:43.108528 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:43.112308 13867 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:06:43.113231 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:43.118948 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:43.119884 13873 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:43.121040 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:43.123725 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:43.124562 14558 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:43.125638 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:43.126581 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:43.069911 19636 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.142590 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:43.144845 14554 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:06:43.147636 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:43.103884 19874 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.103896 19904 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.164671 14559 raft_consensus.cc:2393] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:43.069725 19884 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:43.132030 19910 consensus_meta.cc:287] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.697s	user 0.007s	sys 0.007s
I20260709 13:06:43.178645 19910 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:43.179754 19908 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:06:43.151919 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.182986 13941 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:43.160974 13868 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:06:43.186125 13867 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:43.064489 19866 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:06:43.141208 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:43.167850 19904 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:43.076009 19782 consensus_queue.cc:1059] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:06:43.201660 19782 raft_consensus.cc:3055] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:43.171667 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:43.145097 19328 consensus_meta.cc:287] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.701s	user 0.011s	sys 0.002s
I20260709 13:06:43.066075 19867 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:06:43.218189 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:43.264397 14559 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:43.261523 13839 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:43.278087 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:43.207442 19782 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:43.259974 19956 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:06:43.291548 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:43.270284 19866 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:43.159862 19636 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:43.304024 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:43.315809 13873 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:43.269317 19910 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:43.204440 19874 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:43.276823 19904 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:43.332671 13868 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:43.333606 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:43.334436 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 303241us
I20260709 13:06:43.335602 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:43.345384 13949 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.013s	sys 0.017s
W20260709 13:06:43.345861 13949 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.013s	sys 0.017s
I20260709 13:06:43.346009 19874 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:43.348459 19763 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:43.300629 19888 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:43.350675 19888 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:43.351153 19888 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:43.352030 19888 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:43.363878 13873 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:43.355397 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:43.286020 19821 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 13:06:43.310158 19866 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:43.349951 19763 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:43.367452 19946 consensus_meta.cc:287] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.660s	user 0.010s	sys 0.004s
I20260709 13:06:43.380923 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:43.387341 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 223200us
I20260709 13:06:43.392412 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:43.417366 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:43.425256 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:43.428794 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 184256us
I20260709 13:06:43.393530 19866 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:43.400580 19884 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:43.369760 19782 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:43.454694 14558 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:43.451165 19824 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:43.462949 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:43.438486 19882 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.400192 19668 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:06:43.463728 14524 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:43.470368 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:43.476259 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:43.477385 19957 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:43.401446 19763 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:43.483076 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 264983us
I20260709 13:06:43.449661 19884 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:43.499375 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3034) took 1496 ms. Trace:
I20260709 13:06:43.500033 14019 rpcz_store.cc:276] 0709 13:06:41.992436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.992582 (+   146us) service_pool.cc:224] Handling call
0709 13:06:43.488203 (+1495621us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54003bbfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:43.488378 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875982592}
W20260709 13:06:43.508390 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.509863 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:43.511439 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:43.513473 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:43.514732 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2800) took 1011 ms. Trace:
I20260709 13:06:43.515281 13949 rpcz_store.cc:276] 0709 13:06:42.503017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.504834 (+  1817us) service_pool.cc:224] Handling call
0709 13:06:42.505195 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:42.505436 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:42.593002 (+ 87566us) log.cc:844] Serialized 14 byte log entry
0709 13:06:43.514258 (+921256us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.514334 (+    76us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:43.514391 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.514484 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.514647 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":87517,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":25625,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":920768,"threads_started":2,"wal-append.queue_time_us":401845}
I20260709 13:06:43.519430 13946 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
W20260709 13:06:43.525419 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.492004 19884 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:43.538503 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:43.549546 13868 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:43.549816 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:43.552619 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:43.507899 19874 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:43.483489 19882 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:43.486472 19957 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
W20260709 13:06:43.532123 19960 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.247s	user 0.001s	sys 0.001s
I20260709 13:06:43.557782 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:06:43.526072 19843 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.025s	sys 0.009s
I20260709 13:06:43.559252 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:43.558691 13840 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:43.501169 19814 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:06:43.508435 19904 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:06:43.564851 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:43.548099 19774 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:43.565331 19843 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.025s	sys 0.009s
I20260709 13:06:43.560897 19910 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.556217 19982 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:43.585285 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:43.533085 19811 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:43.567308 19824 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:43.569070 19821 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:43.404192 19946 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:43.575332 19776 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:43.571805 19978 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:43.583050 19636 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:43.587257 19668 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.626892 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:43.625000 14552 raft_consensus.cc:2368] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 14. Current term is 15.
I20260709 13:06:43.635334 14551 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:43.690919 19970 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 13:06:43.694656 19966 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:06:43.696237 19811 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:43.698529 19814 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:43.699688 19814 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:43.700330 19814 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:43.704186 19763 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:43.705137 19763 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:43.705574 19763 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.593346 19982 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:43.587659 19983 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:43.583863 19774 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:43.712855 13868 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
W20260709 13:06:43.698699 19971 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:06:43.597821 19975 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:43.625402 19818 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:43.759253 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:43.762065 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:43.590480 19910 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:43.705984 19763 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:43.648558 19821 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:43.771580 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:43.774762 14016 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:43.774682 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:43.849328 14023 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:43.850605 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:43.834972 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:43.852124 14022 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 17: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:43.761750 19972 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
W20260709 13:06:43.907742 14020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 681) took 1864 ms (client timeout 1771 ms). Trace:
W20260709 13:06:43.908545 14020 rpcz_store.cc:269] 0709 13:06:42.043357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.053880 (+ 10523us) service_pool.cc:224] Handling call
0709 13:06:43.907464 (+1853584us) spinlock_profiling.cc:243] Waited 8.61 ms on lock 0x7b540023a354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:43.907622 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19797888}
W20260709 13:06:43.921422 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 413010us
I20260709 13:06:43.955319 14015 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:43.964270 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57836 (request call id 23) took 2942 ms. Trace:
I20260709 13:06:43.973330 13872 rpcz_store.cc:276] 0709 13:06:41.022374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.022622 (+   248us) service_pool.cc:224] Handling call
0709 13:06:43.963900 (+2941278us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b54000e9a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c342d853 00007fb2c83fb60a 00007fb2c19817d4 00007fb2c1981762 00007fb2c1981691 00007fb2c198160c 00007fb2c19808c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:43.964109 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4293922688}
I20260709 13:06:43.975095 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:43.967551 19881 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:43.983688 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:43.984525 14016 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:43.984731 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 56595.7us
I20260709 13:06:43.650682 19668 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:43.631731 19776 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:43.996093 14019 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:43.997148 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:44.000538 14020 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:06:44.007571 13988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.000s	sys 0.016s
I20260709 13:06:43.877549 19821 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:44.014859 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:44.005064 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:44.008003 13988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.000s	sys 0.016s
I20260709 13:06:44.007627 14023 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:44.016858 14021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.029s	sys 0.028s
W20260709 13:06:44.017228 14021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.029s	sys 0.028s
I20260709 13:06:44.018895 13869 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:06:44.003274 13916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.000s	sys 0.010s
I20260709 13:06:44.000824 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:44.022306 14022 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
W20260709 13:06:44.026499 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 441267us
I20260709 13:06:44.028347 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:44.003374 19811 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.806861 19975 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:44.020572 14024 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
W20260709 13:06:44.030738 19354 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.000s	sys 0.014s
W20260709 13:06:44.032238 13916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.000s	sys 0.010s
I20260709 13:06:44.038072 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:44.037313 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 512718us
W20260709 13:06:43.984800 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 427130us
I20260709 13:06:44.014995 19821 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:44.017951 19628 consensus_meta.cc:287] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.958s	user 0.015s	sys 0.003s
I20260709 13:06:44.054708 13838 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:44.051487 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:44.055266 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:44.049744 14016 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:44.017297 19328 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:44.055989 13869 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
W20260709 13:06:44.069494 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.071533 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:44.084007 19996 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.083734 19975 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.084651 19996 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.040241 19982 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:44.039208 19882 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.086134 19328 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.086805 19882 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:44.087319 19882 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.088598 19328 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:44.089385 19988 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:44.090234 19988 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:44.091193 19988 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:44.095119 19897 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.036s	user 0.008s	sys 0.008s
I20260709 13:06:44.095772 13869 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:44.096534 19321 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.180s	user 0.012s	sys 0.004s
I20260709 13:06:44.096947 19321 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.101069 19321 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:44.088616 19776 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:44.102200 14016 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:06:44.103096 19998 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:44.103641 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:44.103734 19998 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.102180 19818 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:44.105674 19341 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:44.106334 19628 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.106269 19341 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.106877 19628 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.107986 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:44.108433 19341 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.108981 19628 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:44.109166 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:44.109555 19782 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.109820 14016 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
I20260709 13:06:44.110570 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:44.111361 14020 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 12: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:44.112713 19989 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:06:44.120827 19668 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.120496 19821 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:44.123059 19993 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:06:44.127954 19994 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:06:44.129554 19990 consensus_queue.cc:1059] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:06:44.131031 19910 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:44.131256 19814 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:44.056643 19354 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.000s	sys 0.015s
W20260709 13:06:44.143311 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71899.9us
I20260709 13:06:44.081854 14023 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:44.103543 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:44.149128 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:44.066917 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:44.037155 19953 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.017s	sys 0.026s
I20260709 13:06:44.089463 19882 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.151086 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:44.132263 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 844) took 1063 ms. Trace:
W20260709 13:06:44.152992 14525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.017s	sys 0.028s
I20260709 13:06:44.131040 19990 raft_consensus.cc:3055] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:06:44.155917 14525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.017s	sys 0.028s
I20260709 13:06:44.155578 13875 rpcz_store.cc:276] 0709 13:06:43.069337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.069494 (+   157us) service_pool.cc:224] Handling call
0709 13:06:43.122606 (+ 53112us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:43.123016 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:43.388519 (+265503us) log.cc:844] Serialized 14 byte log entry
0709 13:06:43.582633 (+194114us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:43.583291 (+   658us) log.cc:844] Serialized 26 byte log entry
0709 13:06:43.583469 (+   178us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:43.583515 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.583608 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:44.132024 (+548416us) raft_consensus.cc:1685] finished
0709 13:06:44.132047 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.132197 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19207,"prepare.queue_time_us":432945,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":50925,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":42730,"thread_start_us":365139,"threads_started":2,"wal-append.queue_time_us":465819}
I20260709 13:06:44.159387 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:44.040848 19811 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.144924 19973 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:44.152407 19628 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:44.163661 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:44.148604 19997 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:44.154618 19953 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.017s	sys 0.027s
I20260709 13:06:44.152096 19983 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.153616 19995 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:44.151830 19998 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:44.166034 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 99248us
I20260709 13:06:44.194132 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:44.195097 14554 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:44.198012 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:44.171260 13942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.014s	sys 0.010s
I20260709 13:06:44.166674 19882 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:44.175567 19821 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:44.177356 19996 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:44.217605 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:44.178045 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:44.219381 14553 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:44.178941 19904 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:44.179890 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:44.180825 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:44.181833 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:44.183063 19946 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:44.184233 19811 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.170336 19997 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.161348 19990 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:44.186894 19776 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.187878 19341 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:44.161186 19867 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:44.192322 19668 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:44.281032 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 60162.3us
I20260709 13:06:44.220890 13839 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:44.221287 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:44.214188 19882 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:44.237138 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:44.173256 19774 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:44.199420 13942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.014s	sys 0.010s
I20260709 13:06:44.287814 14553 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
W20260709 13:06:44.192149 19838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.012s	sys 0.011s
W20260709 13:06:44.219539 19888 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.002s	sys 0.021s
I20260709 13:06:44.215445 19998 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:44.253944 19996 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:44.256345 19866 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.808s	user 0.012s	sys 0.001s
I20260709 13:06:44.271943 19628 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.274293 19776 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.278566 13914 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:44.286636 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 65849.8us
W20260709 13:06:44.291740 19838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.013s	sys 0.012s
I20260709 13:06:44.285204 19995 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:44.291548 19882 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:44.191130 19983 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.241914 19904 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:44.355723 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 134507us
I20260709 13:06:44.309046 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:44.293810 19888 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.003s	sys 0.021s
W20260709 13:06:44.231547 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.292608 19520 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:06:44.321477 19884 consensus_meta.cc:287] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.754s	user 0.019s	sys 0.000s
I20260709 13:06:44.307626 19998 raft_consensus.cc:697] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:44.291064 19763 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:44.380378 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 71452.6us
I20260709 13:06:44.383116 13945 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:44.319499 19878 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:44.305363 19996 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.394268 19884 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.313649 19628 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:44.418195 20008 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:06:44.420491 20016 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:44.422348 20009 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:06:44.422600 19904 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:44.424160 19983 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:44.373330 19973 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:44.374341 19990 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.388005 19668 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.425976 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2801) took 1358 ms. Trace:
I20260709 13:06:44.427712 19998 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.393018 19992 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.431020 19884 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.431223 19996 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:44.306489 19866 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:44.435496 19959 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.165s	user 0.001s	sys 0.008s
I20260709 13:06:44.439119 13872 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
W20260709 13:06:44.441504 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:44.441375 13874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3098) took 2346 ms (client timeout 1839 ms). Trace:
W20260709 13:06:44.491839 13874 rpcz_store.cc:269] 0709 13:06:42.095169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.095322 (+   153us) service_pool.cc:224] Handling call
0709 13:06:44.441130 (+2345808us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b54000e9a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:44.441281 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1870478080}
I20260709 13:06:44.376096 19520 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.449654 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 1092) took 1270 ms. Trace:
I20260709 13:06:44.495770 14552 rpcz_store.cc:276] 0709 13:06:43.163103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.163307 (+   204us) service_pool.cc:224] Handling call
0709 13:06:44.432796 (+1269489us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b5400530f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:44.432956 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":959107456}
I20260709 13:06:44.387595 19867 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:44.322072 19995 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.400982 19878 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:44.475739 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3051) took 2039 ms. Trace:
I20260709 13:06:44.473966 13942 rpcz_store.cc:276] 0709 13:06:43.067900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.068338 (+   438us) service_pool.cc:224] Handling call
0709 13:06:43.114807 (+ 46469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:43.301048 (+186241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.301089 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:43.301646 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:06:44.288030 (+986384us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:44.288088 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.288165 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:44.425679 (+137514us) raft_consensus.cc:1685] finished
0709 13:06:44.425710 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.425895 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":63625,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":133,"thread_start_us":1151613,"threads_started":2,"wal-append.queue_time_us":1020666}
I20260709 13:06:44.489022 19884 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:44.484732 19992 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.500394 19884 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.502502 19992 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.502650 19884 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:44.505497 19980 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.933s	user 0.003s	sys 0.000s
I20260709 13:06:44.498373 14021 rpcz_store.cc:276] 0709 13:06:42.436802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.812702 (+375900us) service_pool.cc:224] Handling call
0709 13:06:42.824374 (+ 11672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:42.824662 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:42.895526 (+ 70864us) log.cc:844] Serialized 14 byte log entry
0709 13:06:43.244146 (+348620us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b18001316e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd323a 00007fb2c343da2a
0709 13:06:44.475301 (+1231155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:44.475363 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:44.475416 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.475491 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.475643 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70222,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":6000,"raft.queue_time_us":88567,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"spinlock_wait_cycles":5958400,"thread_start_us":1230608,"threads_started":2,"wal-append.queue_time_us":1485355}
I20260709 13:06:44.519819 13869 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:44.497668 19520 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:44.530295 13988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.020s	sys 0.032s
I20260709 13:06:44.498755 19990 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:44.530948 13988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.020s	sys 0.032s
I20260709 13:06:44.459735 20016 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.485101 19668 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.518229 19878 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:44.542164 13950 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:06:44.551234 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:44.551781 13950 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:44.551885 13948 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:06:44.562168 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:44.563162 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:44.534147 19884 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:44.535377 19992 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:44.564749 14560 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:44.546838 19818 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.566401 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3195) took 1500 ms. Trace:
W20260709 13:06:44.555229 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 484681us
I20260709 13:06:44.569979 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:44.546358 19996 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:44.571941 13866 rpcz_store.cc:276] 0709 13:06:43.065727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.065856 (+   129us) service_pool.cc:224] Handling call
0709 13:06:43.066270 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:43.066614 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.066651 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:43.110043 (+ 43392us) log.cc:844] Serialized 26 byte log entry
0709 13:06:43.374690 (+264647us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:43.374753 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.374861 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:44.504719 (+1129858us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:44.559830 (+ 55111us) raft_consensus.cc:1685] finished
0709 13:06:44.559856 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.566309 (+  6453us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1103,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":36410,"spinlock_wait_cycles":5024640,"thread_start_us":205719,"threads_started":1,"wal-append.queue_time_us":370698}
I20260709 13:06:44.590339 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:44.592825 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:44.601786 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:44.604364 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:44.605028 13947 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:06:44.601738 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 288531us
I20260709 13:06:44.616683 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:44.573796 19884 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.628050 19867 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:06:44.637720 19782 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:06:44.639655 19782 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:44.642411 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:44.642886 19995 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:44.643141 13874 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:44.579854 19818 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.646314 19668 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.647090 19884 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.583597 19996 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.650450 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:44.579133 19992 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.647603 19628 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.594830 19763 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:44.578871 19983 consensus_queue.cc:1059] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 13:06:44.682973 19969 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.255s	user 0.014s	sys 0.024s
I20260709 13:06:44.691119 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:44.692111 13866 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:44.703833 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:44.705655 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:44.706492 13866 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:44.693178 13947 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:44.714869 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:44.724797 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:44.709962 19866 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:44.713838 19983 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:44.742074 19975 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:44.743072 20024 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:44.719216 20004 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:44.740548 19995 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:44.740254 19818 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:44.768985 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 162225us
I20260709 13:06:44.790129 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:44.758910 19903 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:44.764148 19975 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:44.795856 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:44.810612 13874 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:44.765378 19973 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:44.817826 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:44.773254 20024 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.823589 20033 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:44.788532 20004 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.815850 19903 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.830577 19887 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:44.772707 20023 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:44.838878 13866 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:44.838830 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:44.765049 19983 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.842201 14522 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:44.805302 19995 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:44.853786 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:44.815640 19975 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:44.816885 19992 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:44.875816 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.887614 20024 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:44.866643 20023 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.861268 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:44.861411 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:44.862640 19903 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:44.864580 19983 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:44.833331 20016 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:44.869151 13866 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:06:44.904165 13866 raft_consensus.cc:606] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
W20260709 13:06:44.870232 20021 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:06:44.871021 19995 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.899384 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:44.869757 19882 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.004s
I20260709 13:06:44.909014 13950 raft_consensus.cc:2410] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:44.915683 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1030) took 2375 ms. Trace:
I20260709 13:06:44.895025 19875 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:44.916318 13941 rpcz_store.cc:276] 0709 13:06:42.540214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.540377 (+   163us) service_pool.cc:224] Handling call
0709 13:06:42.598283 (+ 57906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.183613 (+585330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.915280 (+1731667us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b5400086954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:44.915589 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1343992064}
I20260709 13:06:44.920183 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:44.922905 13950 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:44.924146 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:44.925390 13950 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:44.891834 20023 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.889884 19776 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:44.933660 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:44.934355 13868 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 851) took 1421 ms (client timeout 1882 ms). Trace:
W20260709 13:06:44.934762 13868 rpcz_store.cc:269] 0709 13:06:43.512555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.551536 (+ 38981us) service_pool.cc:224] Handling call
0709 13:06:44.934185 (+1382649us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54002e6dd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:44.934284 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343582976}
I20260709 13:06:44.936735 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:44.937873 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:44.933307 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:44.942090 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:44.945833 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:44.856421 20033 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.947203 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:44.952806 13946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 1039) took 1552 ms (client timeout 1882 ms). Trace:
W20260709 13:06:44.954833 13946 rpcz_store.cc:269] 0709 13:06:43.391991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.392138 (+   147us) service_pool.cc:224] Handling call
0709 13:06:44.944323 (+1552185us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b5400229f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:44.944483 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277684608}
I20260709 13:06:44.956593 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:44.957489 13946 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:44.930738 19875 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:44.977339 13874 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:44.920665 19882 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.951037 20016 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:44.953843 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 850) took 1441 ms. Trace:
I20260709 13:06:44.989878 19992 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:44.990693 19992 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.993348 19992 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.929353 19973 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:44.995126 13948 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:44.995250 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:44.995595 13868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:06:44.996013 13944 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:44.996155 13868 raft_consensus.cc:606] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:06:44.996542 13868 raft_consensus.cc:614] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:44.989069 13871 rpcz_store.cc:276] 0709 13:06:43.512202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.067298 (+555096us) service_pool.cc:224] Handling call
0709 13:06:44.073279 (+  5981us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:44.126077 (+ 52798us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:44.126116 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:44.126708 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:06:44.545117 (+418409us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:44.951060 (+405943us) log.cc:844] Serialized 14 byte log entry
0709 13:06:44.953355 (+  2295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.953498 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:44.953559 (+    61us) raft_consensus.cc:1685] finished
0709 13:06:44.953577 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.953766 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76151,"raft.queue_time_us":57477,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":215,"thread_start_us":808875,"threads_started":2,"wal-append.queue_time_us":611805}
I20260709 13:06:44.994704 19875 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.003839 20004 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:45.007184 19878 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:44.982322 19874 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:45.015365 19997 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:45.016347 20033 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:45.019503 19882 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:44.994820 20016 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.021870 20016 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:45.039923 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1089) took 1935 ms. Trace:
I20260709 13:06:45.040637 14551 rpcz_store.cc:276] 0709 13:06:43.104495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.104638 (+   143us) service_pool.cc:224] Handling call
0709 13:06:43.107331 (+  2693us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.667709 (+560378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:45.039467 (+1371758us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400750154. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:45.039817 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1213559552}
I20260709 13:06:44.998564 19992 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:44.977926 19520 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.061998 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 691) took 1295 ms. Trace:
I20260709 13:06:45.047269 19878 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:45.066897 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3054) took 1311 ms. Trace:
I20260709 13:06:45.066766 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:06:45.068003 14024 rpcz_store.cc:276] 0709 13:06:43.755524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.755670 (+   146us) service_pool.cc:224] Handling call
0709 13:06:45.066690 (+1311020us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400299fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:45.066816 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599047040}
I20260709 13:06:45.071630 19986 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:45.054093 19838 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.063777 14015 rpcz_store.cc:276] 0709 13:06:43.766817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.978407 (+211590us) service_pool.cc:224] Handling call
0709 13:06:43.978919 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:44.215542 (+236623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:44.215634 (+    92us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:44.216339 (+   705us) log.cc:844] Serialized 26 byte log entry
0709 13:06:44.584686 (+368347us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:06:45.046305 (+461619us) log.cc:844] Serialized 14 byte log entry
0709 13:06:45.061639 (+ 15334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:45.061730 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:45.061769 (+    39us) raft_consensus.cc:1685] finished
0709 13:06:45.061785 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:45.061933 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20618,"prepare.queue_time_us":164326,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":72,"raft.queue_time_us":126643,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"thread_start_us":937060,"threads_started":3,"wal-append.queue_time_us":539058}
W20260709 13:06:45.075910 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.072130 19986 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:45.084297 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:45.084462 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:45.086613 14017 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:45.087311 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:45.078756 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 265304us
I20260709 13:06:45.087924 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:45.088579 14016 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:45.099035 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:45.123467 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:45.126519 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:45.126978 14017 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:45.127205 14016 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:45.076264 19997 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.118947 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:45.135453 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 149378us
I20260709 13:06:45.147641 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:45.149139 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:45.150022 19874 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:45.150130 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:45.150610 19874 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.151088 19874 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:45.140185 20035 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:45.151952 20035 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.077312 19998 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:45.177420 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:45.178934 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:45.067011 19992 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.194695 14024 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:45.192659 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:45.132030 19668 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.148132 19628 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:45.203045 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:45.145080 19884 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:45.231496 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:45.231839 14016 raft_consensus.cc:2410] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:06:45.232218 14017 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:45.180465 19957 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:45.078459 19838 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.236025 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:45.246080 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.249318 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:45.249974 14020 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:45.202831 20004 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:45.189890 20000 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:06:45.258728 13870 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:45.233726 19628 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:45.236207 19983 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:45.187796 19997 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.234989 19884 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:45.254815 20004 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.200345 19998 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.336737 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:45.237229 19668 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.316671 19957 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:45.321959 19628 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:45.327841 19884 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:45.233181 19986 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.336771 20004 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:45.332458 19983 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:45.367277 13942 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:06:45.372454 13914 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:45.330107 19975 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.377909 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:45.365594 20046 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.277s	user 0.001s	sys 0.001s
I20260709 13:06:45.382357 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:45.388283 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:45.394238 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:45.398082 13914 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:45.398655 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:45.398977 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 247553us
I20260709 13:06:45.380208 19986 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:45.404861 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:45.407034 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 693) took 1375 ms. Trace:
I20260709 13:06:45.407557 14022 rpcz_store.cc:276] 0709 13:06:44.033864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.039845 (+  5981us) service_pool.cc:224] Handling call
0709 13:06:44.120219 (+ 80374us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:44.120719 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:44.120750 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:44.830573 (+709823us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b2c000e8788. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:06:44.831368 (+   795us) log.cc:844] Serialized 78 byte log entry
0709 13:06:45.120853 (+289485us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:45.190524 (+ 69671us) log.cc:844] Serialized 14 byte log entry
0709 13:06:45.190792 (+   268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:45.190872 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:45.406644 (+215772us) raft_consensus.cc:1685] finished
0709 13:06:45.406777 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:45.406961 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":80210,"raft.queue_time_us":4098,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":8117,"spinlock_wait_cycles":40491648,"thread_start_us":288927,"threads_started":1,"wal-append.queue_time_us":520862}
W20260709 13:06:45.392947 20039 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 13:06:45.410455 14558 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:45.411178 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 692) took 1644 ms. Trace:
I20260709 13:06:45.411697 14019 rpcz_store.cc:276] 0709 13:06:43.767099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.015995 (+248896us) service_pool.cc:224] Handling call
0709 13:06:44.032959 (+ 16964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:44.535794 (+502835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:44.535832 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:44.545068 (+  9236us) log.cc:844] Serialized 26 byte log entry
0709 13:06:44.905723 (+360655us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:44.905773 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.907031 (+  1258us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:45.410966 (+503935us) raft_consensus.cc:1685] finished
0709 13:06:45.410985 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:45.411120 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4704,"raft.queue_time_us":75705,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":857867,"threads_started":2,"wal-append.queue_time_us":529294}
W20260709 13:06:45.397087 20049 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.065s	user 0.001s	sys 0.001s
W20260709 13:06:45.360853 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 130955us
I20260709 13:06:45.416065 14019 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:45.409868 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:45.419301 14022 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:45.419358 13914 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:45.381178 20052 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:45.424042 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:45.424427 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:45.425115 14020 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:45.437654 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:45.438042 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:45.438848 14020 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:45.396274 19992 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:45.442756 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:45.446561 13870 raft_consensus.cc:2368] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17.
I20260709 13:06:45.447917 19838 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:45.448518 20045 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 13:06:45.448999 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:45.443902 13873 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:45.443938 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:45.424114 19595 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:45.451532 14019 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:45.451478 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:45.387034 19878 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:06:45.454347 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:45.457649 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.464602 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:45.430662 20052 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:45.467231 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:45.478796 14022 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:06:45.469866 14020 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:45.434916 19882 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.480659 19882 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.414542 20000 consensus_queue.cc:1059] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:06:45.445338 19905 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:45.484880 19776 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:45.514411 20058 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:06:45.491040 20052 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.515599 19957 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:45.511826 20064 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:45.441028 19983 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.521358 20000 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:45.548043 13944 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:45.525712 19905 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:45.553064 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 131087us
I20260709 13:06:45.533294 19998 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:45.578734 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 114857us
I20260709 13:06:45.545258 19997 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:45.545928 20000 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:45.582391 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 121727us
I20260709 13:06:45.524523 20059 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:06:45.554266 19668 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:45.596746 19838 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:45.597294 20061 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:06:45.594782 20078 raft_consensus.cc:3055] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:45.619777 14551 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:06:45.620327 20063 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:06:45.621171 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:45.618999 20078 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:45.624831 20078 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.626317 20078 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.626530 19905 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.628633 19595 raft_consensus.cc:3055] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:45.629097 19595 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:45.632598 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:45.634191 20073 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:45.634876 20073 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.609908 20000 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:45.638758 13989 leader_election.cc:336] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.720s (SENT)
I20260709 13:06:45.637286 19838 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.649315 19838 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:45.651578 20016 consensus_meta.cc:287] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.603s	user 0.007s	sys 0.005s
W20260709 13:06:45.652609 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.651023 20052 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.664692 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:45.665154 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:45.616402 19998 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.614166 19874 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.653100 19992 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:45.657367 20042 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:45.659974 20075 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:45.652756 19997 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:06:45.683223 13945 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:45.667586 19975 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:45.668768 20069 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:45.652091 20016 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.691330 14558 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:45.652484 19595 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.673234 19977 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:06:45.703871 14015 raft_consensus.cc:2368] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12.
I20260709 13:06:45.682484 20010 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:45.674300 19998 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.681798 20075 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:45.680938 19992 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:45.715945 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 1.34393e+06us
I20260709 13:06:45.716771 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:45.703073 19996 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:45.696517 19881 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
W20260709 13:06:45.723135 20072 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:06:45.724713 20004 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.727183 20004 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:45.728755 19668 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:45.729615 19998 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:45.681488 20042 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:45.730695 20083 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:06:45.734040 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:45.717985 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:45.709244 20075 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:45.739038 19818 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:45.740568 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:45.743146 20042 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:45.743805 19818 raft_consensus.cc:2764] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:06:45.750342 19988 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:45.750902 19988 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:45.751300 19988 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:45.710711 20010 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.756518 20010 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:45.757349 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:45.758147 13873 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:45.759071 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:45.729379 19668 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.765996 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.701073 19884 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:45.768318 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 101261us
I20260709 13:06:45.769075 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:45.772811 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:45.773098 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:45.774466 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.775125 13947 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:45.775718 13915 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:45.776744 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:45.697093 20069 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.779281 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.778635 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:45.782848 13947 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:45.779991 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:45.774806 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.786026 19824 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:45.786672 19824 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:45.787592 19824 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.789211 19874 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:45.789803 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3114) took 2244 ms. Trace:
I20260709 13:06:45.780706 19983 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:45.790604 19983 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:45.790253 13869 rpcz_store.cc:276] 0709 13:06:43.551715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.096590 (+544875us) service_pool.cc:224] Handling call
0709 13:06:44.096909 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.520314 (+423405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:45.789556 (+1269242us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b54004510d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:45.789742 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":971970816}
I20260709 13:06:45.780936 20051 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:45.792148 20069 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:45.770546 19905 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:45.791211 19983 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:45.797344 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.799263 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:45.710829 19628 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.799263 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.807631 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.807631 14024 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:45.802364 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.766791 20010 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:45.804773 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.779317 19998 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:45.780570 19995 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:45.811308 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.780629 19996 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:45.809912 13943 raft_consensus.cc:2410] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:06:45.809387 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:45.816742 13872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3113) took 2271 ms (client timeout 2231 ms). Trace:
I20260709 13:06:45.815855 19884 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:45.817510 13868 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:45.820137 13874 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:06:45.823220 13872 rpcz_store.cc:269] 0709 13:06:43.545584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.974745 (+429161us) service_pool.cc:224] Handling call
0709 13:06:45.816548 (+1841803us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b540026c5d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:45.816668 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1065193216}
I20260709 13:06:45.771706 20042 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 13:06:45.848088 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.852632 14023 raft_consensus.cc:2393] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 17: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:45.796501 19997 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:06:45.939139 19997 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:45.939635 19997 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:06:45.940332 20088 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:06:45.940653 19997 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:45.941959 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 125083us
I20260709 13:06:45.825647 20010 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:45.806468 20051 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:45.785190 19818 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:06:45.828707 19341 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:45.958005 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 153691us
I20260709 13:06:45.973374 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:45.976198 13945 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:45.812635 19986 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:06:45.978938 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 173378us
I20260709 13:06:45.831779 20041 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.160s
I20260709 13:06:45.827090 19996 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.978986 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.005018 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:45.982128 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:45.983654 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:46.007592 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:45.992952 20087 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:06:45.945969 19997 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:45.813612 19905 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:45.942040 19668 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:46.012508 20051 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:46.015640 19884 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:46.016101 19884 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.016429 19884 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:45.987426 19995 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:46.025553 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:46.030864 19520 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:46.019388 20016 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:46.047989 19668 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:45.990836 20022 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:06:45.989370 19341 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:46.050489 20010 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:46.051419 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.051081 19995 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:46.051930 20075 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:46.056214 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.058521 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:46.003691 19996 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:46.065816 14024 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:06:46.020921 19986 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:46.069198 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 122895us
I20260709 13:06:46.051116 19341 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:46.070533 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:46.062348 19998 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:46.072050 19998 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:46.072430 19998 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:46.073942 19884 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:46.075068 19865 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:46.075484 19865 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:46.076371 19884 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:46.062775 20075 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:46.088929 14559 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 11.
I20260709 13:06:46.070899 19986 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:46.076329 19865 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:46.067970 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 62220us
I20260709 13:06:46.093267 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:46.100950 14024 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:46.101533 13914 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:46.103147 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 124812us
I20260709 13:06:46.106492 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:46.108299 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:46.092784 19867 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:46.090384 20041 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.122650 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:46.127624 13866 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:46.128230 14019 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:46.123306 14020 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:46.129566 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:46.129732 13873 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:46.128029 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:46.130012 14024 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:46.131287 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:46.141054 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:46.147912 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:46.140425 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:46.148548 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:46.141229 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:46.149119 13873 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:46.149897 13874 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:46.151199 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:46.151450 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:46.151726 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:46.154906 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:46.103330 19973 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:46.159582 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:46.160957 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:46.163177 19977 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:46.163805 19977 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:46.140009 19884 consensus_queue.cc:1059] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:06:46.167393 19889 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:46.167591 19884 raft_consensus.cc:3055] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:46.168077 19884 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:46.111142 19904 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:46.171494 19904 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:46.172525 19904 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:46.176985 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.180567 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:46.181137 19668 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.181775 19668 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:46.183457 19628 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:46.168023 19889 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:46.184682 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:46.184754 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:46.185850 14024 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:06:46.190265 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:46.194082 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:46.195093 13868 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:46.195595 13866 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:46.196339 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:46.197407 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:46.198586 13866 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:46.169382 19884 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:46.245016 19889 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:46.199671 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:46.118726 19865 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:46.225898 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.096391 19818 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:06:46.249490 14024 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:46.251066 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:46.251741 14024 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:46.251925 19905 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:46.201464 13874 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:46.131021 19867 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:46.194101 20041 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:46.262857 14024 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
W20260709 13:06:46.261673 13989 leader_election.cc:336] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.839s (SENT)
I20260709 13:06:46.264164 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:46.264648 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:46.265558 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:46.266680 13873 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:46.106994 20075 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:46.257193 19867 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:46.271426 20041 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:46.273030 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3070) took 1185 ms. Trace:
I20260709 13:06:46.277042 14021 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: [13.9-14.10]
I20260709 13:06:46.286685 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.273599 14023 rpcz_store.cc:276] 0709 13:06:45.087192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.087547 (+   355us) service_pool.cc:224] Handling call
0709 13:06:46.272740 (+1185193us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b54003250d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:46.272942 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1519017216}
I20260709 13:06:46.293247 13945 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:46.308058 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.308552 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.316304 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.308141 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.323939 14024 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:06:46.242949 19668 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:46.359490 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.295022 19628 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:46.347499 19988 consensus_meta.cc:287] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.596s	user 0.012s	sys 0.000s
I20260709 13:06:46.327536 19977 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:46.380113 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.262229 19998 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:46.297560 20075 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:46.405377 20093 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:06:46.418943 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.361581 19884 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:46.437640 13989 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.000s (SENT)
I20260709 13:06:46.458841 13989 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:46.439031 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:46.442673 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.443630 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:46.416774 20078 consensus_meta.cc:287] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.790s	user 0.009s	sys 0.004s
I20260709 13:06:46.403138 19988 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:46.588172 14020 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 17: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:46.446511 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:46.370250 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48351.8us
I20260709 13:06:46.589154 13945 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:46.525731 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.590703 13948 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
W20260709 13:06:46.593937 13989 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.043s (SENT)
I20260709 13:06:46.585438 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:46.522733 20016 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:06:46.631563 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 106337us
I20260709 13:06:46.630726 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.631196 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:46.637876 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:46.638792 13948 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:46.634984 13950 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:46.645243 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:46.646217 13944 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:46.648334 13915 leader_election.cc:400] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17.
I20260709 13:06:46.666455 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:46.687079 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:46.655383 20016 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:46.763875 20016 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:46.764719 20016 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.765256 20016 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:46.787710 20075 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:46.792230 19998 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:46.797230 13945 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:46.668753 20035 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:46.647558 19628 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 13:06:46.824406 13948 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:46.685504 19341 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:46.870172 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.873154 13944 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
W20260709 13:06:46.864897 19867 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.596s	user 0.013s	sys 0.000s
W20260709 13:06:46.898813 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 211812us
I20260709 13:06:46.900067 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:46.883018 20075 consensus_queue.cc:1059] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:06:46.877368 19341 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:46.925371 20065 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.398s	user 0.013s	sys 0.017s
I20260709 13:06:46.907856 19998 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:46.895053 19867 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:46.937414 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 280682us
W20260709 13:06:46.973825 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 72051.8us
W20260709 13:06:46.980515 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:46.974777 19986 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.867s	user 0.005s	sys 0.009s
I20260709 13:06:46.967262 20073 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:46.974212 19988 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:47.018008 14019 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.036s	sys 0.055s
W20260709 13:06:47.039008 14019 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.056s	sys 0.055s
I20260709 13:06:47.007987 20100 raft_consensus.cc:3055] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:47.003551 19986 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:47.012423 20000 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.353s	user 0.012s	sys 0.004s
I20260709 13:06:47.017726 19998 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:47.054070 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:47.055131 13944 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:47.039448 20078 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:47.016541 20073 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:06:47.044852 20100 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:47.065114 20100 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.018297 20105 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:06:47.073378 13944 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
W20260709 13:06:47.068852 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 126673us
I20260709 13:06:47.048604 20000 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.060776 19998 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.102669 20096 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.109232 19694 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.147769 20096 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.126567 20109 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:47.148873 20109 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.151379 20109 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.127287 20112 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:47.152508 20000 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:47.152241 20112 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.153206 20000 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:47.155625 20112 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:47.129081 20067 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 1.596s	user 0.004s	sys 0.015s
I20260709 13:06:47.156034 19998 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:47.135831 19997 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.110s	user 0.013s	sys 0.002s
I20260709 13:06:47.153769 20000 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.137075 20007 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:47.139828 19996 consensus_meta.cc:287] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.071s	user 0.012s	sys 0.002s
I20260709 13:06:47.141664 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:47.172739 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:47.143352 19986 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.144150 19341 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:47.067596 20078 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.147264 19867 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:47.147889 19694 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:47.117202 20104 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:06:47.156965 20096 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.176991 20007 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:47.178071 19983 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.349s	user 0.013s	sys 0.004s
W20260709 13:06:47.183877 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.186434 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:47.193818 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:47.196988 20016 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.200747 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:47.180241 19998 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:47.207643 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:47.209479 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:47.212630 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:47.214062 20105 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:47.213505 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:47.214506 20105 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:47.214923 20105 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:47.215758 19776 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:47.219384 20115 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:47.219657 19694 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.219990 20115 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.216818 19878 consensus_queue.cc:1059] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:06:47.222177 19878 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:47.222679 19878 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:47.223903 19878 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.211187 20004 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:47.226445 20004 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:47.226853 20004 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:47.229393 19884 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.004s
I20260709 13:06:47.206430 19983 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.232271 19983 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.182142 19996 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.237407 20000 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.238363 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:47.239724 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:47.240295 20078 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.240538 19983 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:47.241271 19983 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:47.241714 19983 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:47.242197 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3084) took 1062 ms. Trace:
I20260709 13:06:47.243991 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:47.242817 14024 rpcz_store.cc:276] 0709 13:06:46.179783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.327090 (+147307us) service_pool.cc:224] Handling call
0709 13:06:46.327499 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:46.383495 (+ 55996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:46.383546 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:46.384220 (+   674us) log.cc:844] Serialized 26 byte log entry
0709 13:06:46.888875 (+504655us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:46.888951 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.889071 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:47.241846 (+352775us) raft_consensus.cc:1685] finished
0709 13:06:47.241883 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:47.242109 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55419,"raft.queue_time_us":56532,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":10830,"thread_start_us":301812,"threads_started":1,"wal-append.queue_time_us":562936}
I20260709 13:06:47.244367 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:47.245756 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:47.248169 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:47.247975 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2551) took 1069 ms. Trace:
I20260709 13:06:47.249977 14019 rpcz_store.cc:276] 0709 13:06:46.178954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.324507 (+145553us) service_pool.cc:224] Handling call
0709 13:06:46.324909 (+   402us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:46.374462 (+ 49553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:46.374514 (+    52us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:46.404579 (+ 30065us) log.cc:844] Serialized 78 byte log entry
0709 13:06:47.039881 (+635302us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:47.185315 (+145434us) log.cc:844] Serialized 14 byte log entry
0709 13:06:47.185918 (+   603us) log.cc:844] Serialized 14 byte log entry
0709 13:06:47.191068 (+  5150us) log.cc:844] Serialized 14 byte log entry
0709 13:06:47.247616 (+ 56548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.247705 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:47.247750 (+    45us) raft_consensus.cc:1685] finished
0709 13:06:47.247764 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:47.247915 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":189996,"raft.queue_time_us":113298,"raft.run_cpu_time_us":485,"raft.run_wall_time_us":46877,"thread_start_us":762295,"threads_started":2,"wal-append.queue_time_us":683669}
I20260709 13:06:47.247494 19776 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:47.225942 19878 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:47.253181 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:47.255638 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:47.248188 19997 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:47.259509 14019 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:47.272012 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:47.250393 19341 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:47.272903 19341 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:47.273985 19341 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.257601 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:47.278988 14024 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:47.263172 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:47.261505 19776 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.273399 20004 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.263945 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:47.265646 19867 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:47.267933 20091 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:47.268893 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:47.251623 19998 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.262465 20007 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.283859 14019 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:47.250039 20121 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:47.242218 19983 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:47.292827 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 103973us
I20260709 13:06:47.285090 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:47.294309 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:47.295115 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:47.252713 20119 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:47.254957 20016 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:47.296452 13840 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:47.283469 20107 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:47.316922 13840 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:47.297518 19884 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:47.299196 19595 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:47.300258 19992 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:47.301496 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:47.296200 20113 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:47.302004 19878 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:47.303143 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:47.310882 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:47.294827 20114 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:47.320526 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:47.321436 20007 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:47.290812 20115 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.325066 14024 raft_consensus.cc:2410] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:06:47.302386 20035 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:47.326617 14021 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:47.328509 20122 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:47.296365 20121 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.340917 13947 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:47.341557 20106 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.342113 20106 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.342586 20121 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.341583 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:47.343870 14021 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:47.329092 20004 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.319274 19867 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:47.330886 19884 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:06:47.331621 14522 leader_election.cc:336] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Timed out: RequestConsensusVote RPC to 127.12.109.66:43953 timed out after 1.882s (SENT)
I20260709 13:06:47.332309 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.363479 13870 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:47.364113 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3127) took 1204 ms. Trace:
I20260709 13:06:47.332427 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:47.364390 13870 rpcz_store.cc:276] 0709 13:06:46.159877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.331962 (+1172085us) service_pool.cc:224] Handling call
0709 13:06:47.364043 (+ 32081us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:47.314891 20119 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:47.371160 20119 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:47.365372 20124 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:47.373349 20105 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.373939 19983 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.319553 19975 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:47.338865 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:47.345012 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:47.296726 19998 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.343945 20113 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:47.349974 20007 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:06:47.324561 20107 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:47.334601 20114 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:47.366608 14019 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
W20260709 13:06:47.367796 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.381043 19776 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:47.370369 20121 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:47.370491 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.416678 13870 raft_consensus.cc:2410] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:06:47.389573 14021 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:47.367856 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:47.418720 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3087) took 1229 ms. Trace:
W20260709 13:06:47.418726 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 51005.6us
W20260709 13:06:47.408093 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 84447.9us
I20260709 13:06:47.381003 19975 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:47.426614 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:47.436173 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:47.437318 14559 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 17: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:47.419234 14021 rpcz_store.cc:276] 0709 13:06:46.189756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.388609 (+1198853us) service_pool.cc:224] Handling call
0709 13:06:47.389079 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:47.418403 (+ 29324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.418650 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:47.425643 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:47.439591 14021 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:47.440754 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.441699 13870 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:47.405581 20124 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:47.439143 20007 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:06:47.453730 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 158962us
I20260709 13:06:47.415527 20129 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.414618 20114 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:47.438994 20022 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:47.446674 19595 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:47.478452 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:47.446827 20007 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:47.439368 19998 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:47.438922 19975 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:47.482843 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 71827.2us
I20260709 13:06:47.478780 20120 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:47.483879 20130 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:47.442845 19776 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:47.491499 20007 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.492946 20007 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:47.493067 19882 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:47.494297 20113 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.485898 20120 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:47.472446 20124 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.491133 20107 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.415666 19884 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.494448 19983 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.439613 20121 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:47.503834 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 53961.5us
I20260709 13:06:47.507007 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:47.508296 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3086) took 1328 ms. Trace:
I20260709 13:06:47.489701 20105 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.485122 20022 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:47.508718 14024 rpcz_store.cc:276] 0709 13:06:46.189278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.381677 (+1192399us) service_pool.cc:224] Handling call
0709 13:06:47.382291 (+   614us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:47.382876 (+   585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:47.382924 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:47.383813 (+   889us) log.cc:844] Serialized 52 byte log entry
0709 13:06:47.472136 (+ 88323us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:47.472194 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.472284 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:47.508060 (+ 35776us) raft_consensus.cc:1685] finished
0709 13:06:47.508083 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:47.508236 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5450,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":2622,"thread_start_us":87748,"threads_started":1,"wal-append.queue_time_us":99711}
I20260709 13:06:47.507616 20121 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:47.511857 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:47.510459 19983 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:47.512814 19983 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:47.512943 14024 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 12: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:47.513260 19983 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:47.492869 19776 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.513965 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:47.514576 14024 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:47.520046 20121 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:47.520713 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:47.520645 20121 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.521471 13948 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 22.
I20260709 13:06:47.524190 20121 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.524327 20022 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.509086 20120 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.528240 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:47.528882 14020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3082) took 1409 ms (client timeout 1741 ms). Trace:
W20260709 13:06:47.529328 14020 rpcz_store.cc:269] 0709 13:06:46.119977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.129358 (+  9381us) service_pool.cc:224] Handling call
0709 13:06:47.528659 (+1399301us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54001220d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:47.528820 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1052360960}
I20260709 13:06:47.531023 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:47.531473 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:47.532145 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:47.534297 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:47.534934 14020 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:06:47.535789 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:47.481731 20129 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.543489 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:47.544907 20004 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:47.555675 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:47.541148 14024 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:47.509809 19595 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:06:47.578507 19975 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.579288 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.580581 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.581020 19975 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.585244 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.585093 20114 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.587441 19665 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:47.588146 19665 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:47.590946 19665 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:47.599521 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:47.581701 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.593698 14558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 816) took 2156 ms. Trace:
I20260709 13:06:47.603226 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:47.597035 20114 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:47.607954 20129 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.608150 13871 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 17: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:47.609104 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.609824 20119 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.611845 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.629961 13872 raft_consensus.cc:2410] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:06:47.591952 20043 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:47.625038 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:47.591253 19983 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:47.629894 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:47.636008 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.623564 20091 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:48.111188 20091 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.111712 20091 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:47.639449 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.642009 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:47.643196 13839 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:41676 dropped due to backpressure: service queue is full with 50 items [suppressed 1 similar messages]
I20260709 13:06:47.644016 20119 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:47.645429 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:48.118319 14024 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:48.119251 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:48.119830 14024 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:48.120615 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:48.121119 14024 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:47.681687 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:47.736922 13869 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.581s	user 0.009s	sys 0.023s
I20260709 13:06:47.765261 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.765507 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:48.122746 14021 raft_consensus.cc:2368] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17.
I20260709 13:06:47.768699 20121 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.790462 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.121879 14023 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:06:47.797287 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.798220 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2536) took 5483 ms. Trace:
I20260709 13:06:47.797374 13841 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42526 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3149)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3173)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3167)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3157)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3143)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3168)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3142)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3178)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3254)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3170)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3179)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3251)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3176)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3151)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3150)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3180)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3169)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3171)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3161)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3174)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3164)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3163)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3162)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3175)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3146)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3240)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3148)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3147)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3156)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3247)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3242)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3239)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3172)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3154)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3152)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3140)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3160)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 867)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3238)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3243)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3244)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3245)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3246)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3252)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3253)
I20260709 13:06:47.814594 20129 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:47.814092 20043 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.127282 20129 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:47.815419 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 190335us
I20260709 13:06:48.128185 20129 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.128934 20043 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:48.115993 13839 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:41676 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3167)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3157)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3143)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3168)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3142)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3178)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3254)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3170)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3179)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3251)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3176)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3151)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3150)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3180)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3169)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3171)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3161)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3174)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3164)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3163)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3162)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3175)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3146)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3240)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3148)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3147)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3156)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3247)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3242)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3239)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3172)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3154)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3152)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3140)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3160)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 867)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3238)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3243)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3244)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3245)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3246)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3252)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3253)
W20260709 13:06:47.832841 13989 leader_election.cc:336] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.918s (SENT)
I20260709 13:06:47.871222 20106 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:47.893308 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:48.130234 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 297425us
I20260709 13:06:47.896340 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:48.130956 14016 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:48.131132 13989 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.106s (SENT) [suppressed 41 similar messages]
I20260709 13:06:47.893111 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:47.913098 14019 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:48.132176 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.914876 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:47.918308 19996 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:47.929571 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:48.133874 14022 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:47.601640 14558 rpcz_store.cc:276] 0709 13:06:45.436912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.437071 (+   159us) service_pool.cc:224] Handling call
0709 13:06:45.437444 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:45.694551 (+257107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.593260 (+1898709us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54007102d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:47.593578 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583191936}
W20260709 13:06:47.940456 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:47.952498 13950 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3113) took 1646 ms (client timeout 2131 ms). Trace:
W20260709 13:06:48.135205 13950 rpcz_store.cc:269] 0709 13:06:46.306616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.307714 (+  1098us) service_pool.cc:224] Handling call
0709 13:06:47.952235 (+1644521us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b5400088754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:47.952413 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":729062656}
W20260709 13:06:47.953136 20075 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.956s	user 0.006s	sys 0.000s
I20260709 13:06:47.966564 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:48.135921 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3138) took 1014 ms. Trace:
I20260709 13:06:48.136740 13867 rpcz_store.cc:276] 0709 13:06:47.121438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.892757 (+771319us) service_pool.cc:224] Handling call
0709 13:06:48.135861 (+243104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.136828 13948 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:47.967273 13949 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:47.967926 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:48.137969 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:47.980971 19824 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:47.981997 20007 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.128983 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:48.142364 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.142434 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:48.143776 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:47.991333 13838 connection.cc:256] RPC call timeout handler was delayed by -1.105765861000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
I20260709 13:06:47.556823 14020 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:47.994922 13942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 1049) took 2545 ms (client timeout 1882 ms). Trace:
W20260709 13:06:47.997766 13943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3107) took 2041 ms (client timeout 1835 ms). Trace:
I20260709 13:06:48.007030 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:48.019546 20052 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:48.031857 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:48.036459 20078 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:48.045202 20000 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:48.172434 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.173954 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:48.049291 14522 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:48.174914 14559 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
W20260709 13:06:48.175315 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 126082us
W20260709 13:06:48.053699 13947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3103) took 2249 ms (client timeout 1697 ms). Trace:
W20260709 13:06:48.054790 14015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3064) took 2977 ms (client timeout 2260 ms). Trace:
W20260709 13:06:48.141716 20075 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.142s	user 0.007s	sys 0.000s
I20260709 13:06:48.065757 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:48.067003 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:48.067337 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:48.067960 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:48.068620 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:48.068845 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:06:48.074658 20022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.007s	sys 0.003s
I20260709 13:06:48.077046 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3094) took 2307 ms. Trace:
I20260709 13:06:48.091516 13868 raft_consensus.cc:2376] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12. Re-sending same reply.
I20260709 13:06:48.095368 13871 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:48.100021 20100 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:48.100989 20023 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.188596 13946 rpcz_store.cc:276] 0709 13:06:45.770115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.770255 (+   140us) service_pool.cc:224] Handling call
0709 13:06:45.770825 (+   570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:46.062769 (+291944us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:46.069899 (+  7130us) log.cc:844] Serialized 14 byte log entry
0709 13:06:46.300537 (+230638us) log.cc:844] Serialized 14 byte log entry
0709 13:06:46.300812 (+   275us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:46.300881 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:46.300960 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.151231 (+850271us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b18003a8760. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:48.052053 (+900822us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b18003a8760. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:48.076776 (+ 24723us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b5400053f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:48.076831 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.076976 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7163,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":83,"raft.queue_time_us":14071,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":32845,"spinlock_wait_cycles":779356416,"thread_start_us":221788,"threads_started":1,"wal-append.queue_time_us":578770,"wal-append.run_cpu_time_us":6030,"wal-append.run_wall_time_us":1382796}
I20260709 13:06:48.092407 19983 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:48.116894 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:48.121809 13869 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.966s	user 0.010s	sys 0.023s
I20260709 13:06:48.125437 20121 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.124384 14018 rpcz_store.cc:276] 0709 13:06:42.315382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.771733 (+456351us) service_pool.cc:224] Handling call
0709 13:06:43.910803 (+1139070us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:44.351277 (+440474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:44.351346 (+    69us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:44.379062 (+ 27716us) log.cc:844] Serialized 52 byte log entry
0709 13:06:44.735791 (+356729us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:45.055925 (+320134us) log.cc:844] Serialized 14 byte log entry
0709 13:06:45.061203 (+  5278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:45.061295 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:45.061340 (+    45us) raft_consensus.cc:1685] finished
0709 13:06:45.061352 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:47.798000 (+2736648us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540023a350. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:47.798157 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5776,"prepare.queue_time_us":86329,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":203350,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":5469,"spinlock_wait_cycles":2584992000,"thread_start_us":918556,"threads_started":3,"wal-append.queue_time_us":409610,"wal-append.run_cpu_time_us":2796,"wal-append.run_wall_time_us":1238900}
I20260709 13:06:48.126284 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.213683 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.214564 13874 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:48.215737 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:48.217129 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:48.122834 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3139) took 1001 ms. Trace:
I20260709 13:06:48.218938 13875 rpcz_store.cc:276] 0709 13:06:47.121756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.764834 (+643078us) service_pool.cc:224] Handling call
0709 13:06:48.122775 (+357941us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.126902 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3134) took 1007 ms. Trace:
I20260709 13:06:48.220014 13873 rpcz_store.cc:276] 0709 13:06:47.119998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.789969 (+669971us) service_pool.cc:224] Handling call
0709 13:06:48.126835 (+336866us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:47.826923 20124 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:48.129171 19988 consensus_queue.cc:1059] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:47.927980 20101 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:48.139003 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:48.145937 13942 rpcz_store.cc:269] 0709 13:06:45.449632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.450406 (+   774us) service_pool.cc:224] Handling call
0709 13:06:47.994540 (+2544134us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b5400229f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:47.994728 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557777152}
I20260709 13:06:48.148226 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:48.231583 13948 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:06:48.150344 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:48.153896 14016 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:48.234033 14016 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:48.155655 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:48.156991 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:48.146209 13943 rpcz_store.cc:269] 0709 13:06:45.956070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.957357 (+  1287us) service_pool.cc:224] Handling call
0709 13:06:47.997539 (+2040182us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54000f4e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:47.997677 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770981632}
I20260709 13:06:48.159606 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:48.174274 13949 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:48.047533 19694 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:48.239723 14017 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
W20260709 13:06:48.177317 14015 rpcz_store.cc:269] 0709 13:06:45.077695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.078139 (+   444us) service_pool.cc:224] Handling call
0709 13:06:48.054570 (+2976431us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b540023c654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:48.054707 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1416054528}
I20260709 13:06:48.247757 14015 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:06:48.178908 13944 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
W20260709 13:06:48.055934 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 512705us
I20260709 13:06:48.252733 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:48.180866 14559 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:48.176494 14522 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:48.254369 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 77985us
I20260709 13:06:48.255385 14522 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:48.177007 13947 rpcz_store.cc:269] 0709 13:06:45.804345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.804533 (+   188us) service_pool.cc:224] Handling call
0709 13:06:48.053490 (+2248957us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400381654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:48.053614 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3520774528}
I20260709 13:06:48.183573 14022 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:06:48.257928 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3089) took 1058 ms. Trace:
I20260709 13:06:48.186830 13950 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:48.258266 19694 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:48.185798 13914 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:48.166107 19999 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:48.258952 19694 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:48.259315 19999 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:48.259829 19999 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:48.258292 14022 rpcz_store.cc:276] 0709 13:06:47.200631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.142020 (+941389us) service_pool.cc:224] Handling call
0709 13:06:48.183235 (+ 41215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.257663 (+ 74428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.257874 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.207792 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2552) took 1017 ms. Trace:
I20260709 13:06:48.154790 19867 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:48.262645 14022 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:06:48.263060 19867 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:48.188373 20022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.008s	sys 0.004s
I20260709 13:06:48.262260 14019 rpcz_store.cc:276] 0709 13:06:47.190147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.410335 (+220188us) service_pool.cc:224] Handling call
0709 13:06:47.530201 (+119866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.132545 (+602344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.152044 (+ 19499us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b54001220d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:48.207485 (+ 55441us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54001220d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:48.207702 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1154710656}
I20260709 13:06:48.259584 19694 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:48.225526 14020 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:48.181958 20120 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:48.203285 19782 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:48.272619 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3090) took 1071 ms. Trace:
I20260709 13:06:48.295010 14023 rpcz_store.cc:276] 0709 13:06:47.201490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.217092 (+1015602us) service_pool.cc:224] Handling call
0709 13:06:48.271953 (+ 54861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.272196 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:48.272233 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:48.272292 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:48.272340 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.272413 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.272540 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.280123 14556 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:48.173313 20007 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:48.291383 14015 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:48.293018 14021 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:48.172142 19824 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.177006 20078 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.305012 14020 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:06:48.230751 20101 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:48.214789 20023 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:48.299887 20043 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:06:48.340651 14556 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:48.333798 20000 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:48.347323 20157 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:48.356350 20023 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:48.363178 20007 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:48.360626 19867 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:48.371172 20121 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:48.371582 20078 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:48.379228 19999 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.387188 20152 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:48.396240 20120 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:48.387961 20152 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:48.404862 20152 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:48.407449 20152 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:48.135991 13841 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42526 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3157)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3143)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3168)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3142)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3178)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3254)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3170)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3179)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3251)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3176)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3151)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3150)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3180)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3169)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3171)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3161)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3174)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3164)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3163)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3162)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3175)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3146)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3240)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3148)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3147)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 871)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3156)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3247)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3242)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3256)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3239)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3172)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3154)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3152)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3140)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3160)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 867)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3238)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3243)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3244)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3245)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3246)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3252)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3253)
W20260709 13:06:48.455435 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 309086us
I20260709 13:06:48.467535 13866 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:48.187304 13867 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:48.475229 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3143) took 1348 ms. Trace:
I20260709 13:06:48.197777 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:48.475829 13867 rpcz_store.cc:276] 0709 13:06:47.126379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.172035 (+1045656us) service_pool.cc:224] Handling call
0709 13:06:48.474903 (+302868us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.477193 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3157) took 1084 ms. Trace:
I20260709 13:06:48.189314 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.477939 13870 rpcz_store.cc:276] 0709 13:06:47.392169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.191204 (+799035us) service_pool.cc:224] Handling call
0709 13:06:48.476882 (+285678us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.479314 13868 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
W20260709 13:06:48.479857 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 227064us
I20260709 13:06:48.480299 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:48.209820 13872 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:48.226567 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.482587 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3149) took 1092 ms. Trace:
I20260709 13:06:48.483193 13874 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:48.483237 13872 rpcz_store.cc:276] 0709 13:06:47.389585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.115029 (+725444us) service_pool.cc:224] Handling call
0709 13:06:48.482306 (+367277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.484920 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.485389 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.487283 13872 raft_consensus.cc:2410] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:06:48.489156 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:48.491820 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 360519us
I20260709 13:06:48.494547 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:48.496027 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 715) took 1013 ms. Trace:
I20260709 13:06:48.498376 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:48.498974 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:48.499996 13944 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:06:48.501653 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:48.496552 14016 rpcz_store.cc:276] 0709 13:06:47.482345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.235066 (+752721us) service_pool.cc:224] Handling call
0709 13:06:48.235473 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:48.235780 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:48.235819 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:48.236471 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 13:06:48.494930 (+258459us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:48.495079 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.495254 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:48.495347 (+    93us) raft_consensus.cc:1685] finished
0709 13:06:48.495356 (+     9us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.495654 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":257668,"threads_started":1,"wal-append.queue_time_us":112631}
I20260709 13:06:48.502651 13947 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:48.504977 14016 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: [16.14-18.15]
I20260709 13:06:48.509186 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:48.509603 14016 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:48.490921 13872 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:48.481765 13870 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:48.526453 19665 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:48.527802 19665 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:48.532596 19975 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:48.533048 19668 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.536000 19824 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:48.533891 19668 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:48.536314 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 397223us
I20260709 13:06:48.538887 13988 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:48.540455 19884 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:48.515514 14020 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:48.516587 13874 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:06:48.481724 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:48.548887 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3150) took 1159 ms. Trace:
W20260709 13:06:48.489778 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 303925us
I20260709 13:06:48.552222 13874 rpcz_store.cc:276] 0709 13:06:47.389894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.484472 (+1094578us) service_pool.cc:224] Handling call
0709 13:06:48.548802 (+ 64330us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:48.560566 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 78993.5us
I20260709 13:06:48.564535 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:48.569363 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 95354us
I20260709 13:06:48.573045 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3151) took 1177 ms. Trace:
I20260709 13:06:48.578426 13868 rpcz_store.cc:276] 0709 13:06:47.390259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.481135 (+1090876us) service_pool.cc:224] Handling call
0709 13:06:48.567343 (+ 86208us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:48.600793 13866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3137) took 1479 ms (client timeout 1835 ms). Trace:
W20260709 13:06:48.601578 13866 rpcz_store.cc:269] 0709 13:06:47.121092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.914523 (+793431us) service_pool.cc:224] Handling call
0709 13:06:48.600511 (+685988us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54001aac54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:48.600702 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":767388544}
I20260709 13:06:48.603360 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:48.604632 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3164) took 1164 ms. Trace:
I20260709 13:06:48.604944 13945 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:48.605157 13866 rpcz_store.cc:276] 0709 13:06:47.439982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.602854 (+1162872us) service_pool.cc:224] Handling call
0709 13:06:48.604518 (+  1664us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.611790 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.612633 19999 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:48.613828 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.616235 20075 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:48.616734 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2564) took 1132 ms. Trace:
I20260709 13:06:48.573199 19975 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:48.637247 13943 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:48.618335 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.609380 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:48.621814 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 83487.4us
I20260709 13:06:48.632625 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:48.633426 19782 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:48.633708 13950 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:48.637058 19975 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.638684 13872 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:48.643916 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3171) took 1081 ms. Trace:
I20260709 13:06:48.644362 13872 rpcz_store.cc:276] 0709 13:06:47.562704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.568194 (+1005490us) service_pool.cc:224] Handling call
0709 13:06:48.643813 (+ 75619us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.646526 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3162) took 1238 ms. Trace:
I20260709 13:06:48.630182 14015 rpcz_store.cc:276] 0709 13:06:47.484181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.252240 (+768059us) service_pool.cc:224] Handling call
0709 13:06:48.276352 (+ 24112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.298096 (+ 21744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.616194 (+318098us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b540023b250. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:48.616508 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54587648}
I20260709 13:06:48.650507 13875 rpcz_store.cc:276] 0709 13:06:47.407694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.613262 (+1205568us) service_pool.cc:224] Handling call
0709 13:06:48.646445 (+ 33183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.646694 19782 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:48.638439 20075 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.658121 19665 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:48.659525 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.660699 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3175) took 1080 ms. Trace:
I20260709 13:06:48.661156 13866 rpcz_store.cc:276] 0709 13:06:47.579850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.642833 (+1062983us) service_pool.cc:224] Handling call
0709 13:06:48.660584 (+ 17751us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:48.662292 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080032bd00 after lost signal to thread 12728
W20260709 13:06:48.664179 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080032f660 after lost signal to thread 13807
I20260709 13:06:48.666178 20075 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:48.645622 13867 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:06:48.668985 13947 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:48.669485 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3251) took 1030 ms. Trace:
I20260709 13:06:48.673708 13867 rpcz_store.cc:276] 0709 13:06:47.638958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.478046 (+839088us) service_pool.cc:224] Handling call
0709 13:06:48.669417 (+191371us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.678210 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:48.680532 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 52843.6us
I20260709 13:06:48.679075 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3142) took 1553 ms. Trace:
I20260709 13:06:48.684537 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.684710 13871 rpcz_store.cc:276] 0709 13:06:47.126086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.189784 (+1063698us) service_pool.cc:224] Handling call
0709 13:06:48.608998 (+419214us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b5800003698. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:06:48.678990 (+ 69992us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6939008}
I20260709 13:06:48.693151 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.695921 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:48.696316 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080020e4e0 after lost signal to thread 13899
I20260709 13:06:48.698760 13873 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:48.699108 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:48.699560 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3163) took 1284 ms. Trace:
I20260709 13:06:48.700106 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3156) took 1308 ms. Trace:
I20260709 13:06:48.700657 13871 rpcz_store.cc:276] 0709 13:06:47.391869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.698602 (+1306733us) service_pool.cc:224] Handling call
0709 13:06:48.700038 (+  1436us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:48.702443 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.703255 19668 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:48.703114 20101 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.383651 20091 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:48.704970 20101 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:48.706244 14021 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:48.707122 20091 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:48.707645 20129 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.708860 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2555) took 1517 ms. Trace:
I20260709 13:06:48.708499 20129 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:48.713238 20129 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:48.714254 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:48.715984 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:48.716813 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3247) took 1094 ms. Trace:
I20260709 13:06:48.717132 13871 rpcz_store.cc:276] 0709 13:06:47.622962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.715553 (+1092591us) service_pool.cc:224] Handling call
0709 13:06:48.716746 (+  1193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.718240 19905 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:48.719247 19905 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:48.720994 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 717) took 1238 ms. Trace:
I20260709 13:06:48.700006 13873 rpcz_store.cc:276] 0709 13:06:47.415476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.611162 (+1195686us) service_pool.cc:224] Handling call
0709 13:06:48.699485 (+ 88323us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:48.730634 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.730898 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.724169 13988 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:48.727548 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080029d5e0 after lost signal to thread 13974
I20260709 13:06:48.729493 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:48.718376 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:48.737543 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 95708.4us
W20260709 13:06:48.740770 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08003197a0 after lost signal to thread 13975
I20260709 13:06:48.722404 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.730504 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:48.692055 20165 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.056s	user 0.000s	sys 0.002s
W20260709 13:06:48.750895 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 89368.8us
I20260709 13:06:48.751253 19668 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:48.756542 14558 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
W20260709 13:06:48.724157 20162 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:06:48.756811 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.743165 19964 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:48.757745 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3161) took 1364 ms. Trace:
W20260709 13:06:48.758227 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08003b0200 after lost signal to thread 14048
W20260709 13:06:48.758718 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08002b15a0 after lost signal to thread 14049
I20260709 13:06:48.759057 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:48.758253 13874 rpcz_store.cc:276] 0709 13:06:47.402491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.756399 (+1353908us) service_pool.cc:224] Handling call
0709 13:06:48.757676 (+  1277us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:48.760958 13896 debug-util.cc:398] Leaking SignalData structure 0x7b08000b8940 after lost signal to thread 14585
I20260709 13:06:48.758013 19964 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:48.762481 19628 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:48.761384 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:48.751616 13945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3133) took 1566 ms (client timeout 1850 ms). Trace:
I20260709 13:06:48.763796 13867 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:06:48.764569 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3147) took 1375 ms. Trace:
W20260709 13:06:48.761358 13896 debug-util.cc:398] Leaking SignalData structure 0x7b080005ae60 after lost signal to thread 14586
I20260709 13:06:48.764977 13867 rpcz_store.cc:276] 0709 13:06:47.388834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.681120 (+1292286us) service_pool.cc:224] Handling call
0709 13:06:48.764517 (+ 83397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.766206 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:48.754786 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3174) took 1175 ms. Trace:
I20260709 13:06:48.768330 13868 rpcz_store.cc:276] 0709 13:06:47.579527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.628542 (+1049015us) service_pool.cc:224] Handling call
0709 13:06:48.754717 (+126175us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:48.763666 13945 rpcz_store.cc:269] 0709 13:06:47.184754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.186098 (+  1344us) service_pool.cc:224] Handling call
0709 13:06:48.751347 (+1565249us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400092254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:48.751520 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3247538560}
I20260709 13:06:48.721999 19905 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:48.771171 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3242) took 1168 ms. Trace:
I20260709 13:06:48.772454 13874 rpcz_store.cc:276] 0709 13:06:47.599376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.760939 (+1161563us) service_pool.cc:224] Handling call
0709 13:06:48.767512 (+  6573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.774003 13866 raft_consensus.cc:2368] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12.
I20260709 13:06:48.773648 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:48.709343 14024 rpcz_store.cc:276] 0709 13:06:47.191500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.146996 (+955496us) service_pool.cc:224] Handling call
0709 13:06:48.157867 (+ 10871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.158136 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:48.160173 (+  2037us) log.cc:844] Serialized 14 byte log entry
0709 13:06:48.707693 (+547520us) log.cc:844] Serialized 14 byte log entry
0709 13:06:48.708081 (+   388us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:48.708176 (+    95us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:48.708288 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.708436 (+   148us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.708624 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":107349,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":80,"raft.queue_time_us":29815,"raft.run_cpu_time_us":489,"raft.run_wall_time_us":494,"thread_start_us":392327,"threads_started":2,"wal-append.queue_time_us":188038}
I20260709 13:06:48.776266 13872 raft_consensus.cc:2410] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:06:48.777158 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:48.721447 14017 rpcz_store.cc:276] 0709 13:06:47.482841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.240718 (+757877us) service_pool.cc:224] Handling call
0709 13:06:48.241176 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.241463 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:06:48.497981 (+256518us) log.cc:844] Serialized 14 byte log entry
0709 13:06:48.719602 (+221621us) log.cc:844] Serialized 14 byte log entry
0709 13:06:48.720317 (+   715us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:48.720398 (+    81us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:48.720496 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.720600 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.720757 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116134,"prepare.queue_time_us":109872,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":93,"raft.queue_time_us":57388,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":24068,"thread_start_us":300577,"threads_started":2}
I20260709 13:06:48.771128 19668 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:48.771970 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3250) took 1136 ms. Trace:
I20260709 13:06:48.784076 13871 rpcz_store.cc:276] 0709 13:06:47.635579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.717895 (+1082316us) service_pool.cc:224] Handling call
0709 13:06:48.771899 (+ 54004us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.785478 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3148) took 1396 ms. Trace:
I20260709 13:06:48.785775 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3146) took 1658 ms. Trace:
I20260709 13:06:48.786067 13866 rpcz_store.cc:276] 0709 13:06:47.389225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.662044 (+1272819us) service_pool.cc:224] Handling call
0709 13:06:48.785383 (+123339us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.786461 13872 rpcz_store.cc:276] 0709 13:06:47.127448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.646519 (+1519071us) service_pool.cc:224] Handling call
0709 13:06:48.785711 (+139192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.820891 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:48.818401 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 457727us
W20260709 13:06:48.824347 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 51289us
I20260709 13:06:48.824923 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:48.820910 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.828308 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:48.830114 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.830986 13874 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:48.831501 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3239) took 1233 ms. Trace:
I20260709 13:06:48.831730 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:48.832674 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:48.832859 20164 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:48.834472 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.834757 13989 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:48.835556 13871 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:48.836894 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:48.838265 19665 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:48.843201 19628 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:48.843600 20129 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:48.844295 19628 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:48.831848 13874 rpcz_store.cc:276] 0709 13:06:47.598556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.778451 (+1179895us) service_pool.cc:224] Handling call
0709 13:06:48.829876 (+ 51425us) spinlock_profiling.cc:243] Waited 5.11 ms on lock 0x7b5800003698. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:06:48.831429 (+  1553us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11757312}
I20260709 13:06:48.845562 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:48.846379 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:48.850637 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3116) took 2724 ms. Trace:
I20260709 13:06:48.854322 13867 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:48.855000 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3249) took 1225 ms. Trace:
I20260709 13:06:48.855508 13867 rpcz_store.cc:276] 0709 13:06:47.630614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.770308 (+1139694us) service_pool.cc:224] Handling call
0709 13:06:48.854933 (+ 84625us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.856809 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.854768 13869 rpcz_store.cc:276] 0709 13:06:46.125784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.127261 (+  1477us) service_pool.cc:224] Handling call
0709 13:06:46.155075 (+ 27814us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:48.209867 (+2054792us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:48.209902 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:48.225470 (+ 15568us) log.cc:844] Serialized 26 byte log entry
0709 13:06:48.512574 (+287104us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:48.512656 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.512800 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:48.850143 (+337343us) raft_consensus.cc:1685] finished
0709 13:06:48.850172 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.850401 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79662,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":152,"thread_start_us":2096030,"threads_started":2,"wal-append.queue_time_us":41754}
I20260709 13:06:48.846625 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3152) took 1456 ms. Trace:
I20260709 13:06:48.862962 13874 rpcz_store.cc:276] 0709 13:06:47.390629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.845107 (+1454478us) service_pool.cc:224] Handling call
0709 13:06:48.846546 (+  1439us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.833844 13838 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:48.878544 13870 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:48.878961 13947 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:48.879285 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3176) took 1299 ms. Trace:
I20260709 13:06:48.879643 13870 rpcz_store.cc:276] 0709 13:06:47.580151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.479533 (+899382us) service_pool.cc:224] Handling call
0709 13:06:48.879213 (+399680us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.883813 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3237) took 1286 ms. Trace:
I20260709 13:06:48.884351 13873 rpcz_store.cc:276] 0709 13:06:47.597900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.754429 (+1156529us) service_pool.cc:224] Handling call
0709 13:06:48.883745 (+129316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.850245 19776 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:06:48.887562 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 66538.7us
I20260709 13:06:48.887979 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.889226 13869 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:48.890151 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3095) took 1531 ms. Trace:
I20260709 13:06:48.890497 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.891494 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3172) took 1312 ms. Trace:
I20260709 13:06:48.891881 13866 rpcz_store.cc:276] 0709 13:06:47.578770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.823011 (+1244241us) service_pool.cc:224] Handling call
0709 13:06:48.891415 (+ 68404us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.847554 19964 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.893055 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.893560 19964 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:48.895913 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.896850 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3154) took 1505 ms. Trace:
I20260709 13:06:48.897176 13872 rpcz_store.cc:276] 0709 13:06:47.391239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.819819 (+1428580us) service_pool.cc:224] Handling call
0709 13:06:48.896783 (+ 76964us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:48.900528 20170 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.216s	user 0.005s	sys 0.000s
I20260709 13:06:48.902032 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:48.902760 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 56588.5us
I20260709 13:06:48.784667 19782 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:48.903810 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:48.904560 19776 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:48.905395 19782 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:48.905578 13989 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:48.906431 19668 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:48.926656 19668 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:48.928215 19668 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:48.891831 19694 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.624s	user 0.004s	sys 0.017s
I20260709 13:06:48.906993 19628 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:48.907110 13765 catalog_manager.cc:5697] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 6 to 14. New cstate: current_term: 14 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:48.890183 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:48.935343 13874 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:06:48.937170 13874 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:48.921298 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3102) took 1418 ms. Trace:
I20260709 13:06:48.890689 14018 rpcz_store.cc:276] 0709 13:06:47.207202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.226041 (+1018839us) service_pool.cc:224] Handling call
0709 13:06:48.226529 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:48.227027 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:48.227074 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:48.227707 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 13:06:48.674533 (+446826us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:48.694203 (+ 19670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.697276 (+  3073us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:48.732711 (+ 35435us) raft_consensus.cc:1685] finished
0709 13:06:48.732732 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.737498 (+  4766us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1065,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":39891,"thread_start_us":81367,"threads_started":1,"wal-append.queue_time_us":342046}
I20260709 13:06:48.922931 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:48.923236 20152 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.515s	user 0.006s	sys 0.012s
I20260709 13:06:48.885558 20075 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.925083 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:48.905913 19776 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:48.893754 13866 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:48.945710 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3160) took 1552 ms. Trace:
I20260709 13:06:48.946197 13866 rpcz_store.cc:276] 0709 13:06:47.393097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.892674 (+1499577us) service_pool.cc:224] Handling call
0709 13:06:48.945622 (+ 52948us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.933032 13873 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:06:48.883553 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:48.937906 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3241) took 1338 ms. Trace:
W20260709 13:06:48.853433 13767 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54072 (request call id 140) took 27050 ms (client timeout 15000 ms). Trace:
I20260709 13:06:48.948002 13874 rpcz_store.cc:276] 0709 13:06:47.599086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.936314 (+1337228us) service_pool.cc:224] Handling call
0709 13:06:48.936855 (+   541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.937618 (+   763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.937830 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.953652 20171 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:48.954205 14561 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:48.954471 13875 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
W20260709 13:06:48.953687 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 132635us
W20260709 13:06:48.958247 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 52829.4us
I20260709 13:06:48.958724 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:48.958845 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:48.951948 19905 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:06:48.960285 13915 leader_election.cc:400] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17.
I20260709 13:06:48.960665 19905 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:48.960855 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:48.954118 20114 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:48.961310 19905 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:48.961637 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:48.962247 19999 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:48.962393 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:48.961661 20114 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.963238 14015 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:48.963853 20114 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:48.965876 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3140) took 1843 ms. Trace:
I20260709 13:06:48.966312 20097 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:06:48.966995 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 133378us
I20260709 13:06:48.967407 20097 raft_consensus.cc:3055] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:48.967805 20097 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:48.968912 20097 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.961834 19905 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:48.970496 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45612.2us
I20260709 13:06:48.951728 19694 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.971472 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:48.973289 19997 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:48.973575 19999 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:48.960556 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:48.960634 13915 leader_election.cc:403] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:06:48.975865 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3238) took 1351 ms. Trace:
I20260709 13:06:48.976387 13873 rpcz_store.cc:276] 0709 13:06:47.598253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.920979 (+1322726us) service_pool.cc:224] Handling call
0709 13:06:48.921579 (+   600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.947141 (+ 25562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.949176 (+  2035us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.966338 13867 rpcz_store.cc:276] 0709 13:06:47.122121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.856384 (+1734263us) service_pool.cc:224] Handling call
0709 13:06:48.965536 (+109152us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b54000a7ea8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c9210924 00007fb2c83f9227 00007fb2c83f8ab8 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:48.965816 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42366208}
I20260709 13:06:48.938277 19628 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:48.940897 20164 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:48.962481 14017 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:48.939172 14022 rpcz_store.cc:276] 0709 13:06:47.502867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.263882 (+761015us) service_pool.cc:224] Handling call
0709 13:06:48.264360 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:48.272070 (+  7710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:48.272123 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:48.302475 (+ 30352us) log.cc:844] Serialized 26 byte log entry
0709 13:06:48.676660 (+374185us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:48.694713 (+ 18053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.721792 (+ 27079us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:48.899506 (+177714us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:48.908327 (+  8821us) raft_consensus.cc:1685] finished
0709 13:06:48.908350 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.921043 (+ 12693us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7023,"raft.queue_time_us":7207,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":20292,"spinlock_wait_cycles":4297984,"thread_start_us":234924,"threads_started":1,"wal-append.queue_time_us":324549}
I20260709 13:06:48.988880 13872 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:48.973594 19694 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:48.993583 13870 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
W20260709 13:06:48.990473 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 132377us
I20260709 13:06:48.961104 20022 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:06:49.013968 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 53521.6us
I20260709 13:06:49.014349 20022 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:49.015843 20022 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.017282 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:49.017594 20022 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:48.992992 13874 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:49.003770 13949 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:49.018674 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3244) took 1417 ms. Trace:
I20260709 13:06:49.003650 20152 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:49.019115 13874 rpcz_store.cc:276] 0709 13:06:47.600842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.949230 (+1348388us) service_pool.cc:224] Handling call
0709 13:06:48.992716 (+ 43486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.018313 (+ 25597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.018598 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:49.019956 20152 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.985543 19782 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:06:49.021045 13874 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:49.021667 19782 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:49.003995 20169 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:49.022351 19782 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.021732 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3252) took 1380 ms. Trace:
I20260709 13:06:49.025032 13874 rpcz_store.cc:276] 0709 13:06:47.641681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.020168 (+1378487us) service_pool.cc:224] Handling call
0709 13:06:49.020690 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.021482 (+   792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.021666 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.985834 19628 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.010356 19982 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17.
I20260709 13:06:48.995575 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 867) took 1548 ms. Trace:
W20260709 13:06:48.951658 13767 rpcz_store.cc:269] 0709 13:06:21.803032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.803296 (+   264us) service_pool.cc:224] Handling call
0709 13:06:48.853041 (+27049745us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:41.508101 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:41.533748 (+ 25647us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:41.533818 (+    70us) write_op.cc:435] Acquired schema lock
0709 13:06:41.533839 (+    21us) tablet.cc:662] Decoding operations
0709 13:06:41.535229 (+  1390us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:41.535339 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:41.535367 (+    28us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:06:41.626767 (+ 91400us) tablet.cc:701] Row locks acquired
0709 13:06:41.638174 (+ 11407us) write_op.cc:260] PREPARE: finished
0709 13:06:41.638450 (+   276us) write_op.cc:270] Start()
0709 13:06:41.638648 (+   198us) write_op.cc:276] Timestamp: P: 1783602401638343 usec, L: 0
0709 13:06:41.638689 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:06:41.639536 (+   847us) log.cc:844] Serialized 5196 byte log entry
0709 13:06:41.952052 (+312516us) op_driver.cc:464] REPLICATION: finished
0709 13:06:42.144700 (+192648us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c03290ac 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c6e69ef0
0709 13:06:42.316207 (+171507us) write_op.cc:301] APPLY: starting
0709 13:06:42.319884 (+  3677us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:42.775803 (+455919us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:42.775853 (+    50us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:43.040721 (+264868us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:43.041111 (+   390us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 13:06:43.041172 (+    61us) write_op.cc:312] APPLY: finished
0709 13:06:43.062580 (+ 21408us) log.cc:844] Serialized 121 byte log entry
0709 13:06:43.371684 (+309104us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:43.371890 (+   206us) write_op.cc:454] Released schema lock
0709 13:06:43.371994 (+   104us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:43.398925 (+ 26931us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:43.413187 (+ 14262us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":437097,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":348205,"apply.run_cpu_time_us":48681,"apply.run_wall_time_us":1066265,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58389,"num_ops":13,"prepare.queue_time_us":209693,"prepare.run_cpu_time_us":52537,"prepare.run_wall_time_us":277489,"raft.queue_time_us":10556,"raft.run_cpu_time_us":13777,"raft.run_wall_time_us":354646,"replication_time_us":286489,"spinlock_wait_cycles":26414336,"thread_start_us":838508,"threads_started":5,"wal-append.queue_time_us":623958,"wal-append.run_cpu_time_us":7351,"wal-append.run_wall_time_us":2330050}]]}
I20260709 13:06:49.029717 13872 rpcz_store.cc:276] 0709 13:06:47.443737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.908928 (+1465191us) service_pool.cc:224] Handling call
0709 13:06:48.953812 (+ 44884us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.989436 (+ 35624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.991972 (+  2536us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25472}
I20260709 13:06:48.974175 19997 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:48.990314 20097 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:48.985366 20164 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:49.034152 13838 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:49.047765 13873 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:06:49.049441 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3246) took 1431 ms. Trace:
I20260709 13:06:49.050601 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3245) took 1438 ms. Trace:
I20260709 13:06:49.051138 13873 rpcz_store.cc:276] 0709 13:06:47.612496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.982850 (+1370354us) service_pool.cc:224] Handling call
0709 13:06:49.041747 (+ 58897us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:49.050250 (+  8503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.050526 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:49.049892 13867 rpcz_store.cc:276] 0709 13:06:47.612890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.984690 (+1371800us) service_pool.cc:224] Handling call
0709 13:06:49.024198 (+ 39508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.039602 (+ 15404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:49.039641 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:49.039703 (+    62us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:49.043491 (+  3788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.043568 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.043772 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:49.053033 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:49.059319 13874 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:49.060850 20181 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:49.064585 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3105) took 1552 ms. Trace:
I20260709 13:06:49.065001 14019 rpcz_store.cc:276] 0709 13:06:47.512241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.265747 (+753506us) service_pool.cc:224] Handling call
0709 13:06:48.266149 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.266467 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:48.673026 (+406559us) log.cc:844] Serialized 14 byte log entry
0709 13:06:49.032233 (+359207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:49.039768 (+  7535us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:49.049355 (+  9587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.056345 (+  6990us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.064384 (+  8039us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56883,"prepare.queue_time_us":70394,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":1449,"raft.queue_time_us":77513,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":11003,"thread_start_us":519824,"threads_started":2,"wal-append.queue_time_us":342015}
I20260709 13:06:49.067269 20023 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:49.067757 20023 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.068137 20023 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:49.070640 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 78878.1us
I20260709 13:06:49.072166 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:49.072335 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3240) took 1473 ms. Trace:
I20260709 13:06:49.072914 13875 rpcz_store.cc:276] 0709 13:06:47.598817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.652985 (+1054168us) service_pool.cc:224] Handling call
0709 13:06:49.072053 (+419068us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54000da554. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:49.072238 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":613223296}
I20260709 13:06:49.075145 20075 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:49.078197 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 44520.5us
W20260709 13:06:49.078816 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.079660 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:49.081105 19983 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:49.082505 19986 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:49.084895 19986 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:49.085601 19986 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:49.086479 20169 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:49.081907 19983 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:49.087716 19997 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:49.088693 19983 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.093907 19986 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.094825 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:49.045349 20164 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:49.098655 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 125821us
I20260709 13:06:49.100875 13873 raft_consensus.cc:3055] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:49.060153 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3253) took 1418 ms. Trace:
I20260709 13:06:49.102015 13874 rpcz_store.cc:276] 0709 13:06:47.642001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.026340 (+1384339us) service_pool.cc:224] Handling call
0709 13:06:49.055796 (+ 29456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.059783 (+  3987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.060080 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:49.101311 13873 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.023526 19782 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.111140 14558 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:49.009590 13765 catalog_manager.cc:5697] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 5 to 12, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 12 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:49.098776 20152 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:49.127805 14561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 995) took 1053 ms. Trace:
I20260709 13:06:49.128275 14561 rpcz_store.cc:276] 0709 13:06:48.067561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.068578 (+  1017us) service_pool.cc:224] Handling call
0709 13:06:49.106326 (+1037748us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54005609d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:49.120564 (+ 14238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778954496}
I20260709 13:06:49.133111 20158 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:06:49.119817 20129 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:49.124833 19997 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:49.136729 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:49.110816 20169 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:06:49.136371 20129 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:49.139063 19997 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:49.160315 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.160055 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:49.160169 20043 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:49.169669 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:49.170637 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 60533.5us
I20260709 13:06:49.171447 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:49.172431 20171 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:49.173132 19668 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:49.173715 19668 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:49.174113 19668 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:49.176759 19665 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:49.177697 19665 consensus_queue.cc:1243] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:06:49.166962 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 120799us
I20260709 13:06:49.162694 20129 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:49.164543 19997 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:49.202587 13871 raft_consensus.cc:2410] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:06:49.168972 13873 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.208714 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:49.214568 20180 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:49.217741 20181 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:06:49.129771 20152 consensus_queue.cc:1175] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" responder_term: 17 status { last_received { term: 17 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:49.225062 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3243) took 1625 ms. Trace:
I20260709 13:06:49.226519 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3248) took 1598 ms. Trace:
I20260709 13:06:49.226761 20181 consensus_queue.cc:1243] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:06:49.227101 13870 rpcz_store.cc:276] 0709 13:06:47.628500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.011115 (+1382615us) service_pool.cc:224] Handling call
0709 13:06:49.011687 (+   572us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:49.012180 (+   493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:49.012217 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:49.012963 (+   746us) log.cc:844] Serialized 52 byte log entry
0709 13:06:49.147576 (+134613us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:49.147636 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.147713 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:49.226195 (+ 78482us) raft_consensus.cc:1685] finished
0709 13:06:49.226225 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.226418 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6112,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":134,"thread_start_us":134036,"threads_started":1,"wal-append.queue_time_us":209017}
W20260709 13:06:49.235445 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 112443us
I20260709 13:06:49.237787 13945 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:49.238183 20114 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.240798 20114 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.242529 20169 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:49.247373 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:49.248263 13949 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:49.207331 19665 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:49.225684 13866 rpcz_store.cc:276] 0709 13:06:47.599674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.953165 (+1353491us) service_pool.cc:224] Handling call
0709 13:06:49.029903 (+ 76738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.046042 (+ 16139us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:49.104155 (+ 58113us) log.cc:844] Serialized 14 byte log entry
0709 13:06:49.224638 (+120483us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:49.224706 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:49.224773 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.224851 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.224981 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21720,"prepare.queue_time_us":49881,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":2479,"raft.queue_time_us":23082,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"thread_start_us":68324,"threads_started":1}
I20260709 13:06:49.194028 19905 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.226533 20180 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:49.264679 20180 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.265872 20180 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.267483 20180 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.267877 19905 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.258123 20124 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:06:49.283934 20177 log.cc:927] Time spent T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.122s	user 0.003s	sys 0.000s
I20260709 13:06:49.194445 13765 catalog_manager.cc:5697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 12, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 12 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:49.287285 19782 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.261921 20023 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.289606 20023 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:49.295997 20184 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.135s	user 0.001s	sys 0.001s
I20260709 13:06:49.297653 20023 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.298381 20023 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.299912 13872 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:06:49.300293 13872 pending_rounds.cc:85] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 14
I20260709 13:06:49.300509 13872 pending_rounds.cc:107] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:06:49.300802 13872 raft_consensus.cc:2889] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:49.301011 20023 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:49.301627 20023 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:49.308060 20007 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:49.270463 13949 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 23.
I20260709 13:06:49.310026 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:49.271342 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 98915.2us
I20260709 13:06:49.277649 20101 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:49.318007 20185 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.158s	user 0.000s	sys 0.003s
I20260709 13:06:49.280403 19867 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:49.249003 20158 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:06:49.283788 20124 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:49.317157 19988 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:49.251745 19665 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.286494 13765 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 11 to 14, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 14 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:49.263290 19668 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.321100 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:49.316864 19905 consensus_queue.cc:1059] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:06:49.320658 20152 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.278294 20181 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:49.329939 14524 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:49.338402 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 869) took 1211 ms. Trace:
I20260709 13:06:49.350875 13869 rpcz_store.cc:276] 0709 13:06:48.126846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.943249 (+816403us) service_pool.cc:224] Handling call
0709 13:06:49.309897 (+366648us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b5800003698. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030eeee 0000000000375939 00007fb2c83821e0 00007fb2c844b0c4 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:06:49.338329 (+ 28432us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24708224}
W20260709 13:06:49.340526 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.318186 20091 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.341378 13950 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:49.345091 19665 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.346935 13765 catalog_manager.cc:5697] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
W20260709 13:06:49.352662 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.353384 14558 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.337718 20171 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:49.339167 19867 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.341985 20169 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.341979 20124 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:06:49.364853 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 156223us
I20260709 13:06:49.363051 19628 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.341962 20101 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.381588 19628 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:49.382014 20101 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.370217 14554 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:06:49.370570 20169 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.371804 13765 catalog_manager.cc:5697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 14, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 14 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:49.373093 20120 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:49.373932 19867 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.375274 19668 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:49.375478 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.354203 20152 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.369833 19665 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.389161 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:49.390211 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 60426.2us
I20260709 13:06:49.390658 20152 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.391055 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:49.391402 13873 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:49.392814 19997 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.386394 20120 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:49.394306 20120 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:49.369535 20091 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.395598 20169 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:49.396517 20007 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.396731 20091 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.358788 19988 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.397132 20007 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:49.388422 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.391526 19824 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:49.399101 13989 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:49.399250 19988 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:49.388952 19867 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:49.399470 19824 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.399888 13765 catalog_manager.cc:5697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: config changed from index 3 to 10, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New cstate: current_term: 12 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:49.400509 19824 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.390131 14019 raft_consensus.cc:2393] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 12: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:49.402910 20114 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:49.382272 20194 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:06:49.403370 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:49.403592 20114 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.404011 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:49.404893 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:49.388806 20124 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.409972 20121 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:06:49.415869 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.416910 20078 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:06:49.417259 20182 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:49.401913 19824 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:49.420718 20169 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:49.421105 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:49.405906 20114 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.414954 19975 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.423084 20169 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:49.430580 13866 raft_consensus.cc:2368] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17.
I20260709 13:06:49.422986 19905 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 13:06:49.433971 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:49.401995 13765 catalog_manager.cc:5697] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.422968 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.434823 20022 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.435272 19986 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:49.435388 19983 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.436504 20129 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.438452 19997 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:49.436410 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:49.443193 13989 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:49.443974 20007 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:49.444075 13872 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:49.414623 20121 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:49.444927 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:49.446493 13870 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:49.448979 14558 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:49.457263 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:49.464856 13945 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:49.471732 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:49.473980 20121 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:49.462452 20169 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:49.458421 14551 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:49.473381 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:49.455952 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:49.481104 13873 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:49.456496 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:49.485865 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:49.482501 14522 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:49.476802 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.473981 19997 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:49.457768 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:49.486141 13914 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:49.499878 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.465541 20180 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:49.505237 13944 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:49.501391 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.465337 19905 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.504791 19668 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:49.507035 19905 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.507536 13947 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:49.507990 20111 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:49.508594 20111 raft_consensus.cc:697] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:49.509370 19905 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.509738 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:49.510115 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.510816 20075 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:49.465214 19983 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.507573 20182 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:49.512543 20124 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:49.513168 19983 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.464588 20022 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.512573 20182 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:49.462468 19975 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.509644 20111 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.514935 20182 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.515628 20022 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.499136 19997 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.516183 19975 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.505179 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.516533 19997 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:49.464892 19986 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.522447 19986 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.467424 20129 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:49.503446 20097 raft_consensus.cc:3055] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:49.526472 20097 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:49.526603 20129 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:49.527089 20129 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.527477 20129 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:49.511234 20075 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:49.481426 19982 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:49.471966 20114 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:49.561825 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.565485 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:49.572808 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:49.527582 20097 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:49.574010 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 88260.7us
I20260709 13:06:49.579006 14552 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:49.579514 14019 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:06:49.581714 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.558352 19694 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 13:06:49.582896 13948 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:49.583108 19694 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:49.583491 19694 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:49.584072 13942 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:06:49.584824 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:49.584518 19694 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.585145 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.586426 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.588109 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:49.588979 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.589363 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:49.590390 20164 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.591342 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:49.592805 20164 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.593492 20164 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.532845 19982 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:49.594270 13945 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:49.594377 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:49.595444 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:49.528241 20180 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:49.595681 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 122479us
I20260709 13:06:49.598067 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:49.598382 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:49.598708 20097 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.599597 13989 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:49.600553 20194 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:49.600967 20194 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:49.601356 20194 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:49.603675 13838 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:49.610286 20023 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:49.464226 13765 catalog_manager.cc:5697] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 16, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 16 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.612025 13989 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:49.533959 20075 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:49.616182 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.596606 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.618993 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.619853 13942 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:49.584163 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:49.585917 19694 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:49.630432 14554 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:49.635594 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:49.631052 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:49.636894 14022 raft_consensus.cc:2368] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12.
I20260709 13:06:49.633315 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:49.634280 14554 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:49.637629 13949 raft_consensus.cc:2368] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15.
I20260709 13:06:49.612600 19982 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:49.615429 20007 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:49.650542 13942 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:49.616987 20180 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:49.647905 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:49.653972 14017 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:49.596597 19997 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.657826 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:49.657722 19782 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:49.659314 19782 raft_consensus.cc:3055] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:49.659372 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:49.659758 19782 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.661648 13838 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:49.593966 20164 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.666404 13838 leader_election.cc:400] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15.
I20260709 13:06:49.662797 14015 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:49.666761 13838 leader_election.cc:403] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:06:49.651563 14554 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.667220 13941 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:49.667769 14554 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:49.623395 19665 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:06:49.667819 14018 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:49.647552 19668 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:49.670842 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:49.531932 20171 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:49.674621 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 62727.8us
I20260709 13:06:49.667775 20091 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:49.677251 20091 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.677661 20091 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:49.681182 19997 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:49.683139 20181 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:49.684118 20121 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.684260 19782 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.684985 14554 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:06:49.685654 19782 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.686913 14552 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:49.687420 14015 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:49.662606 20114 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:49.697098 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.648144 20152 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:06:49.674310 19668 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:49.716288 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:49.725273 20007 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:49.725816 20007 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.726909 20121 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:49.727653 20169 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.730412 19665 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:49.730135 20007 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.731040 19665 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.731489 20007 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.733098 19665 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.734388 19668 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:49.734946 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:49.740147 13937 tablet_service.cc:1558] Processing DeleteTablet for tablet 02dcf286c6bd48e3a6e548def7d786cc with delete_type TABLET_DATA_TOMBSTONED (TS 3779649036364744823fc035dac892c7 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:36032
I20260709 13:06:49.742663 20091 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:49.748589 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 157343us
I20260709 13:06:49.758438 13950 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:49.751673 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:49.761029 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:49.764973 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:49.765900 13943 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:49.766640 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 52767.9us
I20260709 13:06:49.771885 13946 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:49.754189 20091 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:49.775385 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 93096.6us
I20260709 13:06:49.775724 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.771854 19694 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:49.784117 14022 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:06:49.766736 20158 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:06:49.786023 13948 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:49.783953 19694 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:49.795698 13946 raft_consensus.cc:1240] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:49.795944 19694 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:49.798221 20164 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.775208 20022 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:06:49.789223 20114 consensus_queue.cc:1059] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:06:49.806272 20114 raft_consensus.cc:3055] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:49.806704 20114 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:49.763958 20043 consensus_queue.cc:1059] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:06:49.812918 20091 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.815521 20091 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.763939 19983 consensus_queue.cc:1059] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:06:49.778342 20169 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:49.818316 20091 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:49.822973 13838 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:49.821002 20114 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.824656 20114 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.826679 19997 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:49.827001 19668 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:49.828990 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.823261 20129 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.830436 19668 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:49.831110 20129 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:49.788359 20152 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:06:49.835126 19824 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:49.835563 19824 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.837816 20043 raft_consensus.cc:3055] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:06:49.839787 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:49.844592 20199 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:06:49.844429 20022 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:49.846923 19983 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:49.847376 19983 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:49.838255 20043 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.849471 20043 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.851161 20043 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:49.858251 13943 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:49.853539 20120 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.864511 14022 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:49.865973 14551 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:49.866186 20101 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:49.866725 20101 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:49.867182 20101 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.871124 13838 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:49.871744 20022 consensus_queue.cc:1059] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:49.872927 20091 consensus_queue.cc:1059] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:49.873720 20122 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:49.874104 20091 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:49.874300 20122 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:49.874717 20091 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.875727 20091 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.695706 13765 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 9 to 17, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 17 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:49.879122 20164 raft_consensus.cc:3055] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:49.879549 20164 raft_consensus.cc:740] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.878146 13765 catalog_manager.cc:5697] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 15, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 15 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.880484 20164 consensus_queue.cc:260] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:49.881774 20164 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.882658 20120 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:49.882082 19983 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.859261 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:49.787897 20158 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:49.886895 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:49.891453 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 138493us
I20260709 13:06:49.836506 19824 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:49.900749 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.915126 14018 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:49.919813 14022 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:49.926659 19824 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.931782 20078 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:49.932732 19982 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.934161 13946 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:49.944556 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:49.950534 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:49.951272 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:49.952430 13872 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:49.953338 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:49.953936 14017 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:49.954022 14019 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.956570 14018 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:49.957674 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:49.958423 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:49.965508 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.958362 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:49.944469 20129 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.958784 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:49.959270 13989 leader_election.cc:400] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12.
I20260709 13:06:49.974647 13989 leader_election.cc:403] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:49.960083 19867 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.000s
I20260709 13:06:49.956445 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:49.969400 14020 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:06:49.969441 14018 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 17: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:49.978912 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:49.979954 14554 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:49.965301 14021 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.985914 14022 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:49.875739 20122 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:49.986481 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:49.991156 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:49.991418 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:49.991894 14561 raft_consensus.cc:2376] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12. Re-sending same reply.
I20260709 13:06:49.992555 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:49.993304 14558 raft_consensus.cc:2376] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12. Re-sending same reply.
I20260709 13:06:49.994382 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:49.944795 19982 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:49.992758 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:50.001752 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:50.002110 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:50.006290 13866 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:49.944211 20111 consensus_queue.cc:1059] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:06:49.985577 19983 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:50.001453 20207 tablet_replica.cc:333] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:06:50.022403 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:50.009074 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:50.023455 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:49.998853 20169 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:50.027762 14020 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:50.053728 14019 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:50.060034 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:50.060890 14017 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:49.986137 20129 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.014597 20023 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:50.038892 20124 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.526s	user 0.015s	sys 0.000s
I20260709 13:06:50.066140 19964 consensus_queue.cc:1059] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:50.068411 20207 raft_consensus.cc:2243] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:50.071208 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.071576 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:50.073411 20111 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:50.049252 19982 raft_consensus.cc:3055] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:06:50.103212 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 144090us
I20260709 13:06:50.043156 20122 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:50.092480 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 90497.4us
I20260709 13:06:50.127697 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:50.136610 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.138230 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:50.116155 19982 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:50.144906 19964 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:50.145282 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:50.112227 20111 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:49.997125 20158 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.057982 19694 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.146472 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:50.154917 14019 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
W20260709 13:06:50.155407 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 179582us
I20260709 13:06:50.063393 20169 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:50.150374 20075 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.004s
W20260709 13:06:50.153671 20097 consensus_meta.cc:287] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.554s	user 0.013s	sys 0.000s
I20260709 13:06:50.152992 20111 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:50.167351 20194 consensus_meta.cc:287] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.000s
I20260709 13:06:50.182760 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:50.162856 20171 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:50.185654 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.160926 20181 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15.
I20260709 13:06:50.145984 20023 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.155025 19982 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.056207 13765 catalog_manager.cc:5697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 14, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 14 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:50.213793 20078 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:50.220049 19982 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:50.221443 19694 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:50.186100 20158 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:50.171324 20075 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.179054 20114 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:06:50.232090 20217 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:50.232745 20217 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.234970 20217 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:50.235507 20129 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:50.236282 20210 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:06:50.241503 20169 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:50.242412 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 107509us
I20260709 13:06:50.243361 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:50.243646 20075 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:50.247007 20023 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:50.186379 20194 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:50.252446 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:50.253132 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.259706 14021 raft_consensus.cc:2410] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:06:50.264190 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:50.265105 14019 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 20: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:50.267017 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 87378.1us
I20260709 13:06:50.228323 20207 raft_consensus.cc:2272] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.279165 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:50.265827 19997 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12.
W20260709 13:06:50.287662 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.295940 13988 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:50.277299 20023 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:50.302158 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:50.304669 20169 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:50.305286 20169 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:50.306262 20169 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:50.306910 20180 consensus_meta.cc:287] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.002s
I20260709 13:06:50.307407 20180 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.310001 20180 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:50.270948 20075 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:50.322115 20075 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.324571 20075 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:50.327021 14021 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:50.303375 14552 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
W20260709 13:06:50.330245 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.347525 20121 raft_consensus.cc:3055] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:06:50.348023 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.348241 20121 raft_consensus.cc:740] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:06:50.350559 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 107313us
I20260709 13:06:50.352947 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:50.354426 13765 catalog_manager.cc:5697] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 10, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 10 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:50.356775 13765 catalog_manager.cc:5697] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:50.364173 13947 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:50.289402 20007 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.369841 20007 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.377314 20101 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.380390 20101 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:50.382162 20194 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:50.385041 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:50.386516 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 90705.9us
I20260709 13:06:50.388794 20120 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:50.388876 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:50.391259 14554 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
W20260709 13:06:50.391396 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 402920us
W20260709 13:06:50.310158 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.354445 20023 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:50.397097 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 126843us
I20260709 13:06:50.397964 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:50.361809 20207 ts_tablet_manager.cc:1916] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:50.399981 14522 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:50.402076 13915 leader_election.cc:400] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17.
I20260709 13:06:50.373668 20007 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:50.401456 20122 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:50.401046 13914 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:50.427227 13839 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:50.400300 20120 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.416618 20194 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:50.424623 19694 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:50.436731 13839 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:50.349575 20121 consensus_queue.cc:260] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:50.437481 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 46752.2us
I20260709 13:06:50.424036 19986 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:06:50.431017 20075 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:06:50.439266 20007 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:50.451596 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.443758 19975 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:50.440631 20194 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:50.432454 20122 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.439679 13765 catalog_manager.cc:5697] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 9 to 10, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 10 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.464927 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:50.467195 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:50.480978 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:50.482090 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:50.482764 13943 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:50.483858 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:50.477831 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:50.453557 20101 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:50.492054 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:50.495036 14554 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
W20260709 13:06:50.496238 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 59623us
I20260709 13:06:50.454226 19975 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:50.507292 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:50.537693 13943 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:50.501502 20194 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.498034 20121 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:50.506466 19975 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:50.508864 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.505970 13988 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:50.506552 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:50.532420 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:50.546216 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:50.542517 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:50.547034 13947 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 9.
I20260709 13:06:50.548617 13873 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 11: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:06:50.552028 20216 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.254s	user 0.004s	sys 0.000s
I20260709 13:06:50.554736 20122 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:50.562681 20215 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.378s	user 0.003s	sys 0.000s
I20260709 13:06:50.567548 13871 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 23.
W20260709 13:06:50.574698 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 221831us
I20260709 13:06:50.575608 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:50.585144 20007 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:50.586215 20101 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:50.586658 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.567636 20111 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:50.585016 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 84469.6us
I20260709 13:06:50.608227 13838 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:50.570806 20023 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:50.591503 20223 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:50.591964 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.548508 20218 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:50.462086 13765 catalog_manager.cc:5697] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 5 to 11, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 11 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:50.658769 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:50.539928 19694 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:50.594290 14553 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 9.
I20260709 13:06:50.660485 13943 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:06:50.607673 14522 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:50.661418 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 53863us
W20260709 13:06:50.590333 19983 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.004s
I20260709 13:06:50.609351 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:50.614424 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:50.611296 20101 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.611655 20091 raft_consensus.cc:3055] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:50.671600 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:50.627228 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:50.628412 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:50.629222 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:50.629999 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:50.630415 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:50.630738 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:50.631047 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:50.631575 14024 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:50.637117 13949 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:50.633031 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:50.675827 14024 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:50.646803 13942 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:50.664006 14021 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:06:50.610728 20225 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.675073 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:50.684429 14016 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:06:50.673054 13914 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:50.682564 14019 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:50.632313 20111 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:50.663967 19824 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:50.698632 19824 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:50.699757 20164 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.700376 20164 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.701401 20101 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:50.703979 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:50.704157 19982 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:50.704726 19982 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:50.705413 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:50.706262 20114 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:50.634344 20122 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:06:50.665834 19964 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:50.707942 13765 catalog_manager.cc:5697] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 7 to 14. New cstate: current_term: 14 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:50.659291 20218 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:50.706684 20114 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:50.709750 13874 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:06:50.689783 14017 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:50.687721 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 48252.8us
I20260709 13:06:50.714110 20129 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.714587 20043 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:50.719252 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 159073us
I20260709 13:06:50.723301 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:50.715818 14024 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.671399 20091 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:50.719276 14023 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:50.727837 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:50.729162 14024 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:50.665868 20007 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:50.748893 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:50.754839 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.765553 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:50.771340 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 144184us
I20260709 13:06:50.772611 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:50.765772 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:50.664294 20223 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.718369 19964 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:50.781509 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:50.724711 20180 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:50.814194 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:50.818621 20171 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:50.814103 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 47017.6us
I20260709 13:06:50.724870 20114 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:50.821883 19986 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.822639 19986 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.825650 19986 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:50.722026 20158 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:50.827915 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:50.784454 14021 raft_consensus.cc:1240] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:50.785576 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:50.786657 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:50.721392 20218 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:50.762246 19983 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:50.720602 19782 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:06:50.796139 20181 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:50.724611 20206 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:50.845147 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 235900us
W20260709 13:06:50.845544 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.846115 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:50.848604 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:50.768682 20078 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 13:06:50.800518 14018 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:50.799480 13765 catalog_manager.cc:5697] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 11 to 17, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 17 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:06:50.777113 20043 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.734653 20129 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.768949 20101 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:50.806860 20120 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:50.784482 20023 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:50.814707 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.816529 19964 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:50.819659 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.832640 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:50.834628 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:50.705744 19982 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.769447 20122 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.855422 13839 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:50.852360 13987 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:50.861532 13941 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:50.787261 19997 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:50.788666 20111 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:50.896849 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:50.897810 14553 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:50.899541 14024 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:06:50.699785 19824 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.851353 20171 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:06:50.864868 20101 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.870956 20097 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:06:50.881505 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 109021us
I20260709 13:06:50.853955 20181 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.884860 13839 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:50.912454 13914 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:50.883764 20164 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:50.869886 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.776909 20007 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.902979 14024 pending_rounds.cc:85] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 8
I20260709 13:06:50.718329 20225 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:50.908748 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:50.895956 20111 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:50.950215 13947 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 20: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:50.910688 20101 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:50.911510 19982 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:50.894629 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 167712us
I20260709 13:06:50.920044 14024 pending_rounds.cc:107] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 12.9
W20260709 13:06:50.921700 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.858821 19694 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.960083 14024 raft_consensus.cc:2889] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:50.903321 20023 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:06:50.916731 19983 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:50.912612 20181 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:50.958184 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:50.916386 20075 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.899333 20122 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:50.963996 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.951740 20234 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:50.984293 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.987272 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:50.953697 20231 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.956497 20180 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:50.955580 19782 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.903775 20091 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.963835 20097 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:50.958529 20164 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
W20260709 13:06:51.005623 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 96967.2us
W20260709 13:06:51.007759 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 155496us
I20260709 13:06:50.975786 19983 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:51.014384 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 129614us
W20260709 13:06:51.024665 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 260963us
I20260709 13:06:51.026146 13839 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:06:51.000871 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 236036us
I20260709 13:06:51.024044 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:51.030861 13915 leader_election.cc:400] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 11 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 11. Current term is 12.
I20260709 13:06:51.009640 20235 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:50.989100 19975 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:50.985877 20075 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.030197 19983 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:51.020318 20239 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:51.049458 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:51.031401 20238 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:51.016294 19782 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.064096 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.036476 19997 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:06:51.077971 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 53599.5us
W20260709 13:06:51.081588 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.082093 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.042263 20235 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.083503 13950 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:51.083328 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.085223 13941 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:51.083160 13946 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:06:51.096530 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 70511.2us
I20260709 13:06:51.044611 20245 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:51.055672 20223 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:51.089737 13945 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:51.022624 20111 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.013230 20231 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.077497 20238 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:51.115861 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 103275us
I20260709 13:06:51.065574 20217 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 13:06:51.087069 20043 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.127573 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:51.163291 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:51.164121 14552 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:06:51.117317 20023 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:51.093238 20235 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:51.156255 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:51.151011 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:51.160491 13838 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:51.113580 20239 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:06:51.165205 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 136819us
I20260709 13:06:51.175168 13869 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:51.117589 20129 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.177309 14559 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 20: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:51.119076 20097 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:51.177870 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:51.188005 14024 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:51.179973 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.179685 13838 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:51.176226 14553 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:51.181370 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.119024 20007 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:51.185956 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 147233us
I20260709 13:06:51.117465 20225 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:51.126518 20245 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:51.155929 20180 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.187887 14018 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:51.191440 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.192672 13873 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:06:51.193040 13870 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:06:51.202786 13870 pending_rounds.cc:85] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 9
I20260709 13:06:51.185657 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.194326 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.204085 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:51.203143 13870 pending_rounds.cc:107] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:06:51.204787 13870 raft_consensus.cc:2889] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:51.195461 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:51.207183 14021 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:51.207093 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.207147 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.189934 13867 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:51.156486 20238 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:51.199501 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.200320 14022 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:06:51.203856 14015 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:51.197150 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.196087 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:51.174767 20239 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:51.228538 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:51.229460 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:51.170320 20023 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:51.208729 13869 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
W20260709 13:06:51.224248 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44618.1us
I20260709 13:06:51.178751 20043 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.190938 20158 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.042028 13765 catalog_manager.cc:5697] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 4 to 11, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 11 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:51.232319 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.242503 14019 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:51.243031 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.243853 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.233872 14024 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:51.233959 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:51.232311 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:51.238170 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 125052us
I20260709 13:06:51.239843 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:51.245687 13942 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:51.250593 13871 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:51.253590 14023 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 20: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:51.253691 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.255509 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:51.252478 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.221863 20225 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:51.258329 13867 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:51.263796 14019 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:51.230541 20129 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:51.273247 14016 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:51.276417 20164 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.277083 20164 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.279673 20239 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.280571 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:51.280966 20239 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:51.220552 20180 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.209824 20245 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.282260 20254 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:51.282766 20254 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.283133 20254 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:51.281840 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:51.297324 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:51.207283 20097 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:51.316072 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 75545.6us
I20260709 13:06:51.259809 20122 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:51.321700 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:51.265898 20225 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:51.235970 20182 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.078s
I20260709 13:06:51.339339 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:51.340015 13871 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 20: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:51.242416 20043 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.304072 20007 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:51.286484 20129 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.252842 20158 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.338627 19975 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:51.251935 19997 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.339800 20225 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:51.427068 20247 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.427524 20246 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:51.442298 13867 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:06:51.431326 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 134104us
I20260709 13:06:51.306550 20114 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.430938 13868 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
W20260709 13:06:51.438563 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 116906us
I20260709 13:06:51.390017 20007 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.338614 20248 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:51.447037 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:51.389523 20129 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:51.421598 20169 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:51.429947 20232 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.305s	user 0.002s	sys 0.000s
I20260709 13:06:51.421445 20180 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.421514 20075 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.455665 13871 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:51.447240 20237 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.370s	user 0.001s	sys 0.001s
W20260709 13:06:51.494505 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 270554us
I20260709 13:06:51.336503 20023 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.443034 20223 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:06:51.498312 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 432013us
I20260709 13:06:51.503127 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:51.490231 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 44720.4us
I20260709 13:06:51.454962 20007 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:51.507000 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.486151 20253 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.520586 13874 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:51.497925 20180 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:51.500947 20230 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:51.537040 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:51.448668 20247 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.525197 20251 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:51.506667 20075 consensus_queue.cc:1059] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:06:51.486188 20255 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.329s
I20260709 13:06:51.552486 19983 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.553145 20164 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.553053 20253 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:51.531625 20252 log.cc:927] Time spent T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:06:51.530583 20258 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:06:51.578605 13871 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:51.532449 20242 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:51.409281 13765 catalog_manager.cc:5697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 4 to 15. New cstate: current_term: 15 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:51.572856 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 69828.2us
I20260709 13:06:51.503706 20122 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:06:51.563383 20194 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:51.577629 13872 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:06:51.518466 19986 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:06:51.534020 20223 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:06:51.566277 19975 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:51.592267 13871 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:51.555889 20022 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:06:51.602672 13874 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:51.606086 13868 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:51.607756 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.608389 20261 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:51.608600 13866 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:51.608986 20261 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.612047 20261 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:51.612828 20266 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.613353 20091 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:51.614799 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:51.616101 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.617177 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:51.583298 20242 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:51.617333 20247 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:51.623684 20242 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:51.626550 20268 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:51.627041 20272 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:06:51.627281 20268 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.569633 20231 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:51.592373 20194 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:51.644582 19982 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.681s	user 0.009s	sys 0.004s
I20260709 13:06:51.646934 13765 catalog_manager.cc:5697] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
W20260709 13:06:51.649057 19964 consensus_meta.cc:287] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.749s	user 0.008s	sys 0.005s
I20260709 13:06:51.648535 13765 catalog_manager.cc:5697] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 10, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 10 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.650650 13871 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:06:51.650728 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:51.651132 13871 pending_rounds.cc:85] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 9
I20260709 13:06:51.651448 13871 pending_rounds.cc:107] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:06:51.651695 13871 raft_consensus.cc:2889] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:51.652118 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:51.652810 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:51.653301 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.655076 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.655294 20114 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:51.655730 13867 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:51.613585 20266 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.656666 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:51.658620 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.596107 19975 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:51.659904 19975 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:51.660570 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.661137 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 1079) took 1195 ms. Trace:
I20260709 13:06:51.661468 20230 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.661516 13950 rpcz_store.cc:276] 0709 13:06:50.465788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.469903 (+  4115us) service_pool.cc:224] Handling call
0709 13:06:51.660935 (+1191032us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b5400086454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:51.661077 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1363686784}
I20260709 13:06:51.662425 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.662220 20230 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.663305 13947 raft_consensus.cc:2410] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:06:51.663358 20235 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.664876 20230 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:51.649510 19964 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.665688 20235 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:51.669150 13841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.001s	sys 0.007s
W20260709 13:06:51.669503 13841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.001s	sys 0.007s
W20260709 13:06:51.659281 20101 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.656s	user 0.011s	sys 0.001s
I20260709 13:06:51.674787 13868 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:51.678041 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.678897 13947 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:51.604480 20097 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:51.692785 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:51.693306 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.695880 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:51.675071 20257 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.330s	user 0.000s	sys 0.002s
I20260709 13:06:51.666975 20194 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:51.596861 19668 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
W20260709 13:06:51.700268 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.705991 13914 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:51.703380 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:51.704392 13838 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:51.702620 13839 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:51.707722 13868 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:51.677464 20114 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:51.696749 20181 consensus_meta.cc:287] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.684s	user 0.011s	sys 0.002s
I20260709 13:06:51.739094 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.739200 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.667157 20266 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.674669 20158 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:51.743721 20218 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.872s	user 0.011s	sys 0.002s
I20260709 13:06:51.736991 13765 catalog_manager.cc:5697] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:51.746029 14018 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:51.746592 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.753717 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:51.692472 19824 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.757400 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.757601 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.696856 20043 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.764753 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.773136 14021 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:51.769004 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.734825 20097 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:51.769676 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:51.771746 14553 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:51.775355 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.781235 14019 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:51.765182 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.787678 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:51.693053 20101 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.788769 14554 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 10: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:51.790861 13950 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:06:51.790982 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.793663 13950 pending_rounds.cc:85] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 10
I20260709 13:06:51.794423 13950 pending_rounds.cc:107] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:06:51.794734 13950 raft_consensus.cc:2889] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:51.778770 20158 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:51.784063 20097 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:51.793615 20254 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.000s
I20260709 13:06:51.803784 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.796391 13945 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:51.826333 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:51.802735 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:51.796141 20075 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:51.792491 20238 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.002s
I20260709 13:06:51.801676 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.828464 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.832790 13875 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:51.838519 20238 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.839042 20274 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.839672 20274 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.841015 20238 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:51.842123 20244 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:51.842733 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:51.842873 20250 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:51.843343 19986 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:51.843797 19986 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:51.843590 20250 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.844281 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.844785 19986 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.845441 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:51.846576 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.846796 20245 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.847648 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:51.848410 20241 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.696s	user 0.002s	sys 0.003s
I20260709 13:06:51.842789 20244 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.849386 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:51.849985 20282 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:51.851223 20244 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:51.848716 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:51.851824 20111 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:51.852530 14019 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:51.854307 19964 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.854516 20075 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.856694 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 726) took 1115 ms. Trace:
I20260709 13:06:51.850319 20253 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:51.857304 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:51.751643 20181 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.765852 20114 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.870460 13872 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 20: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:51.797120 20281 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:51.875669 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.881660 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:51.883914 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 181341us
I20260709 13:06:51.857110 14017 rpcz_store.cc:276] 0709 13:06:50.741138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.745014 (+  3876us) service_pool.cc:224] Handling call
0709 13:06:50.792152 (+ 47138us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:50.828720 (+ 36568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:50.828769 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:50.829613 (+   844us) log.cc:844] Serialized 52 byte log entry
0709 13:06:50.943980 (+114367us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:50.944055 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.944188 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:51.847545 (+903357us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:51.856338 (+  8793us) raft_consensus.cc:1685] finished
0709 13:06:51.856363 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:51.856606 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35942,"raft.queue_time_us":58451,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":99,"spinlock_wait_cycles":10428288,"thread_start_us":113712,"threads_started":1,"wal-append.queue_time_us":211628}
I20260709 13:06:51.800848 20182 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.826351 20158 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.782347 19824 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.800988 20043 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:51.890348 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.780428 20218 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.891836 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.900830 13874 raft_consensus.cc:2368] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14.
W20260709 13:06:51.894904 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 153189us
I20260709 13:06:51.901790 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:51.837845 20097 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:51.861527 20274 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.826630 20223 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:06:51.915602 20223 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:51.916047 20223 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:51.916463 20223 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:51.894378 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.861340 20254 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.892673 19986 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:51.907101 20279 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:51.926697 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.889695 20253 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.901288 20158 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:51.879953 20250 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:51.910220 13943 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:51.910392 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.913367 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.894982 13765 catalog_manager.cc:5697] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:51.923098 13945 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:51.925976 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:51.908568 20114 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.888500 20122 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.909080 19824 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.946105 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.900951 20268 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:51.889779 19997 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:06:51.951689 13867 raft_consensus.cc:2368] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16.
I20260709 13:06:51.953478 20253 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.018738 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:52.020339 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:52.021003 14018 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:51.954807 20101 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.954945 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.955873 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:51.956557 20181 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:51.957479 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:51.959481 20254 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:51.960626 20225 consensus_meta.cc:287] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.001s
I20260709 13:06:51.965552 20097 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.917060 20043 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:51.896891 20182 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.926931 20279 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.970160 20218 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:51.969727 20242 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:51.973143 13871 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:51.973625 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:51.976727 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:52.069993 13874 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:51.978150 13948 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:06:51.978753 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:52.071113 13944 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:51.927413 13872 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:52.001736 13946 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:52.015049 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:52.012136 13840 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:52.019449 14023 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:52.024546 20253 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:52.057130 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:52.059581 13870 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
W20260709 13:06:52.063548 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 256541us
W20260709 13:06:52.063594 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 161940us
I20260709 13:06:52.075697 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:52.075985 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 61055.1us
W20260709 13:06:52.066627 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.076980 13989 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:52.073721 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:52.077689 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:52.078529 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:52.078831 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:52.079321 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:52.079630 13950 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 12: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:52.075874 13914 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:52.081519 13869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:06:52.082067 20111 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:52.082252 20261 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:52.082365 13869 raft_consensus.cc:606] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:52.020684 20250 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:06:52.082898 13869 raft_consensus.cc:614] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:52.084427 13869 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 17 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:52.075997 20268 consensus_queue.cc:1059] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:06:52.086100 20268 raft_consensus.cc:3055] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:52.086473 20268 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:52.086508 13765 catalog_manager.cc:5697] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 6 to 12, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 12 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:52.087563 20268 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:52.088901 20268 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:52.088575 13765 catalog_manager.cc:5697] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 14 to 16, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 16 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.091051 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:52.092536 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:52.093362 14016 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:52.096204 20043 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:52.098387 20182 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:52.102249 20114 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:52.102793 20114 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:52.103185 20114 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:52.077855 14023 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 11.
I20260709 13:06:52.104125 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:52.106329 20292 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:52.107496 20097 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:52.108187 20022 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:52.108894 20022 consensus_queue.cc:1243] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:06:52.108939 20283 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:52.109498 20283 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.109848 20283 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:52.110672 20111 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:52.110873 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:52.114151 14553 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:52.111706 20293 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:52.111784 20111 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.111737 20267 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:52.116185 13765 catalog_manager.cc:5697] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 14, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 14 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:52.119092 20097 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:06:52.123852 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 111793us
I20260709 13:06:52.124742 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:52.112972 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:52.114688 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:52.126622 14554 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:52.127817 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 48578us
W20260709 13:06:52.128588 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51690.7us
I20260709 13:06:52.128523 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:52.128926 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.129515 13989 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:52.129576 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:52.130993 14018 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:52.130971 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:52.133553 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:52.138092 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:52.021929 20207 ts_tablet_manager.cc:1929] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.12
I20260709 13:06:52.144223 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:52.143148 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:52.061919 20225 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.068730 20279 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:52.155114 13989 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:52.152046 13874 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 13 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:52.018573 20122 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.147380 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:52.166666 13943 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:52.145042 14020 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:52.185954 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:52.155843 14554 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:52.131664 20097 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:52.144287 20253 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:52.139724 20292 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:52.136852 20022 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:52.189877 13840 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:52.152594 19997 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:52.145601 19824 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:52.192256 13989 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:52.167416 19982 raft_consensus.cc:3055] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:52.151146 20293 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:52.179608 20239 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.898s	user 0.013s	sys 0.001s
I20260709 13:06:52.195078 13874 raft_consensus.cc:1240] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:52.162797 20295 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:52.163761 19782 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:52.199103 13947 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:52.179106 20075 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:52.197507 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 58137us
I20260709 13:06:52.210188 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:52.190963 20043 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:52.196153 20218 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:06:52.194089 20101 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:52.164819 20255 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:06:52.119269 13765 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 18 to 20, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 20 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.318320 13943 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 20: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:52.212505 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:52.197316 20292 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:52.198516 19983 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:52.246448 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.202714 20254 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:06:52.318857 20301 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 13:06:52.319729 20300 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:06:52.201452 19997 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:52.321353 20225 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:52.289722 20277 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.322750 20295 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:52.199674 20207 log.cc:1199] T 02dcf286c6bd48e3a6e548def7d786cc P 3779649036364744823fc035dac892c7: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-1-root/wals/02dcf286c6bd48e3a6e548def7d786cc
I20260709 13:06:52.323293 20298 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:52.324177 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:52.327405 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 115031us
W20260709 13:06:52.246940 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 46123us
I20260709 13:06:52.193534 20244 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:52.335662 13947 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:06:52.343978 19782 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:52.343719 20114 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.202898 19982 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:52.360729 20248 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:52.341809 13874 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:52.201267 19824 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:52.349169 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.363746 20254 raft_consensus.cc:697] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:52.215171 20280 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.348788 20292 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:52.354490 13943 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:52.246642 20268 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:06:52.317739 20043 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.201987 20267 consensus_queue.cc:1059] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.151s
I20260709 13:06:52.348508 20255 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:06:52.387974 19975 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.726s	user 0.012s	sys 0.001s
W20260709 13:06:52.362496 20091 consensus_meta.cc:287] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.749s	user 0.011s	sys 0.002s
W20260709 13:06:52.383329 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 59247.1us
I20260709 13:06:52.362653 19997 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:52.203549 20022 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:52.246428 20007 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.718s	user 0.006s	sys 0.007s
W20260709 13:06:52.385087 20194 consensus_meta.cc:287] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.626s	user 0.005s	sys 0.007s
I20260709 13:06:52.381191 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:52.416929 13989 leader_election.cc:400] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14.
I20260709 13:06:52.387409 20097 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:06:52.390722 20129 consensus_meta.cc:287] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.890s	user 0.014s	sys 0.000s
I20260709 13:06:52.411813 13873 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:52.393762 19824 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:52.397006 20043 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:52.385099 20277 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.434141 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:52.435351 13874 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:52.434819 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:52.392274 19782 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:52.450615 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:52.433612 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:52.506804 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:06:52.436570 19986 consensus_meta.cc:287] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.001s
I20260709 13:06:52.458109 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:52.510586 14553 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:52.510536 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 75864.7us
I20260709 13:06:52.513684 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:52.533164 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:52.530927 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 80417.7us
I20260709 13:06:52.432246 20239 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:52.400842 20280 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:52.551946 13944 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:52.435557 20097 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:52.573203 13989 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:52.407645 19975 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:52.521898 20304 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:06:52.449837 20114 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:52.395785 20248 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:52.454900 20253 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.575397 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:52.436897 20122 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:52.452932 20091 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:52.431878 20194 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.416450 20007 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:52.586607 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 184108us
I20260709 13:06:52.446223 19982 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.539628 20255 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:52.553493 20228 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:06:52.528019 20267 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:52.573778 20298 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:52.596082 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:52.579072 20097 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:52.453312 20120 consensus_queue.cc:1059] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.129s
I20260709 13:06:52.532171 20295 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.580214 20247 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:52.577119 20254 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:52.455127 20129 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.582887 20218 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:52.446847 20022 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.583302 19983 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:52.737916 20228 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:52.709651 20225 consensus_queue.cc:1059] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:06:52.678287 14024 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:52.597856 20239 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:52.704103 20023 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:52.739609 20120 raft_consensus.cc:3055] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:06:52.740940 20288 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.792s	user 0.002s	sys 0.004s
I20260709 13:06:52.741324 20231 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.741644 20007 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:52.615551 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:52.748044 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 169527us
I20260709 13:06:52.742758 20218 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:52.695072 20303 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
W20260709 13:06:52.764042 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 329998us
I20260709 13:06:52.704164 20293 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:52.695942 20275 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Append to log took a long time: real 1.006s	user 0.000s	sys 0.003s
I20260709 13:06:52.692396 20207 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36032 (request call id 0) took 2904 ms. Trace:
I20260709 13:06:52.635789 20101 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.742558 19982 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:52.693872 20280 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:52.737963 20247 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:52.697359 20306 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:06:52.745741 20267 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:52.737478 20298 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:52.782615 14017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:06:52.783130 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:52.820999 14017 raft_consensus.cc:606] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:06:52.821426 14017 raft_consensus.cc:614] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:52.786911 14524 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:52.789223 20218 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:52.601708 20122 consensus_queue.cc:1059] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.169s
I20260709 13:06:52.601366 20253 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:52.636849 20268 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.174s
I20260709 13:06:52.506920 13765 catalog_manager.cc:5697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 8 to 11, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 11 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:52.765652 20255 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.804339 14553 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 12.
I20260709 13:06:52.769644 20120 raft_consensus.cc:740] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:52.786170 20231 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:52.809732 20101 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:52.812094 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:52.824391 20298 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:52.769156 20292 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:52.743840 20111 consensus_queue.cc:1175] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e1649107c781428692458f10597c5742" responder_term: 13 status { last_received { term: 11 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: [12.8-12.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:52.800156 20239 raft_consensus.cc:3055] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:52.812006 19997 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.742775 20091 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:52.828945 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 231059us
I20260709 13:06:52.763094 19983 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:52.868762 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:52.839957 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 53132us
I20260709 13:06:52.843410 20230 raft_consensus.cc:2764] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:06:52.822897 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:52.841342 20309 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:52.855518 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3152) took 1053 ms. Trace:
I20260709 13:06:52.824416 20267 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:52.834255 20223 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.917s	user 0.015s	sys 0.000s
I20260709 13:06:52.700508 19782 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:52.814193 20247 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:52.790472 19986 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:06:52.856711 20281 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.955s	user 0.011s	sys 0.002s
I20260709 13:06:52.871831 13943 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:52.824833 20280 raft_consensus.cc:2764] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:06:52.873847 20114 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.826459 20228 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:52.907809 20311 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:06:52.908439 20228 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:52.910185 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:52.911703 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:52.912318 13867 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:52.913304 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:52.914529 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:52.914598 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3196) took 1283 ms. Trace:
I20260709 13:06:52.915078 13871 rpcz_store.cc:276] 0709 13:06:51.631605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.652588 (+ 20983us) service_pool.cc:224] Handling call
0709 13:06:52.914366 (+1261778us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54000d9654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:52.914512 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734760320}
I20260709 13:06:52.915794 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:52.916321 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:52.916468 20248 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:52.916913 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:52.876281 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 54189us
I20260709 13:06:52.917598 13867 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:52.917668 13866 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:52.856917 13873 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:52.931294 13840 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:52.879213 13838 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:52.883936 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 894) took 1083 ms. Trace:
I20260709 13:06:52.893018 20316 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:52.934803 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:52.935920 13944 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:52.936239 13867 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:52.879076 14022 rpcz_store.cc:276] 0709 13:06:51.802204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.852915 (+ 50711us) service_pool.cc:224] Handling call
0709 13:06:51.853307 (+   392us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:51.889693 (+ 36386us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:51.911078 (+ 21385us) log.cc:844] Serialized 14 byte log entry
0709 13:06:52.247721 (+336643us) spinlock_profiling.cc:243] Waited 7.63 ms on lock 0x7b18003da6e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd323a 00007fb2c343da2a
0709 13:06:52.413919 (+166198us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:52.512293 (+ 98374us) log.cc:844] Serialized 78 byte log entry
0709 13:06:52.512547 (+   254us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:52.512815 (+   268us) log.cc:844] Serialized 14 byte log entry
0709 13:06:52.695837 (+183022us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b18003da6e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c2fe9d70 00007fb2c2fdd293 00007fb2c2fd2cea 00007fb2c2fd323a 00007fb2c343da2a 00007fb2c344d0bf 00007fb2c344d038
0709 13:06:52.709265 (+ 13428us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b48002115d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:06:52.854853 (+145588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.854967 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:52.855013 (+    46us) raft_consensus.cc:1685] finished
0709 13:06:52.855027 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.855198 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45406,"prepare.queue_time_us":47697,"prepare.run_cpu_time_us":8372,"prepare.run_wall_time_us":20361,"raft.queue_time_us":59363,"raft.run_cpu_time_us":2602,"raft.run_wall_time_us":318898,"spinlock_wait_cycles":127677568,"thread_start_us":310756,"threads_started":2,"wal-append.queue_time_us":532462}
I20260709 13:06:52.859853 20239 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:52.882905 20122 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:52.907269 20114 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.916997 13871 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:52.876197 19983 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:52.918532 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2899) took 1258 ms. Trace:
I20260709 13:06:52.918790 19986 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:52.920132 20242 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:52.921519 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:52.923250 20292 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:52.926743 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3273) took 1757 ms. Trace:
I20260709 13:06:52.930311 20194 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:52.928278 20244 raft_consensus.cc:3055] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:52.874524 20272 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:06:52.872774 19975 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:52.937150 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58030us
I20260709 13:06:52.947983 20225 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:52.997681 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:52.987799 13838 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:52.999418 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:52.934473 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:52.938254 20280 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:52.938158 20316 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:52.938735 20180 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:52.939213 19782 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:52.939666 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:52.799609 20207 rpcz_store.cc:276] 0709 13:06:49.739120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.739719 (+   599us) service_pool.cc:224] Handling call
0709 13:06:49.892321 (+152602us) ts_tablet_manager.cc:1151] Deleting tablet 02dcf286c6bd48e3a6e548def7d786cc
0709 13:06:49.914340 (+ 22019us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:06:51.008796 (+1094456us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:52.006832 (+998036us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:52.640923 (+634091us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32665,"tablet-delete.queue_time_us":101416,"thread_start_us":43073,"threads_started":1}
I20260709 13:06:52.884536 20282 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:52.851735 20255 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.945359 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:53.040889 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:53.041546 13873 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:53.042173 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:52.932216 13872 rpcz_store.cc:276] 0709 13:06:51.803234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.133365 (+330131us) service_pool.cc:224] Handling call
0709 13:06:52.133746 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:52.141844 (+  8098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:52.141874 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:52.142443 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:06:52.395765 (+253322us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:06:52.395810 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.395880 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:52.883684 (+487804us) raft_consensus.cc:1685] finished
0709 13:06:52.883709 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.883869 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7709,"raft.queue_time_us":11697,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":25908,"thread_start_us":202859,"threads_started":1,"wal-append.queue_time_us":273058}
I20260709 13:06:52.946965 13867 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:52.899210 20075 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.882017 20235 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:06:52.904119 20281 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:52.956005 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:53.057161 14553 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
W20260709 13:06:53.006464 20286 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.002s	sys 0.011s
I20260709 13:06:52.950285 13871 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:52.962044 13875 rpcz_store.cc:276] 0709 13:06:51.169258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.169404 (+   146us) service_pool.cc:224] Handling call
0709 13:06:51.169718 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:51.859117 (+689399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.926415 (+1067298us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b54000a6c54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:52.926639 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1524330368}
I20260709 13:06:52.927521 20267 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:52.963474 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3171) took 1743 ms. Trace:
W20260709 13:06:52.944057 20158 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.926s	user 0.013s	sys 0.000s
I20260709 13:06:52.957943 20122 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:52.972350 19986 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:52.973352 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:52.974860 20231 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:52.975816 20101 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:53.059528 20286 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.004s	sys 0.015s
W20260709 13:06:52.990156 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 119800us
I20260709 13:06:53.072960 13914 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:52.951668 20313 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:53.086576 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:52.976572 19983 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:53.087858 13874 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 20.
W20260709 13:06:53.005787 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 137473us
I20260709 13:06:53.088618 13839 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:53.090121 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:53.089735 14522 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:52.931116 20120 consensus_queue.cc:260] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:53.101629 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:53.102146 20120 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:53.102542 14561 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:53.103650 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:53.011119 20225 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:53.045977 13867 pending_rounds.cc:85] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 11
I20260709 13:06:53.068094 20274 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.068866 20319 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:53.106314 19824 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.594s	user 0.008s	sys 0.004s
I20260709 13:06:52.951236 13948 rpcz_store.cc:276] 0709 13:06:51.660181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.660314 (+   133us) service_pool.cc:224] Handling call
0709 13:06:52.918343 (+1258029us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b54001487d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:52.918472 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":728157952}
I20260709 13:06:53.005300 20242 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:53.007920 20180 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.007793 20295 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:53.059424 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.025555 19975 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:53.052470 20279 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.861s	user 0.014s	sys 0.000s
I20260709 13:06:53.065282 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.069084 14559 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:53.062830 13941 rpcz_store.cc:276] 0709 13:06:51.213905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.214079 (+   174us) service_pool.cc:224] Handling call
0709 13:06:51.236505 (+ 22426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:51.420188 (+183683us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:51.441973 (+ 21785us) log.cc:844] Serialized 14 byte log entry
0709 13:06:51.777554 (+335581us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:51.777637 (+    83us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:51.777711 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.956979 (+1179268us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400229a54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:52.957057 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.957249 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20422,"prepare.run_cpu_time_us":8397,"prepare.run_wall_time_us":35225,"raft.queue_time_us":72443,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":354675840,"thread_start_us":324857,"threads_started":2,"wal-append.queue_time_us":176472,"wal-append.run_cpu_time_us":1152,"wal-append.run_wall_time_us":1017438}
I20260709 13:06:52.893581 20223 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.074170 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:52.904477 13765 catalog_manager.cc:5697] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 6 to 13, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 13 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:53.083933 20122 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:53.084468 20320 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:53.086372 20313 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:52.978873 20244 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:53.045914 13872 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:06:53.147740 19997 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:06:53.098888 20043 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.567s	user 0.013s	sys 0.000s
I20260709 13:06:53.100011 19986 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:53.100963 14556 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:53.100878 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:53.107101 13867 pending_rounds.cc:107] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:06:53.206027 13867 raft_consensus.cc:2889] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:53.173254 20194 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.051497 13942 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 10: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:06:53.054173 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 88002.5us
I20260709 13:06:53.208148 20194 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.209084 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:53.056811 20075 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.211107 20255 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:53.211243 20114 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.157949 19824 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.211918 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:53.008059 20239 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.134346 13757 catalog_manager.cc:5028] TS 3779649036364744823fc035dac892c7 (127.12.109.66:43953): tablet 02dcf286c6bd48e3a6e548def7d786cc (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:06:53.139379 20316 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.106235 20281 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:53.087875 20267 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:53.122500 20274 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:53.055514 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44507.2us
I20260709 13:06:53.095546 20282 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:53.243306 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:53.159317 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.244135 13948 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:53.068305 20158 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:53.157475 13840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.652s (SENT) [suppressed 11 similar messages]
W20260709 13:06:53.254279 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 96823.3us
W20260709 13:06:53.147945 20283 consensus_meta.cc:287] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.038s	user 0.011s	sys 0.004s
I20260709 13:06:53.164934 20247 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.164893 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.172298 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.270200 13941 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 20.
I20260709 13:06:53.183992 14559 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:53.172832 13950 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:53.271284 14553 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:53.161082 20180 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:53.207747 19986 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:53.210187 19983 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:53.128672 20319 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:53.163301 19975 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.212945 13873 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:06:53.209102 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:53.214713 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:53.160933 20242 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:53.160827 20317 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:53.215667 13757 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 142222us
I20260709 13:06:53.228647 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:53.160632 20324 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:53.295629 14019 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:53.297288 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:53.297621 14555 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:53.230221 14016 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:53.233074 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 893) took 1433 ms. Trace:
I20260709 13:06:53.272490 20223 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.092449 20231 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:53.246212 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:53.309780 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:53.311513 14020 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 12.
I20260709 13:06:53.312005 13941 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:53.245098 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:53.147419 20314 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:06:53.256021 13838 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:53.162973 13946 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 20.
I20260709 13:06:53.171129 20295 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:53.172590 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:53.314709 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58775.9us
I20260709 13:06:53.271829 13945 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:53.163604 20279 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.266969 13988 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:53.167501 20298 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.275678 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:53.197264 20320 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.284034 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:53.239928 20075 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.296718 20022 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:06:53.288765 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 74126.6us
I20260709 13:06:53.331905 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:53.332659 14020 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 20.
I20260709 13:06:53.214076 20043 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.257378 20254 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.256805 20255 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.254740 20272 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.238585 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:53.306207 19782 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:53.306433 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:53.307322 20324 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:53.405506 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:53.308538 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.298600 13869 rpcz_store.cc:276] 0709 13:06:51.799867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.085251 (+285384us) service_pool.cc:224] Handling call
0709 13:06:52.085676 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.085919 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:52.127396 (+ 41477us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.067334 (+939938us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b180040f7e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd323a 00007fb2c343da2a
0709 13:06:53.232573 (+165239us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.232661 (+    88us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:53.232735 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.232829 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.232998 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17827,"prepare.queue_time_us":41217,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":5082,"raft.queue_time_us":101,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":7597,"spinlock_wait_cycles":10900864,"thread_start_us":163234,"threads_started":1,"wal-append.queue_time_us":1166948}
I20260709 13:06:53.310505 14556 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
W20260709 13:06:53.248641 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 173766us
I20260709 13:06:53.313401 13948 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:06:53.316581 14017 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 13:06:53.294070 20023 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.002s
I20260709 13:06:53.308539 20158 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.321372 14561 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
I20260709 13:06:53.322852 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:53.327595 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 60766.6us
I20260709 13:06:53.417167 14019 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:06:53.325937 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2611) took 1593 ms. Trace:
W20260709 13:06:53.328703 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44775.5us
I20260709 13:06:53.291769 20194 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:53.237973 20267 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.351819 19983 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.381203 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:53.430610 14020 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:53.382848 20298 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.383798 20225 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:53.460052 20331 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:06:53.460309 20277 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.461906 20313 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:53.458896 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3186) took 1260 ms. Trace:
I20260709 13:06:53.463317 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:53.469028 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:53.469785 13873 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:53.471201 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:53.472914 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:53.409503 20253 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:53.479765 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:53.480208 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.481011 14021 raft_consensus.cc:2410] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 18 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:06:53.321017 20231 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:53.391330 20242 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.395043 20319 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.320228 19986 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:53.400650 14023 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:53.258919 20282 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:53.474599 20322 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.302s	user 0.001s	sys 0.001s
I20260709 13:06:53.324018 20293 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:53.280414 20283 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:53.411280 13942 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:53.412492 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:53.510246 13948 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:53.413894 13838 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:53.415997 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:53.417997 14024 rpcz_store.cc:276] 0709 13:06:51.721433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.721747 (+   314us) service_pool.cc:224] Handling call
0709 13:06:53.314751 (+1593004us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b5400411754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:53.314907 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2081622784}
I20260709 13:06:53.420812 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:53.513446 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 92717.6us
I20260709 13:06:53.423646 14015 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:53.378075 20043 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:53.432263 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.432307 20194 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:53.441365 14020 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:06:53.442963 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:53.529783 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:53.531013 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 88162.2us
I20260709 13:06:53.443686 20158 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:53.446889 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.451514 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:53.278445 20244 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:53.456012 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:53.554729 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 103345us
I20260709 13:06:53.458825 20324 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:53.555692 13914 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:53.459252 13947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2913) took 1502 ms (client timeout 1881 ms). Trace:
I20260709 13:06:53.477267 13941 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:53.464210 13943 rpcz_store.cc:276] 0709 13:06:52.198929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.397114 (+198185us) service_pool.cc:224] Handling call
0709 13:06:52.400732 (+  3618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.895412 (+494680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.458606 (+563194us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b54003309d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:53.458826 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1082871040}
I20260709 13:06:53.401804 20254 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.387646 20247 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:53.329810 20218 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.398664 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:53.492321 14016 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:53.409672 20334 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:53.586517 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:53.509640 20293 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.478544 20225 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:53.501458 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:53.406332 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:53.551661 20319 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:53.523625 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.503803 19983 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.568881 20223 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:53.577564 20075 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:06:53.529350 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 115566us
I20260709 13:06:53.541991 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:53.408094 20272 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.491792 20291 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.615024 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.554844 13946 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:53.456562 20335 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:53.558794 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:53.556551 13947 rpcz_store.cc:269] 0709 13:06:51.957007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.957196 (+   189us) service_pool.cc:224] Handling call
0709 13:06:53.459079 (+1501883us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400092254. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:53.459191 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330794752}
I20260709 13:06:53.655367 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.655591 20291 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.493142 19975 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.408275 20255 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:53.567239 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.657491 13943 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:53.565970 20158 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.579094 13949 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:06:53.583492 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:53.591653 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 176465us
W20260709 13:06:53.595376 14560 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.500s	user 0.012s	sys 0.000s
I20260709 13:06:53.523785 20278 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.596669 14021 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:53.609174 20274 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.523020 20023 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:53.598569 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:53.672384 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.672600 14024 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:53.599992 14551 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:53.428292 20129 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.560117 20253 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:53.551993 20309 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:53.695286 20317 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:53.696106 20346 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:06:53.697685 14016 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:53.602948 13942 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:53.464059 19824 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.615276 20075 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:53.605096 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 72554.1us
I20260709 13:06:53.497299 14022 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 10.
I20260709 13:06:53.525681 13915 leader_election.cc:400] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16.
I20260709 13:06:53.611986 13838 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:53.600139 20293 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:53.707454 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:53.634788 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.602696 20319 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.711774 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:53.712936 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:53.653424 13944 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 20.
I20260709 13:06:53.656288 13947 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:53.559823 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:53.575824 13873 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:53.658515 13945 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:53.659492 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:53.603749 20277 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:53.610108 20225 consensus_queue.cc:1243] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:06:53.609804 20223 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.609506 20043 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:53.669296 14560 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:06:53.679862 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.687739 20254 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.689162 20282 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:53.602200 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 184369us
W20260709 13:06:53.604454 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 103136us
I20260709 13:06:53.701885 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:53.706396 13915 leader_election.cc:403] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:06:53.706691 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 94766.2us
I20260709 13:06:53.742869 14024 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
W20260709 13:06:53.748349 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.750316 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:53.753118 14023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2631) took 1719 ms (client timeout 2063 ms). Trace:
I20260709 13:06:53.754225 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:53.708534 13941 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:53.706729 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:53.718901 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:53.720082 13868 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:53.720111 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.720095 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:53.717655 13869 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:53.689913 20295 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:53.756824 14022 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:53.758558 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:53.761693 14024 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:06:53.757305 13873 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
W20260709 13:06:53.753592 14023 rpcz_store.cc:269] 0709 13:06:52.033426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.105128 (+ 71702us) service_pool.cc:224] Handling call
0709 13:06:53.752914 (+1647786us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540029a4d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:53.753056 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2955767168}
W20260709 13:06:53.764185 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 238690us
I20260709 13:06:53.765713 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:53.763371 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:53.771353 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.772850 13873 raft_consensus.cc:2368] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18.
I20260709 13:06:53.756361 13838 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:53.774432 13988 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:53.774638 13942 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:53.708266 20328 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:53.799901 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 43607.5us
I20260709 13:06:53.777204 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:53.803679 13942 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:53.802989 14023 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:53.795431 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:53.914084 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:53.914433 13868 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 20.
I20260709 13:06:53.914746 13942 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:53.915560 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:53.915844 13942 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:53.916671 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:53.724447 20120 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.622s	user 0.012s	sys 0.002s
I20260709 13:06:53.917258 13868 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:53.917897 20120 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:53.918450 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:53.726615 20228 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:53.866568 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:53.919461 13868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "765348f2471a493d92e998132c0c746b"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:06:53.919726 14022 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:06:53.920004 13868 raft_consensus.cc:606] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:06:53.920337 13868 raft_consensus.cc:614] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:53.920608 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:53.921633 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:06:53.921707 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:53.922412 14022 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 20.
I20260709 13:06:53.743284 19824 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:53.867015 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:53.721567 20319 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:53.924054 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:53.924408 14560 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:06:53.925053 13942 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:53.720582 20043 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:53.803580 13944 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:06:53.804322 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:53.754731 20225 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:53.682006 20278 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.664103 20335 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.684804 20023 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.794292 20114 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:53.798898 20282 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16.
W20260709 13:06:53.869247 20332 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.442s	user 0.001s	sys 0.001s
W20260709 13:06:53.893312 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.847500 20254 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:53.816390 20279 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:53.744134 20277 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:53.868680 20231 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.931223 20298 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:53.724093 20223 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:53.751637 20129 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:53.935806 20218 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:53.985340 13948 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
W20260709 13:06:53.928898 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 151490us
I20260709 13:06:53.911585 20247 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:53.940678 20114 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.960875 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 900) took 1517 ms. Trace:
I20260709 13:06:53.988935 13874 rpcz_store.cc:276] 0709 13:06:52.443093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.579672 (+136579us) service_pool.cc:224] Handling call
0709 13:06:53.960706 (+1381034us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b540026bbd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:53.960811 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":805017344}
W20260709 13:06:53.962689 20338 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
I20260709 13:06:53.967340 20279 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:53.866885 20328 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:53.898751 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 64708.4us
I20260709 13:06:53.930698 20043 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:53.957860 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:54.000001 14017 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
W20260709 13:06:53.979564 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 222906us
W20260709 13:06:53.992659 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 290843us
I20260709 13:06:53.975046 20158 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:53.893299 20075 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:53.975782 20254 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:53.979781 20298 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.977332 20351 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:53.993798 20114 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:53.998610 20336 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.387s	user 0.000s	sys 0.001s
W20260709 13:06:53.997841 20097 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.247s	user 0.014s	sys 0.002s
I20260709 13:06:54.073046 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:53.948700 20225 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.061733 13946 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:06:54.013049 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:54.058930 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:54.081754 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 81968.2us
I20260709 13:06:54.000149 19983 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:54.089370 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 331754us
I20260709 13:06:54.091977 14017 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 22: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:54.092602 13838 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:54.093545 14017 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:54.094108 20281 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:54.094511 20281 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.094861 20281 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:54.093549 13838 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:54.101202 20348 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:54.102003 20348 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:54.103209 20348 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.108285 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:54.110075 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:54.111469 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:54.100400 13838 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:54.108134 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:54.113413 20359 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:54.113996 20359 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:54.115191 20359 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:53.993731 20129 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:54.004472 20279 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:54.074280 20272 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:54.130749 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 71979us
I20260709 13:06:54.150352 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:54.076668 20298 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:54.074105 20022 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.040555 20239 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:54.086284 20342 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.383s	user 0.000s	sys 0.001s
I20260709 13:06:54.079829 20291 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:54.154186 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.170394 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3305) took 1527 ms. Trace:
W20260709 13:06:54.130121 20332 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:06:54.036073 20282 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 13:06:54.088409 20158 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:06:54.039964 19975 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.107s
W20260709 13:06:54.162045 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 149166us
I20260709 13:06:54.077380 20274 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:06:54.145572 19983 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:54.146332 19824 raft_consensus.cc:697] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:54.174873 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:54.081243 20349 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 13:06:54.170848 13867 rpcz_store.cc:276] 0709 13:06:52.643227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.045348 (+402121us) service_pool.cc:224] Handling call
0709 13:06:53.045728 (+   380us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:53.206876 (+161148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.206914 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:53.207599 (+   685us) log.cc:844] Serialized 52 byte log entry
0709 13:06:53.326338 (+118739us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:53.326404 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.326507 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.170123 (+843616us) raft_consensus.cc:1685] finished
0709 13:06:54.170148 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.170326 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2453,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":118254,"threads_started":1,"wal-append.queue_time_us":384793}
I20260709 13:06:54.169516 20247 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:54.169924 20344 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:54.192512 14017 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:54.088909 20097 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.200776 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3306) took 1557 ms. Trace:
I20260709 13:06:54.086587 20351 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.197157 13989 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:54.166749 20022 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.201367 13875 rpcz_store.cc:276] 0709 13:06:52.649962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.061563 (+411601us) service_pool.cc:224] Handling call
0709 13:06:53.061930 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.062358 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.062395 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.071158 (+  8763us) log.cc:844] Serialized 26 byte log entry
0709 13:06:53.278591 (+207433us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:06:53.297089 (+ 18498us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.299041 (+  1952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.299133 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.200492 (+901359us) raft_consensus.cc:1685] finished
0709 13:06:54.200513 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.200687 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1636,"prepare.queue_time_us":18284,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":14120,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":4322,"thread_start_us":155047,"threads_started":1,"wal-append.queue_time_us":331488}
I20260709 13:06:54.276458 13943 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:54.206457 19975 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:54.204173 20267 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:54.199801 20324 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.279798 20247 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:06:54.288965 20340 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
W20260709 13:06:54.292791 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 100646us
I20260709 13:06:54.330600 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:54.267710 20268 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.331902 20268 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:54.200307 20344 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.334755 20360 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:54.334417 19824 consensus_queue.cc:237] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.335413 20360 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:54.336052 20180 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.053s	user 0.004s	sys 0.008s
I20260709 13:06:54.277827 20335 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:54.188572 20254 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:54.316478 14524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.003s	sys 0.006s
I20260709 13:06:54.197322 20279 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:54.326895 20228 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:54.290556 20253 consensus_meta.cc:287] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.599s	user 0.012s	sys 0.000s
I20260709 13:06:54.307860 13948 raft_consensus.cc:2410] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:54.287117 19975 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.336515 20180 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.299364 20274 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:06:54.367509 14524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.004s	sys 0.006s
W20260709 13:06:54.382879 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 628693us
I20260709 13:06:54.323088 20309 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:54.306524 20129 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:54.371039 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3292) took 2768 ms. Trace:
I20260709 13:06:54.339444 20247 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:54.391063 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:54.432147 13870 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:06:54.429320 13839 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:54.374763 20228 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:54.330112 20255 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:54.378959 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:54.387673 13866 rpcz_store.cc:276] 0709 13:06:51.602205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.076890 (+474685us) service_pool.cc:224] Handling call
0709 13:06:52.917478 (+840588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.930708 (+ 13230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.370778 (+1440070us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54000d9650. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:54.370962 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1786476928}
W20260709 13:06:54.372195 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.418610 20091 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:54.368969 20282 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:54.421444 20023 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:54.380205 19975 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:54.460383 20372 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:06:54.426244 20328 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:54.462572 20351 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:54.463055 19782 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:54.389243 13873 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:54.336922 20324 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:54.432421 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 48604.9us
W20260709 13:06:54.374908 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 44449.9us
I20260709 13:06:54.443753 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:54.378341 20253 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:54.445441 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:54.388859 20309 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:54.384155 20274 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:54.451932 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 254887us
W20260709 13:06:54.453135 20369 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.108s	user 0.001s	sys 0.001s
W20260709 13:06:54.454648 20368 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:06:54.458468 20312 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:54.459030 20097 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:54.464658 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3315) took 1121 ms. Trace:
I20260709 13:06:54.523452 13873 rpcz_store.cc:276] 0709 13:06:53.343258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.388590 (+1045332us) service_pool.cc:224] Handling call
0709 13:06:54.389044 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.464205 (+ 75161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.464573 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:54.493462 14016 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
W20260709 13:06:54.445617 20319 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.521s	user 0.013s	sys 0.000s
I20260709 13:06:54.534643 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:54.439682 20225 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:54.552654 14524 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:54.511073 13839 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:54.512508 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:54.463513 19782 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.554618 14023 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:54.534720 20319 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:54.536056 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3213) took 1665 ms. Trace:
I20260709 13:06:54.471603 20360 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:54.528220 20357 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:54.527056 14552 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:54.527571 20349 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:54.529438 20247 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.568830 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:54.538813 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:54.472373 20158 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.525839 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:54.573755 14016 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:54.574770 20158 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.561568 20248 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:54.576659 20248 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:54.577033 20248 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:54.534680 20022 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:54.579949 20328 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:54.580585 20328 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.543466 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:54.582229 20360 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.582387 13946 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:06:54.582775 20360 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.569746 20309 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:54.583575 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:54.571482 20349 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:54.537724 20382 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.584940 20360 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:54.584822 20382 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.585474 14016 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:54.587007 20382 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:54.563303 20353 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:54.589439 20353 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.589797 20353 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:54.599005 14558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 848) took 1513 ms. Trace:
I20260709 13:06:54.599653 14558 rpcz_store.cc:276] 0709 13:06:53.085801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.105887 (+ 20086us) service_pool.cc:224] Handling call
0709 13:06:53.107849 (+  1962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.523941 (+416092us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:06:53.742739 (+218798us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.807188 (+ 64449us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.807285 (+    97us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:53.807361 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.598583 (+791222us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54006b2e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:54.598668 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.598884 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":419139,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":134937,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":31934,"spinlock_wait_cycles":956031872,"thread_start_us":222234,"threads_started":2,"wal-append.queue_time_us":228311}
I20260709 13:06:54.567791 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:54.603070 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3310) took 1951 ms. Trace:
I20260709 13:06:54.603688 13870 rpcz_store.cc:276] 0709 13:06:52.651373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.091265 (+439892us) service_pool.cc:224] Handling call
0709 13:06:53.118610 (+ 27345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:54.432727 (+1314117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.602743 (+170016us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54003acad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:54.602985 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3012288896}
I20260709 13:06:54.560730 19997 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.605322 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:54.605466 19997 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:54.556067 13872 rpcz_store.cc:276] 0709 13:06:52.870286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.213777 (+343491us) service_pool.cc:224] Handling call
0709 13:06:53.214143 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.216598 (+  2455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.216653 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.217333 (+   680us) log.cc:844] Serialized 26 byte log entry
0709 13:06:53.653018 (+435685us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:53.686006 (+ 32988us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.700322 (+ 14316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.700406 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.535814 (+835408us) raft_consensus.cc:1685] finished
0709 13:06:54.535846 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.535988 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19832,"prepare.queue_time_us":25299,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":16188,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":16040,"thread_start_us":381532,"threads_started":1,"wal-append.queue_time_us":521244}
I20260709 13:06:54.608457 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:54.608614 19997 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:54.563196 20274 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.609773 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:54.609969 20274 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:54.610746 20291 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:54.611110 20253 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:54.611368 20291 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:54.611744 20253 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.612565 20291 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.613780 20370 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.546165 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:54.537114 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:54.595538 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:54.569839 13944 raft_consensus.cc:2368] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13.
I20260709 13:06:54.544584 20312 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:54.576155 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:54.584201 13946 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:54.586971 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:54.560911 19782 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:54.616855 14555 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 10.
I20260709 13:06:54.591200 20357 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:54.592063 20320 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:54.597313 14018 raft_consensus.cc:2410] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:06:54.597529 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:54.509702 13941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.010s	sys 0.035s
I20260709 13:06:54.533957 20043 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.568383 13839 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:54.641578 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:54.544095 13840 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.028s	sys 0.035s
W20260709 13:06:54.642441 13840 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.029s	sys 0.035s
W20260709 13:06:54.642951 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 665812us
I20260709 13:06:54.630906 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:54.572109 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:54.646842 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 74848.2us
I20260709 13:06:54.615485 13942 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:54.620030 14016 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:54.624684 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:54.626657 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:54.648857 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:54.649443 14016 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:06:54.649657 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:54.651144 13987 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:54.629077 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:54.631114 13941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.011s	sys 0.035s
I20260709 13:06:54.654000 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:54.658284 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:54.655303 13989 leader_election.cc:400] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18.
I20260709 13:06:54.658957 13989 leader_election.cc:403] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:54.660755 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:54.661149 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:54.661808 13872 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:54.662837 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:54.613994 20291 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:54.664776 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:54.661705 13870 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 23.
I20260709 13:06:54.673413 13987 leader_election.cc:400] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13.
I20260709 13:06:54.673606 13870 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:06:54.673995 13987 leader_election.cc:403] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:06:54.675704 13870 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:54.676743 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:06:54.630290 20281 consensus_meta.cc:287] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.535s	user 0.012s	sys 0.001s
I20260709 13:06:54.678814 13866 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:06:54.680371 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 49538.3us
I20260709 13:06:54.680879 14017 raft_consensus.cc:2410] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:06:54.681455 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:54.683197 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:54.642505 20382 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:54.685904 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 117895us
I20260709 13:06:54.694231 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:54.694952 14017 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:06:54.687716 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:54.691047 14016 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
W20260709 13:06:54.665163 20293 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.939s	user 0.014s	sys 0.000s
I20260709 13:06:54.706248 13838 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:54.663480 20239 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:54.663981 20346 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:54.647003 20111 consensus_queue.cc:1059] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:06:54.720130 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:54.685354 19997 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:54.720927 20091 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:54.724043 20376 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:54.717679 20239 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:54.700600 20313 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18.
I20260709 13:06:54.725796 20359 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:54.710847 20388 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:54.679406 20225 consensus_queue.cc:1059] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:06:54.726460 20361 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:06:54.728405 20043 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:54.728636 20357 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:54.730000 20351 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:54.693204 20120 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:54.614483 20370 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.701157 20272 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:54.646534 20023 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:06:54.695973 20254 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.739249 20091 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.740373 20324 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:54.685177 20320 consensus_queue.cc:1059] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:06:54.715029 20293 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.746160 20253 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:54.746824 20328 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:54.718132 20317 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.826308 20324 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:54.827008 20324 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:54.710402 20367 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:54.826586 20317 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:54.829295 20367 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.710609 20267 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13.
I20260709 13:06:54.710155 20318 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:54.714762 20194 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:54.829954 20367 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:54.831522 20317 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:54.742626 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3216) took 1486 ms. Trace:
I20260709 13:06:55.639222 13874 rpcz_store.cc:276] 0709 13:06:53.256376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.990037 (+733661us) service_pool.cc:224] Handling call
0709 13:06:53.990444 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.999165 (+  8721us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:54.039806 (+ 40641us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.580627 (+540821us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.581633 (+  1006us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:54.582190 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:06:54.582331 (+   141us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:54.582363 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.582438 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.742328 (+159890us) raft_consensus.cc:1685] finished
0709 13:06:54.742364 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.742548 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11596,"prepare.queue_time_us":73769,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":5655,"raft.queue_time_us":56507,"raft.run_cpu_time_us":574,"raft.run_wall_time_us":109852,"spinlock_wait_cycles":6869248,"thread_start_us":414508,"threads_started":1,"wal-append.queue_time_us":570736}
W20260709 13:06:54.763445 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.761220 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:54.737437 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 49794.5us
I20260709 13:06:55.849380 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:55.851068 20280 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:54.757536 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3214) took 1887 ms. Trace:
W20260709 13:06:54.754987 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.763949 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:55.957525 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3174) took 1225 ms. Trace:
I20260709 13:06:55.958130 14017 rpcz_store.cc:276] 0709 13:06:54.732469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.732947 (+   478us) service_pool.cc:224] Handling call
0709 13:06:55.957437 (+1224490us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.959487 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:55.960373 14017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3181) took 1210 ms (client timeout 1553 ms). Trace:
W20260709 13:06:55.960815 14017 rpcz_store.cc:269] 0709 13:06:54.750507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.959131 (+1208624us) service_pool.cc:224] Handling call
0709 13:06:55.960309 (+  1178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.961591 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:54.853574 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3313) took 1511 ms. Trace:
I20260709 13:06:55.962275 13875 rpcz_store.cc:276] 0709 13:06:53.342392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.265923 (+923531us) service_pool.cc:224] Handling call
0709 13:06:54.266426 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.276110 (+  9684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:54.276164 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:54.276228 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:54.276277 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.853244 (+576967us) spinlock_profiling.cc:243] Waited 9.39 ms on lock 0x7b54003ac5d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:54.853323 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.853499 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21588352}
I20260709 13:06:54.858670 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3225) took 1110 ms. Trace:
I20260709 13:06:55.964150 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3329) took 1407 ms. Trace:
I20260709 13:06:54.905105 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3167) took 1385 ms. Trace:
I20260709 13:06:55.965271 20315 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:55.965101 14024 rpcz_store.cc:276] 0709 13:06:53.519394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.794183 (+274789us) service_pool.cc:224] Handling call
0709 13:06:53.869280 (+ 75097us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:54.864855 (+995575us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b18003e2600. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c342bbfb 00007fb2c343664b
0709 13:06:54.865021 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:54.865050 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:54.865741 (+   691us) log.cc:844] Serialized 52 byte log entry
0709 13:06:54.875077 (+  9336us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:54.875128 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.875220 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.904652 (+ 29432us) raft_consensus.cc:1685] finished
0709 13:06:54.904681 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.905013 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3189,"raft.queue_time_us":109662,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":14156,"spinlock_wait_cycles":29978368,"thread_start_us":161383,"threads_started":2,"wal-append.queue_time_us":36736,"wal-append.run_cpu_time_us":2151,"wal-append.run_wall_time_us":1002157}
I20260709 13:06:55.967635 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:55.964088 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 907) took 1456 ms. Trace:
I20260709 13:06:54.879521 19782 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:55.972141 19782 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:54.732409 19997 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.974099 19997 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:54.707499 19986 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:06:54.759101 20097 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:55.980749 20097 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.981216 20097 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:54.725381 20111 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:55.984222 20111 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:54.821280 20120 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:55.991322 20120 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:54.848199 20114 consensus_meta.cc:287] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.721s	user 0.013s	sys 0.000s
I20260709 13:06:55.995038 20114 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:55.997412 20114 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:54.837997 20228 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.000563 20228 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:54.853850 19975 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.003549 19975 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:54.751987 20239 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:54.818857 20254 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:55.208834 20244 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:55.221210 20277 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:54.719156 20279 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.024459 20279 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.026587 20254 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:54.693213 20281 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.029619 20281 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:54.879545 20248 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.032752 20248 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:56.033635 13872 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:56.033185 20097 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.035563 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3325) took 1534 ms. Trace:
I20260709 13:06:56.035966 13872 rpcz_store.cc:276] 0709 13:06:54.500685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.662600 (+161915us) service_pool.cc:224] Handling call
0709 13:06:56.035329 (+1372729us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54000d9654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:56.035493 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3150766848}
W20260709 13:06:54.866816 20298 consensus_meta.cc:287] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.000s
I20260709 13:06:55.167429 20268 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:54.860002 20293 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:54.845734 20324 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:55.203634 20283 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:55.078940 20344 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:54.812311 20351 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 20 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:54.834932 20353 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:54.905941 20367 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.861318 20370 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:54.889211 20373 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.470s	user 0.000s	sys 0.005s
I20260709 13:06:54.764773 20376 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:54.707150 20382 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.791313 20359 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:54.796317 20388 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:55.413578 13990 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.007s	sys 0.036s
I20260709 13:06:55.418519 20407 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:55.451114 13759 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.015s	sys 0.007s
I20260709 13:06:55.472963 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 737) took 2243 ms. Trace:
W20260709 13:06:55.489495 20366 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.063s	user 0.000s	sys 0.009s
I20260709 13:06:55.514695 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3185) took 3359 ms. Trace:
I20260709 13:06:55.555508 14553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 1009) took 2482 ms. Trace:
I20260709 13:06:55.582561 20180 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:55.595978 20158 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:55.604485 13945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3192) took 2113 ms (client timeout 1869 ms). Trace:
W20260709 13:06:55.620455 20374 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.243s	user 0.006s	sys 0.007s
I20260709 13:06:55.622399 19983 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:55.626742 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:55.634287 20408 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.637988 20313 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:06:55.645269 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:55.646062 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.645692 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:55.646947 20410 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.655529 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:55.660723 20409 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.661571 20381 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:55.665372 13867 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.910s	user 0.005s	sys 0.022s
I20260709 13:06:55.667946 20312 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:55.677398 20101 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:55.677726 20334 raft_consensus.cc:3055] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:55.691345 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2940) took 1058 ms. Trace:
I20260709 13:06:55.697548 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2933) took 1590 ms. Trace:
I20260709 13:06:55.700011 20309 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:56.039147 20298 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.084525 20298 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.039392 20415 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.703060 20337 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:55.703145 20412 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:56.085853 20415 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.086397 20337 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.086562 20412 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.089016 20337 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:56.089612 20337 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.708233 20278 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:55.715327 20325 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:56.089888 20412 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:56.048528 20351 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.089459 20415 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:55.735741 14023 consensus_meta.cc:287] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.174s	user 0.015s	sys 0.000s
W20260709 13:06:55.740474 13870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.037s	user 0.014s	sys 0.025s
W20260709 13:06:55.743366 13944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.028s	user 0.016s	sys 0.019s
W20260709 13:06:55.744164 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.773885 20286 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:06:55.782624 14015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2615) took 4059 ms (client timeout 1652 ms). Trace:
I20260709 13:06:55.795644 13871 raft_consensus.cc:2368] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18.
I20260709 13:06:55.796638 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3180) took 3665 ms. Trace:
I20260709 13:06:55.810708 20319 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:55.813840 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2638) took 2384 ms. Trace:
W20260709 13:06:55.823194 14561 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.117s	user 0.012s	sys 0.021s
W20260709 13:06:55.822029 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.837306 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:55.853312 20295 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:06:55.853323 14018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3160) took 2625 ms (client timeout 1593 ms). Trace:
I20260709 13:06:55.871698 20349 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:55.865546 19982 consensus_queue.cc:1059] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:06:55.882588 20354 raft_consensus.cc:3055] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:06:55.883801 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 57895.4us
W20260709 13:06:55.885772 14560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 1016) took 2136 ms (client timeout 2113 ms). Trace:
I20260709 13:06:55.887171 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:06:55.679801 13840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.014s	sys 0.024s
I20260709 13:06:55.898792 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:55.903577 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:55.907845 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:55.913915 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:55.900120 20316 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.930568 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 906) took 1423 ms. Trace:
I20260709 13:06:55.944206 13868 rpcz_store.cc:276] 0709 13:06:52.870536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.921062 (+1050526us) service_pool.cc:224] Handling call
0709 13:06:53.925088 (+  4026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:54.164202 (+239114us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:54.317236 (+153034us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.451462 (+134226us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:54.452138 (+   676us) log.cc:844] Serialized 26 byte log entry
0709 13:06:54.452313 (+   175us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:54.456279 (+  3966us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.457149 (+   870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.457414 (+   265us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.757158 (+299744us) raft_consensus.cc:1685] finished
0709 13:06:54.757185 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.757422 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":258414,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":102,"raft.queue_time_us":45584,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":284,"thread_start_us":503770,"threads_started":3,"wal-append.queue_time_us":313606,"wal-append.run_cpu_time_us":5023,"wal-append.run_wall_time_us":1233278}
W20260709 13:06:55.952136 20386 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.354s	user 0.006s	sys 0.003s
I20260709 13:06:54.725314 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:55.966131 20315 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:55.968187 13873 rpcz_store.cc:276] 0709 13:06:54.507189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.963158 (+1455969us) service_pool.cc:224] Handling call
0709 13:06:55.963489 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.963713 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:55.963756 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:55.963805 (+    49us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:55.963850 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.963909 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.964024 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.961858 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:55.964677 13875 rpcz_store.cc:276] 0709 13:06:54.556704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.963408 (+1406704us) service_pool.cc:224] Handling call
0709 13:06:55.963695 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.963874 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:55.963901 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:55.963952 (+    51us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:55.963973 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.964022 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.964105 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:54.849159 20091 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:55.980219 20428 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:55.964339 13866 rpcz_store.cc:276] 0709 13:06:53.748381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.471996 (+723615us) service_pool.cc:224] Handling call
0709 13:06:54.514197 (+ 42201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:54.691654 (+177457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.858381 (+166727us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54000a8f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:54.858596 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377843072}
W20260709 13:06:54.738142 20223 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.003s
I20260709 13:06:55.989382 19997 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:55.985260 20111 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.090500 20325 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:55.719290 20413 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:55.732657 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:56.074483 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:56.066630 13990 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 1.310s	user 0.007s	sys 0.037s
W20260709 13:06:56.262552 13840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.537s	user 0.025s	sys 0.035s
W20260709 13:06:56.067284 13759 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 1.336s	user 0.016s	sys 0.007s
W20260709 13:06:56.263548 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 1.54342e+06us
W20260709 13:06:56.079795 13867 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.324s	user 0.006s	sys 0.023s
W20260709 13:06:56.079147 13947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2944) took 1342 ms (client timeout 1591 ms). Trace:
I20260709 13:06:56.264035 13869 rpcz_store.cc:276] 0709 13:06:54.506820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.840446 (+1333626us) service_pool.cc:224] Handling call
0709 13:06:55.840816 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:55.841057 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:55.841084 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:55.841571 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 13:06:55.913003 (+ 71432us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:55.913058 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.913150 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.930167 (+ 17017us) raft_consensus.cc:1685] finished
0709 13:06:55.930205 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.930466 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":70951,"threads_started":1,"wal-append.queue_time_us":84464}
I20260709 13:06:56.072083 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3169) took 2489 ms. Trace:
I20260709 13:06:56.069548 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3164) took 2551 ms. Trace:
W20260709 13:06:56.257431 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.067441 14019 rpcz_store.cc:276] 0709 13:06:53.229192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.426080 (+196888us) service_pool.cc:224] Handling call
0709 13:06:53.426386 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.428027 (+  1641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.428078 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.428145 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:53.428198 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.472560 (+2044362us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b5400020dd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:55.472633 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.472869 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3133696}
W20260709 13:06:56.262281 13942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3213) took 1510 ms (client timeout 1668 ms). Trace:
I20260709 13:06:56.038034 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3330) took 1481 ms. Trace:
I20260709 13:06:56.234498 14021 rpcz_store.cc:276] 0709 13:06:53.429585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.714292 (+284707us) service_pool.cc:224] Handling call
0709 13:06:53.759629 (+ 45337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.979066 (+219437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.979101 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.979145 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:53.979178 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.813450 (+1834272us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54002ea754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:55.813538 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.813728 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":504095104}
W20260709 13:06:56.298394 13944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.583s	user 0.019s	sys 0.023s
I20260709 13:06:56.072453 14552 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:56.054690 14551 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 21: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:06:56.152920 13989 leader_election.cc:336] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.586s (SENT)
I20260709 13:06:56.078481 13874 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.296275 13868 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 13:06:56.298635 14016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2654) took 1578 ms (client timeout 1543 ms). Trace:
I20260709 13:06:56.309890 14023 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
W20260709 13:06:56.299819 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:56.263924 14018 rpcz_store.cc:269] 0709 13:06:53.228203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.228333 (+   130us) service_pool.cc:224] Handling call
0709 13:06:55.853081 (+2624748us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54000f23d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:55.853236 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3111522176}
W20260709 13:06:56.070647 13945 rpcz_store.cc:269] 0709 13:06:53.491541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.541686 (+ 50145us) service_pool.cc:224] Handling call
0709 13:06:55.604165 (+2062479us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b5400088754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:55.604376 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97919872}
I20260709 13:06:56.393298 13989 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:56.265012 14560 rpcz_store.cc:269] 0709 13:06:53.749627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.749789 (+   162us) service_pool.cc:224] Handling call
0709 13:06:55.885272 (+2135483us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b5400710f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:55.885419 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398739200}
I20260709 13:06:56.453135 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3311) took 3117 ms. Trace:
W20260709 13:06:56.454030 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:56.393321 14016 rpcz_store.cc:269] 0709 13:06:54.720609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.725073 (+  4464us) service_pool.cc:224] Handling call
0709 13:06:56.264837 (+1539764us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:56.279448 14015 rpcz_store.cc:269] 0709 13:06:51.723554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.723700 (+   146us) service_pool.cc:224] Handling call
0709 13:06:55.782364 (+4058664us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b5400298bd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:55.782506 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3860218752}
W20260709 13:06:56.455868 13870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.753s	user 0.016s	sys 0.030s
W20260709 13:06:56.411936 13947 rpcz_store.cc:269] 0709 13:06:54.737143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.655153 (+918010us) service_pool.cc:224] Handling call
0709 13:06:56.079075 (+423922us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:56.457691 14023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2650) took 2257 ms (client timeout 1955 ms). Trace:
W20260709 13:06:56.458252 14023 rpcz_store.cc:269] 0709 13:06:54.200702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.512103 (+311401us) service_pool.cc:224] Handling call
0709 13:06:56.457594 (+1945491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.069689 14553 rpcz_store.cc:276] 0709 13:06:53.073024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.087153 (+ 14129us) service_pool.cc:224] Handling call
0709 13:06:53.271097 (+183944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.271693 (+   596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.555129 (+2283436us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54006f18d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:55.555379 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422205312}
I20260709 13:06:56.265120 13950 rpcz_store.cc:276] 0709 13:06:52.130905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.131049 (+   144us) service_pool.cc:224] Handling call
0709 13:06:52.152921 (+ 21872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.272305 (+1119384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.796318 (+2524013us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400092754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:55.796568 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2345025920}
W20260709 13:06:56.415130 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 527990us
I20260709 13:06:56.081092 13948 rpcz_store.cc:276] 0709 13:06:54.107266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.111230 (+  3964us) service_pool.cc:224] Handling call
0709 13:06:55.697187 (+1585957us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54000f4e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:55.697455 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412489984}
I20260709 13:06:56.415896 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:56.411962 13942 rpcz_store.cc:269] 0709 13:06:54.752065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.836955 (+1084890us) service_pool.cc:224] Handling call
0709 13:06:56.262191 (+425236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.080973 13946 rpcz_store.cc:276] 0709 13:06:54.632670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.707746 (+ 75076us) service_pool.cc:224] Handling call
0709 13:06:54.716112 (+  8366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:54.719337 (+  3225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:54.719383 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:54.762465 (+ 43082us) log.cc:844] Serialized 26 byte log entry
0709 13:06:55.690961 (+928496us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:55.691016 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.691103 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.691152 (+    49us) raft_consensus.cc:1685] finished
0709 13:06:55.691163 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.691283 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1213,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":32396,"thread_start_us":8990,"threads_started":1,"wal-append.queue_time_us":7005}
W20260709 13:06:56.393333 14552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 1022) took 1687 ms (client timeout 1641 ms). Trace:
I20260709 13:06:56.502846 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:56.503326 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:56.503918 14016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3180) took 1764 ms (client timeout 1941 ms). Trace:
I20260709 13:06:56.504647 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:56.504518 14016 rpcz_store.cc:269] 0709 13:06:54.739125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.502509 (+1763384us) service_pool.cc:224] Handling call
0709 13:06:56.503850 (+  1341us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.505671 14021 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
W20260709 13:06:56.506440 14021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 746) took 1783 ms (client timeout 2126 ms). Trace:
I20260709 13:06:56.506589 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:56.506924 14021 rpcz_store.cc:269] 0709 13:06:54.731843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.504328 (+1772485us) service_pool.cc:224] Handling call
0709 13:06:56.506350 (+  2022us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.508708 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.233760 20428 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:56.509531 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3209) took 1810 ms. Trace:
I20260709 13:06:56.521909 13948 rpcz_store.cc:276] 0709 13:06:54.699305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.508448 (+1809143us) service_pool.cc:224] Handling call
0709 13:06:56.509467 (+  1019us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.523077 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:56.524608 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 905) took 2018 ms. Trace:
I20260709 13:06:56.510315 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.525115 13870 rpcz_store.cc:276] 0709 13:06:54.505852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.676727 (+170875us) service_pool.cc:224] Handling call
0709 13:06:54.677066 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.677328 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:06:54.702680 (+ 25352us) log.cc:844] Serialized 14 byte log entry
0709 13:06:56.524165 (+1821485us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.524228 (+    63us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:06:56.524288 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.524378 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.524539 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67523,"prepare.queue_time_us":25148,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":3728,"thread_start_us":1753469,"threads_started":1,"wal-append.queue_time_us":151607,"wal-append.run_cpu_time_us":1267,"wal-append.run_wall_time_us":1010625}
W20260709 13:06:56.527310 14561 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.821s	user 0.015s	sys 0.028s
I20260709 13:06:56.528242 14561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 1018) took 1968 ms. Trace:
I20260709 13:06:56.528601 14561 rpcz_store.cc:276] 0709 13:06:54.559733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.559915 (+   182us) service_pool.cc:224] Handling call
0709 13:06:54.572944 (+ 13029us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:54.627104 (+ 54160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:54.627146 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:54.631431 (+  4285us) log.cc:844] Serialized 26 byte log entry
0709 13:06:56.527852 (+1896421us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:56.527897 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.527985 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:56.528031 (+    46us) raft_consensus.cc:1685] finished
0709 13:06:56.528046 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.528183 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55944,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":3472,"thread_start_us":1821490,"threads_started":1,"wal-append.queue_time_us":201466,"wal-append.run_cpu_time_us":1981,"wal-append.run_wall_time_us":1002299}
W20260709 13:06:56.530499 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 377606us
W20260709 13:06:56.532940 14017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3176) took 1776 ms (client timeout 1725 ms). Trace:
W20260709 13:06:56.533349 14017 rpcz_store.cc:269] 0709 13:06:54.733155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.961546 (+1228391us) service_pool.cc:224] Handling call
0709 13:06:56.509282 (+547736us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.062867 20382 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:56.263379 20354 raft_consensus.cc:740] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:56.065379 20359 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.537519 20348 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:56.555969 20325 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.557215 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:56.511781 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.559574 13945 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:56.512414 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:56.455214 14022 rpcz_store.cc:276] 0709 13:06:53.582875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.922953 (+340078us) service_pool.cc:224] Handling call
0709 13:06:53.923225 (+   272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.925459 (+  2234us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:06:53.925487 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.926102 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 13:06:54.173148 (+247046us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:54.173199 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.173291 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.619847 (+1446556us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:56.071752 (+451905us) raft_consensus.cc:1685] finished
0709 13:06:56.071780 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.072014 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52820,"raft.queue_time_us":62290,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":11044,"spinlock_wait_cycles":90192256,"thread_start_us":246591,"threads_started":1,"wal-append.queue_time_us":356257}
I20260709 13:06:56.513666 14016 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:56.560305 14023 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:56.560693 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:56.514739 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:56.561632 13942 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:56.459306 14020 rpcz_store.cc:276] 0709 13:06:53.518501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.716222 (+197721us) service_pool.cc:224] Handling call
0709 13:06:53.752613 (+ 36391us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:53.911429 (+158816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.911466 (+    37us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:53.931075 (+ 19609us) log.cc:844] Serialized 78 byte log entry
0709 13:06:54.168697 (+237622us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:54.191312 (+ 22615us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.200250 (+  8938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.200379 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.531429 (+331050us) spinlock_profiling.cc:243] Waited 7.72 ms on lock 0x7b18002b2b80. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:06:55.488743 (+957314us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:56.069281 (+580538us) raft_consensus.cc:1685] finished
0709 13:06:56.069312 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.069485 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41280,"prepare.queue_time_us":21356,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":34063,"raft.queue_time_us":198240,"raft.run_cpu_time_us":451,"raft.run_wall_time_us":71398,"spinlock_wait_cycles":40861824,"thread_start_us":352125,"threads_started":2,"wal-append.queue_time_us":426979}
W20260709 13:06:56.541527 14582 debug-util.cc:398] Leaking SignalData structure 0x7b0800322fa0 after lost signal to thread 12728
I20260709 13:06:56.563160 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:56.563740 13948 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:56.541899 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:56.563761 14582 debug-util.cc:398] Leaking SignalData structure 0x7b08002f8840 after lost signal to thread 13807
I20260709 13:06:56.507853 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:56.565605 14021 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
W20260709 13:06:56.566123 14021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2657) took 1844 ms (client timeout 2407 ms). Trace:
W20260709 13:06:56.540031 13947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 1104) took 1805 ms (client timeout 1865 ms). Trace:
W20260709 13:06:56.566588 13947 rpcz_store.cc:269] 0709 13:06:54.734464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.510045 (+1775581us) service_pool.cc:224] Handling call
0709 13:06:56.539932 (+ 29887us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:56.566536 14021 rpcz_store.cc:269] 0709 13:06:54.721465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.507624 (+1786159us) service_pool.cc:224] Handling call
0709 13:06:56.566025 (+ 58401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.567726 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:56.568373 13947 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.080848 20334 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:56.591289 14021 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:06:56.592188 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2659) took 1870 ms. Trace:
I20260709 13:06:56.592564 14021 rpcz_store.cc:276] 0709 13:06:54.721999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.590102 (+1868103us) service_pool.cc:224] Handling call
0709 13:06:56.590555 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.591890 (+  1335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.592123 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:56.468608 14552 rpcz_store.cc:269] 0709 13:06:54.707803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.762969 (+ 55166us) service_pool.cc:224] Handling call
0709 13:06:56.393085 (+1630116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.069213 13949 rpcz_store.cc:276] 0709 13:06:52.155782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.384421 (+228639us) service_pool.cc:224] Handling call
0709 13:06:52.393407 (+  8986us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.662841 (+1269434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.514220 (+1851379us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400086954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:55.514561 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2725996288}
I20260709 13:06:56.463923 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1100) took 1846 ms. Trace:
I20260709 13:06:56.521023 14559 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:06:56.412832 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3332) took 1855 ms. Trace:
I20260709 13:06:56.605834 13868 rpcz_store.cc:276] 0709 13:06:54.557611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.262762 (+1705151us) service_pool.cc:224] Handling call
0709 13:06:56.263192 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:56.412345 (+149153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.412741 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.058367 20324 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:56.467168 13871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3223) took 2674 ms (client timeout 1613 ms). Trace:
I20260709 13:06:56.543884 20430 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.027001 20279 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:56.537045 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:56.537375 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.617961 14015 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
W20260709 13:06:56.618521 14015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3177) took 1885 ms (client timeout 2197 ms). Trace:
W20260709 13:06:56.618865 14015 rpcz_store.cc:269] 0709 13:06:54.733512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.508354 (+1774842us) service_pool.cc:224] Handling call
0709 13:06:56.618455 (+110101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.080623 20312 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:56.066296 20388 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:56.511163 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:56.303699 20223 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.071691 19983 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:56.080909 20101 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.323807 20316 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.073112 20313 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.072765 20408 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.502192 20180 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:56.066946 20407 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.078876 20410 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.538185 20420 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.055292 20353 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.257215 20298 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:56.544353 19982 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:56.060168 20376 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.428943 20415 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.417290 19986 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:56.435681 20413 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.546627 14560 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:56.053869 20367 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:56.079463 20381 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.092053 20337 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.547457 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:56.040112 20239 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.079178 20409 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.639394 20334 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.663666 20334 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:56.665141 20409 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:56.548770 14551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 1019) took 1988 ms (client timeout 2333 ms). Trace:
I20260709 13:06:56.666263 14555 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:56.549654 20370 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:56.667683 20370 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.550187 19782 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:56.669957 19782 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.672230 19782 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.550534 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:56.551004 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:56.551858 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:56.674520 13875 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
I20260709 13:06:56.552390 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:56.675827 13866 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.676107 20381 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.553071 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:56.450443 20111 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:56.678797 20239 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:56.679721 14019 raft_consensus.cc:2393] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:56.076354 20097 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.278811 20286 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:56.501735 20158 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:56.560823 14023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2656) took 1839 ms (client timeout 1983 ms). Trace:
I20260709 13:06:56.640396 20180 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:56.569983 14582 debug-util.cc:398] Leaking SignalData structure 0x7b08002a5f80 after lost signal to thread 13899
W20260709 13:06:56.570206 13945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2942) took 1850 ms (client timeout 1815 ms). Trace:
W20260709 13:06:56.705544 13945 rpcz_store.cc:269] 0709 13:06:54.719673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.511542 (+1791869us) service_pool.cc:224] Handling call
0709 13:06:56.570124 (+ 58582us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:56.574798 14016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2661) took 1852 ms (client timeout 2235 ms). Trace:
W20260709 13:06:56.706301 14016 rpcz_store.cc:269] 0709 13:06:54.722690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.506209 (+1783519us) service_pool.cc:224] Handling call
0709 13:06:56.574729 (+ 68520us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:56.706629 14582 debug-util.cc:398] Leaking SignalData structure 0x7b08002af7e0 after lost signal to thread 13974
W20260709 13:06:56.706980 14582 debug-util.cc:398] Leaking SignalData structure 0x7b08002af820 after lost signal to thread 13975
W20260709 13:06:56.708341 14582 debug-util.cc:398] Leaking SignalData structure 0x7b08000a91a0 after lost signal to thread 14048
W20260709 13:06:56.709829 14582 debug-util.cc:398] Leaking SignalData structure 0x7b0800319e20 after lost signal to thread 14585
I20260709 13:06:56.579560 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:56.710579 14022 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.583348 19997 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:56.535859 20228 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:06:56.584122 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 168410us
I20260709 13:06:56.661949 20315 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:56.542212 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.723773 13950 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:56.467795 19975 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:06:56.594282 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 264230us
I20260709 13:06:56.594753 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:56.727913 13950 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3212) took 1976 ms (client timeout 2035 ms). Trace:
W20260709 13:06:56.728348 13950 rpcz_store.cc:269] 0709 13:06:54.751674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.510823 (+1759149us) service_pool.cc:224] Handling call
0709 13:06:56.727842 (+217019us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.600837 13874 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 21.
I20260709 13:06:56.729506 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 908) took 2027 ms. Trace:
I20260709 13:06:56.729987 13874 rpcz_store.cc:276] 0709 13:06:54.702315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.641749 (+939434us) service_pool.cc:224] Handling call
0709 13:06:56.729414 (+1087665us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.519591 13872 rpcz_store.cc:276] 0709 13:06:54.557020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.037014 (+1479994us) service_pool.cc:224] Handling call
0709 13:06:56.037443 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.037660 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.037688 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.037739 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:56.037778 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.037844 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.037970 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.453636 13867 rpcz_store.cc:276] 0709 13:06:53.335860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.194342 (+858482us) service_pool.cc:224] Handling call
0709 13:06:54.194678 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.194935 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:54.433752 (+238817us) log.cc:844] Serialized 14 byte log entry
0709 13:06:56.452693 (+2018941us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.452763 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:56.452829 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.452920 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.453071 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24496,"prepare.queue_time_us":224870,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":33426,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":100,"thread_start_us":2181249,"threads_started":3,"wal-append.queue_time_us":361666,"wal-append.run_cpu_time_us":1609,"wal-append.run_wall_time_us":1064850}
W20260709 13:06:56.607815 13871 rpcz_store.cc:269] 0709 13:06:53.747804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.772190 (+ 24386us) service_pool.cc:224] Handling call
0709 13:06:56.421958 (+2649768us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b540026cfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:56.422203 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4303837952}
I20260709 13:06:56.510986 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:56.564551 20355 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:56.616271 14020 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:56.736202 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2655) took 2015 ms. Trace:
I20260709 13:06:56.605144 13944 rpcz_store.cc:276] 0709 13:06:54.617603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.657390 (+ 39787us) service_pool.cc:224] Handling call
0709 13:06:54.657725 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:54.658693 (+   968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:54.658733 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:54.659298 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 13:06:56.463507 (+1804209us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:56.463557 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.463647 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:56.463696 (+    49us) raft_consensus.cc:1685] finished
0709 13:06:56.463707 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.463853 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8868,"raft.queue_time_us":8513,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":13765,"thread_start_us":1747569,"threads_started":1,"wal-append.queue_time_us":146196,"wal-append.run_cpu_time_us":1889,"wal-append.run_wall_time_us":1005693}
I20260709 13:06:56.593632 20291 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:56.753757 20295 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.754407 20315 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.754348 20295 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.755084 20315 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.624452 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2926) took 3387 ms. Trace:
I20260709 13:06:56.756532 13941 rpcz_store.cc:276] 0709 13:06:53.237175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.825221 (+588046us) service_pool.cc:224] Handling call
0709 13:06:53.825846 (+   625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.870311 (+ 44465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.870413 (+   102us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.876619 (+  6206us) log.cc:844] Serialized 26 byte log entry
0709 13:06:54.652988 (+776369us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:54.653034 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.653107 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.951703 (+1298596us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b240024a058. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c01cc9c4 00007fb2c01caacb 00007fb2c0385dc9 00007fb2c0385a55 00007fb2c038509d 00007fb2c2fb710e 00007fb2c2fce88c 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221
0709 13:06:56.624118 (+672415us) raft_consensus.cc:1685] finished
0709 13:06:56.624144 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.624382 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41339,"raft.queue_time_us":45186,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":14386,"spinlock_wait_cycles":65199232,"thread_start_us":774898,"threads_started":1,"wal-append.queue_time_us":721145}
I20260709 13:06:56.758407 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:56.759300 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:56.627672 13948 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:56.627287 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 747) took 1895 ms. Trace:
W20260709 13:06:56.630283 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 380386us
I20260709 13:06:56.760694 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.633419 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:56.621825 20351 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.050s	sys 0.071s
I20260709 13:06:56.761518 20291 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:56.643195 14556 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:56.735240 20359 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.769058 20295 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.622069 20382 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:56.769929 20436 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.769704 20359 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.770478 20436 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.629511 20431 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:56.614944 20114 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:06:56.769169 20291 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:56.657763 14553 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 22.
I20260709 13:06:56.642237 20325 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.775172 20325 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.686383 14024 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:06:56.623176 20430 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:56.665856 14551 rpcz_store.cc:269] 0709 13:06:54.564401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.568517 (+  4116us) service_pool.cc:224] Handling call
0709 13:06:56.548504 (+1979987us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400748454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:56.548689 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3414611200}
I20260709 13:06:56.645277 20283 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:56.640341 20244 consensus_queue.cc:1059] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:06:56.681044 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 748) took 1033 ms. Trace:
I20260709 13:06:56.783035 20283 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.783443 20359 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.783337 14017 rpcz_store.cc:276] 0709 13:06:55.647840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.547153 (+899313us) service_pool.cc:224] Handling call
0709 13:06:56.680967 (+133814us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.784349 13948 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:56.693439 20337 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.784611 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:56.702143 14023 rpcz_store.cc:269] 0709 13:06:54.721189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.512182 (+1790993us) service_pool.cc:224] Handling call
0709 13:06:56.560757 (+ 48575us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:56.682034 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.785267 20283 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.785665 20425 raft_consensus.cc:3055] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:56.785724 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:56.786223 20425 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:56.786517 14023 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:56.787444 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:56.787189 20425 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.788074 14023 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:06:56.788460 20425 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:56.789256 14023 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:56.513867 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:56.717001 13989 leader_election.cc:336] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.830s (SENT)
I20260709 13:06:56.779747 14018 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:56.791640 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:56.646565 20298 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:56.792620 14018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2658) took 2070 ms (client timeout 1787 ms). Trace:
W20260709 13:06:56.792994 14018 rpcz_store.cc:269] 0709 13:06:54.721718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.501915 (+1780197us) service_pool.cc:224] Handling call
0709 13:06:56.792414 (+290499us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54002986d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:56.792547 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232561024}
I20260709 13:06:56.794173 20298 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.794577 14018 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:56.795190 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 749) took 1147 ms. Trace:
I20260709 13:06:56.719573 14016 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:56.795501 14018 rpcz_store.cc:276] 0709 13:06:55.648149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.793844 (+1145695us) service_pool.cc:224] Handling call
0709 13:06:56.794394 (+   550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.794954 (+   560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.795136 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.796031 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3175) took 2063 ms. Trace:
I20260709 13:06:56.796365 14016 rpcz_store.cc:276] 0709 13:06:54.732820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.718854 (+1986034us) service_pool.cc:224] Handling call
0709 13:06:56.719219 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.795761 (+ 76542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.795977 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.797540 14016 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:56.583287 13947 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 21.
I20260709 13:06:56.785321 14017 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 22.
I20260709 13:06:56.799043 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.799551 20436 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.800674 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.643426 20420 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.802299 13838 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:56.803174 20248 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:56.803768 20248 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:56.804078 20420 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.804725 20248 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:56.805186 13759 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 545fc4aeb71d491985e296a0721d905d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d1b4fd04200247d99ba9ad1fbeb42b42 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:06:56.807720 14019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3178) took 2073 ms (client timeout 2178 ms). Trace:
W20260709 13:06:56.808265 14019 rpcz_store.cc:269] 0709 13:06:54.733804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.503042 (+1769238us) service_pool.cc:224] Handling call
0709 13:06:56.807453 (+304411us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b5400020dd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:56.807636 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403456384}
I20260709 13:06:56.702848 19782 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.737782 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:56.739269 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 274123us
I20260709 13:06:56.736608 14020 rpcz_store.cc:276] 0709 13:06:54.720912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.587023 (+1866111us) service_pool.cc:224] Handling call
0709 13:06:56.608942 (+ 21919us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.735898 (+126956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.736141 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.825762 14023 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:06:56.744242 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:56.746038 20097 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:56.725303 20091 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:06:56.651881 19982 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.682770 19983 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.751483 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.833918 13950 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:06:56.753445 19986 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.760164 14015 rpcz_store.cc:276] 0709 13:06:54.732170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.619532 (+1887362us) service_pool.cc:224] Handling call
0709 13:06:56.626607 (+  7075us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.626868 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.626906 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.626968 (+    62us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:56.627025 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.627095 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.627218 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.655014 20415 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.767196 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:56.768143 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.770403 20382 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:56.637244 20309 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.556s	user 0.005s	sys 0.006s
I20260709 13:06:56.773787 20431 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.636648 20279 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:06:56.778409 13941 raft_consensus.cc:3060] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.779562 13949 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:06:56.768957 20351 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.051s	sys 0.071s
W20260709 13:06:56.784411 14024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 745) took 2061 ms (client timeout 1739 ms). Trace:
W20260709 13:06:56.857345 14024 rpcz_store.cc:269] 0709 13:06:54.722942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.967336 (+1244394us) service_pool.cc:224] Handling call
0709 13:06:56.784228 (+816892us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54002bdc54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:56.784342 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651440384}
I20260709 13:06:56.698308 14558 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:06:56.543139 20428 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.858372 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:56.768846 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.859302 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2945) took 1184 ms. Trace:
I20260709 13:06:56.859723 13949 rpcz_store.cc:276] 0709 13:06:55.675193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.624317 (+949124us) service_pool.cc:224] Handling call
0709 13:06:56.859236 (+234919us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.827857 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3179) took 2089 ms. Trace:
I20260709 13:06:56.860800 14023 rpcz_store.cc:276] 0709 13:06:54.738851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.788773 (+2049922us) service_pool.cc:224] Handling call
0709 13:06:56.789120 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.827554 (+ 38434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.827792 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.829275 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:56.818532 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.821650 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:56.862087 13871 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 22.
I20260709 13:06:56.864379 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:56.830832 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:56.865947 13944 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:56.853297 14022 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 21.
I20260709 13:06:56.735966 20281 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:56.833130 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 116175us
I20260709 13:06:56.832295 19782 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.758106 19975 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.838584 20408 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:56.839013 20286 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:56.874027 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 223483us
I20260709 13:06:56.874775 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.840186 20407 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.840329 20239 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:56.841593 20180 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:56.840214 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.814977 20101 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:56.835960 20235 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:56.843281 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:56.879209 14556 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 21: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:56.844112 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:56.879758 14553 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:56.848881 13866 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 21.
I20260709 13:06:56.708473 20158 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.778024 20325 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:56.853914 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:56.902518 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 48679.7us
W20260709 13:06:56.855346 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:56.904318 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:56.904814 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.736898 20228 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.828573 20097 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:56.850757 19986 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.861478 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.916658 13948 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:56.872161 20370 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.865272 13947 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:56.919986 13867 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
I20260709 13:06:56.863422 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:56.920971 14560 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:06:56.844933 20413 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:56.874857 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:56.839695 20355 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:56.878319 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:56.832734 20298 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:06:56.860069 19983 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:56.892297 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:56.938424 14555 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 21: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:56.868434 20244 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:56.868430 20309 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:56.903522 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:56.908871 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.892498 13914 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:06:56.945329 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 52923us
I20260709 13:06:56.859467 13945 raft_consensus.cc:2410] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:56.917930 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:56.829501 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 85386.3us
I20260709 13:06:56.952901 13840 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:56.954979 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:56.925618 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.924021 13941 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 21.
I20260709 13:06:56.907083 20286 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:56.931910 13944 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 11.
I20260709 13:06:56.968437 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:56.969367 13944 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:56.877265 20223 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:56.943599 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:56.939594 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 61406.7us
I20260709 13:06:56.857546 20291 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:56.907299 20313 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:56.980317 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3208) took 2281 ms. Trace:
I20260709 13:06:56.980815 13945 rpcz_store.cc:276] 0709 13:06:54.698937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.761245 (+2062308us) service_pool.cc:224] Handling call
0709 13:06:56.980221 (+218976us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.875952 20281 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:56.985546 13987 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:56.962333 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:56.875279 20283 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:06:56.916069 20097 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:56.910086 20319 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:56.995242 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:56.989869 13840 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:56.992820 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 130653us
I20260709 13:06:56.993301 13987 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:57.004657 13944 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:56.997803 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:57.048708 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 58968.1us
I20260709 13:06:56.941556 20244 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:57.049098 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:56.980347 20413 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:56.980914 20316 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:56.984862 20381 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:56.982214 20325 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:56.997799 13949 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 21: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:57.054818 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:56.979676 20409 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.982462 20370 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:56.980028 20101 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:56.941088 20410 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:56.994868 20319 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:56.995834 20254 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:56.991073 20279 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.094s
W20260709 13:06:57.088163 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 138070us
I20260709 13:06:57.054978 20337 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:57.060901 13947 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:57.057924 20413 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.058928 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.063123 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:57.084828 20315 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:57.056981 20091 raft_consensus.cc:2764] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:06:57.062634 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 1106) took 1250 ms. Trace:
W20260709 13:06:57.102326 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 158835us
I20260709 13:06:57.108337 13949 rpcz_store.cc:276] 0709 13:06:55.811916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.966244 (+1154328us) service_pool.cc:224] Handling call
0709 13:06:57.062535 (+ 96291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:57.105664 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:57.088851 20381 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:57.118245 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.096439 20319 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:57.113674 20446 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.154549 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:57.155248 13870 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:57.120455 20283 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:57.126529 19782 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.126879 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.131091 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:57.157418 13866 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:57.132287 13873 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:06:57.158090 19982 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:57.090446 20409 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.135839 20291 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.158862 20376 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:57.092991 20316 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:57.159866 20316 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.160637 20279 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:57.161156 20279 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:57.161993 20316 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.107160 20239 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:57.162252 20279 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.114746 20298 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:57.163277 20298 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.163607 20279 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:57.143085 20382 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:57.147023 13872 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:06:57.148530 13874 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:06:57.164695 13944 raft_consensus.cc:2393] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:57.150507 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:57.165226 13875 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:57.165457 20409 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.103571 20337 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.166451 20337 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:57.151064 20158 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:57.103542 20325 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.170584 20325 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:57.152774 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:57.174419 13869 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:57.177075 13870 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:57.153867 20431 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:57.154834 20446 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:57.178289 13943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3210) took 2416 ms (client timeout 1571 ms). Trace:
W20260709 13:06:57.178740 13943 rpcz_store.cc:269] 0709 13:06:54.699609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.723574 (+ 23965us) service_pool.cc:224] Handling call
0709 13:06:57.111413 (+2387839us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400090954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:57.115698 (+  4285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2825697664}
I20260709 13:06:57.179224 13875 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:57.180042 20446 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.163674 20298 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:57.146592 13871 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:57.165670 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.180944 20244 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.181854 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2947) took 1506 ms. Trace:
I20260709 13:06:57.182226 13944 rpcz_store.cc:276] 0709 13:06:55.675767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.165336 (+1489569us) service_pool.cc:224] Handling call
0709 13:06:57.181785 (+ 16449us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:57.182744 19986 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.183185 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.184122 13944 raft_consensus.cc:2410] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:57.103814 20180 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:57.189031 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:57.190830 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.191771 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:57.192082 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.192864 13871 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:57.193256 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.108631 20413 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:57.196513 13944 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:06:57.184538 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 75758.3us
I20260709 13:06:57.151988 20415 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:57.124285 20442 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:57.180078 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.196725 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.202945 13949 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:57.199427 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:57.203595 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:57.203828 13874 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:57.204380 13870 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 21: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:57.207845 20430 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:57.201921 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:57.222968 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:57.185079 20244 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:57.211206 13949 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:57.212549 20441 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.112s	user 0.003s	sys 0.000s
I20260709 13:06:57.215998 20281 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:57.219748 20428 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:57.221685 20354 consensus_queue.cc:260] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.230638 20381 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.231480 20354 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:57.201900 20442 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:57.234845 20442 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:57.235842 19975 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:57.239977 20324 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.000s
I20260709 13:06:57.242117 14558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 1025) took 1164 ms. Trace:
I20260709 13:06:57.242744 14558 rpcz_store.cc:276] 0709 13:06:56.077362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.508261 (+430899us) service_pool.cc:224] Handling call
0709 13:06:57.241860 (+733599us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400710f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:57.242001 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322355584}
I20260709 13:06:57.224679 13944 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:57.224285 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2660) took 2502 ms. Trace:
I20260709 13:06:57.200438 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:57.225214 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:57.248275 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2941) took 2592 ms. Trace:
I20260709 13:06:57.248674 13944 rpcz_store.cc:276] 0709 13:06:54.656389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.224110 (+2567721us) service_pool.cc:224] Handling call
0709 13:06:57.224487 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.247722 (+ 23235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.248215 (+   493us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:57.249577 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:57.249802 20097 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.113462 20309 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:57.225600 13869 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:06:57.227212 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:57.229919 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:57.252655 13872 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 21: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:57.252697 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:57.230721 20446 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:57.255779 20228 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.247759 14021 rpcz_store.cc:276] 0709 13:06:54.722261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.602372 (+1880111us) service_pool.cc:224] Handling call
0709 13:06:56.602722 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.607746 (+  5024us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:56.832821 (+225075us) log.cc:844] Serialized 14 byte log entry
0709 13:06:57.211719 (+378898us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:57.223760 (+ 12041us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:57.223843 (+    83us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:57.223899 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.224083 (+   184us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.224213 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":300632,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":5131,"raft.queue_time_us":22444,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":50769,"spinlock_wait_cycles":11081984,"thread_start_us":577449,"threads_started":3,"wal-append.queue_time_us":267637}
I20260709 13:06:57.255877 20319 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.256459 20409 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:57.257032 20409 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.257424 20409 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:57.259339 20097 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:57.261420 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 398848us
I20260709 13:06:57.263913 20354 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:57.228886 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3207) took 2530 ms. Trace:
I20260709 13:06:57.225327 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:57.230134 20353 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:57.266230 20353 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:57.266601 13950 raft_consensus.cc:2410] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:06:57.266690 20353 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:57.251405 20309 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.253572 20447 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:57.267709 20447 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.268090 20447 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:57.270149 13949 rpcz_store.cc:276] 0709 13:06:54.698585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.210531 (+2511946us) service_pool.cc:224] Handling call
0709 13:06:57.210990 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.228555 (+ 17565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.228821 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:57.276324 13947 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:57.276733 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.277078 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3211) took 2577 ms. Trace:
I20260709 13:06:57.277454 13947 rpcz_store.cc:276] 0709 13:06:54.699908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.224897 (+2524989us) service_pool.cc:224] Handling call
0709 13:06:57.276118 (+ 51221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.276795 (+   677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.277018 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:57.278841 13945 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:06:57.280584 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.281086 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.281399 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.281690 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.250259 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2943) took 2530 ms. Trace:
I20260709 13:06:57.282511 13944 rpcz_store.cc:276] 0709 13:06:54.720031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.249432 (+2529401us) service_pool.cc:224] Handling call
0709 13:06:57.249740 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.249933 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:57.249961 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:57.250004 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:57.250037 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.250113 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.250208 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:57.287896 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:57.292032 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:57.294201 13944 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:06:57.295154 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2951) took 1508 ms. Trace:
I20260709 13:06:57.295599 13944 rpcz_store.cc:276] 0709 13:06:55.786781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.284056 (+1497275us) service_pool.cc:224] Handling call
0709 13:06:57.284530 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.294773 (+ 10243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.295078 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:57.297464 13944 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:06:57.298190 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2952) took 1478 ms. Trace:
I20260709 13:06:57.298621 13944 rpcz_store.cc:276] 0709 13:06:55.820353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.296628 (+1476275us) service_pool.cc:224] Handling call
0709 13:06:57.297026 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.297853 (+   827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.298126 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:57.274290 19997 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.264532 20354 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.306934 13870 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:57.308734 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:57.306946 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1107) took 1371 ms. Trace:
I20260709 13:06:57.311471 13944 rpcz_store.cc:276] 0709 13:06:55.935288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.305958 (+1370670us) service_pool.cc:224] Handling call
0709 13:06:57.306323 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.306508 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:57.306532 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:57.306574 (+    42us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:57.306606 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.306746 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.306888 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:57.315825 13947 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:57.316773 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2950) took 1638 ms. Trace:
I20260709 13:06:57.317225 13947 rpcz_store.cc:276] 0709 13:06:55.678771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.278259 (+1599488us) service_pool.cc:224] Handling call
0709 13:06:57.315310 (+ 37051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.316408 (+  1098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.316693 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:57.228489 20410 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:57.319789 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:57.324667 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 115880us
W20260709 13:06:57.310928 20388 consensus_meta.cc:287] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.678s	user 0.012s	sys 0.000s
I20260709 13:06:57.326930 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:57.327067 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:06:57.327157 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 333933us
I20260709 13:06:57.225075 20298 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.256513 20228 consensus_queue.cc:1059] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:06:57.331584 14553 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:57.325575 13915 connection.cc:256] RPC call timeout handler was delayed by -1.2362187679999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
I20260709 13:06:57.335986 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.336529 14022 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:57.337808 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:57.242563 20337 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.340778 13947 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:57.355720 14015 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:57.357095 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1112) took 1231 ms. Trace:
I20260709 13:06:57.357471 13947 rpcz_store.cc:276] 0709 13:06:56.126316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.324199 (+1197883us) service_pool.cc:224] Handling call
0709 13:06:57.340539 (+ 16340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.356780 (+ 16241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.357026 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:57.356355 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0895187679999481s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
W20260709 13:06:57.358639 13915 connection.cc:256] RPC call timeout handler was delayed by -1.081418767999935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
I20260709 13:06:57.358479 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:06:57.359350 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:57.323870 20319 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:57.366350 14020 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:57.326452 20376 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.319633 19997 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.354771 14021 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:57.340379 14016 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:57.381163 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:57.377291 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:57.378917 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:57.385076 14024 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:57.368615 13943 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:57.388175 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:57.389986 14016 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:06:57.356544 20449 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.392304 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:06:57.392340 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.393707 14556 raft_consensus.cc:2368] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 22. Current term is 23.
I20260709 13:06:57.387635 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:57.416123 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:57.417762 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:57.418674 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.418834 14022 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:57.402383 14553 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:57.420177 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.416956 14023 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:06:57.426203 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.419582 14021 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:57.428588 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.431248 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:57.432576 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:57.437112 14022 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:57.438549 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.439452 14022 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:06:57.434146 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:57.429174 14021 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:57.380615 19983 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:57.395881 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:57.442442 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.448565 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:57.425761 20223 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.339782 20388 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.454540 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.462658 14018 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 21: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:57.443435 14017 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:57.465399 14023 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:57.451043 14019 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:06:57.394415 20313 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:57.454874 20312 consensus_meta.cc:287] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.819s	user 0.008s	sys 0.007s
I20260709 13:06:57.450522 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.368081 20448 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:57.460145 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 93728.4us
I20260709 13:06:57.380441 20097 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.372987 20409 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.441007 20111 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:57.470937 14558 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:06:57.390527 20235 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:06:57.383452 20376 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.451532 20415 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:57.441406 20451 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.482259 13946 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:57.478395 14016 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:06:57.442910 20319 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:57.389819 20281 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:06:57.489434 14020 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:57.391160 20417 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:06:57.469101 19983 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:57.490398 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 435623us
I20260709 13:06:57.451467 20430 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:57.566011 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:57.564885 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:57.569628 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:57.570309 13869 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:57.573879 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:57.494378 14024 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:06:57.495595 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:57.504916 20442 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.577471 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:57.578409 13870 raft_consensus.cc:2393] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:57.481361 20313 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.506915 20337 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.508822 20376 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.509809 20413 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.512104 20448 consensus_queue.cc:1059] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:57.487716 20334 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:06:57.515267 20409 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.520462 20445 raft_consensus.cc:3055] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:57.600274 13949 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 23. Current term is 24. Ops: []
I20260709 13:06:57.523444 20298 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.477041 20324 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:57.532984 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2953) took 1513 ms. Trace:
I20260709 13:06:57.606560 13950 rpcz_store.cc:276] 0709 13:06:56.019600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.854922 (+835322us) service_pool.cc:224] Handling call
0709 13:06:57.532722 (+677800us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b5400085f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:57.532910 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":898553600}
I20260709 13:06:57.533860 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58096 (request call id 1110) took 1478 ms. Trace:
I20260709 13:06:57.607893 13941 rpcz_store.cc:276] 0709 13:06:56.055903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.759110 (+703207us) service_pool.cc:224] Handling call
0709 13:06:57.533692 (+774582us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b5400091854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:57.533799 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42725760}
I20260709 13:06:57.608424 13869 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:06:57.535328 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.609262 13943 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:06:57.535826 20388 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:57.538626 20410 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:57.540962 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:57.541707 20367 raft_consensus.cc:3055] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:57.443499 20449 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.487176 20444 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:06:57.544621 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:57.546882 13944 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:57.541767 20438 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.607s	user 0.002s	sys 0.000s
I20260709 13:06:57.552067 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:57.553032 20452 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:06:57.523521 20440 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.563s	user 0.000s	sys 0.004s
I20260709 13:06:57.556674 20180 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:57.683983 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 132015us
I20260709 13:06:57.605167 20324 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.564584 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:57.636173 20410 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:57.582789 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:57.584450 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.466554 20223 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.590193 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:57.586241 13839 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:57.591894 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:57.702435 13870 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:57.580883 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.594250 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.704278 14017 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
W20260709 13:06:57.704756 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 140291us
I20260709 13:06:57.557834 20330 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.564121 20091 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:57.567867 20415 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:57.716398 20415 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.715848 19782 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
W20260709 13:06:57.717726 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 131567us
I20260709 13:06:57.718724 13839 leader_election.cc:400] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 22. Current term is 23.
I20260709 13:06:57.719115 13839 leader_election.cc:403] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:06:57.719259 19782 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:57.719802 19782 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:57.719928 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:57.720707 19782 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.721112 20353 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.721235 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:57.588359 20430 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:57.722379 20235 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:57.722913 20430 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.723523 20353 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.615026 20354 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:57.639432 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.641337 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:57.642911 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.735131 13948 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:57.725932 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:57.643720 20355 raft_consensus.cc:3055] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:57.647523 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.738371 13945 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:57.649451 14522 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:57.739256 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 89964.5us
I20260709 13:06:57.636497 20367 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:57.655454 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:57.657689 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.576784 20319 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.574179 20097 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:57.671667 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 253972us
I20260709 13:06:57.600925 20445 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:57.565692 20451 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.588991 20325 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.563684 13942 raft_consensus.cc:2410] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:06:57.686318 13988 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:57.697572 14016 raft_consensus.cc:2410] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:06:57.701542 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.703614 14024 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:06:57.705967 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:57.708395 13946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2949) took 2032 ms (client timeout 1974 ms). Trace:
W20260709 13:06:57.568217 20286 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.584s	user 0.007s	sys 0.005s
I20260709 13:06:57.712683 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:57.714257 20330 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:57.715265 20348 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:57.600389 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:57.686311 20431 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:57.588027 20158 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:57.603137 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:57.724617 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.789201 14020 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 20.
I20260709 13:06:57.724773 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:57.789691 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:06:57.624049 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.695519 20388 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:57.793607 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.793985 13947 raft_consensus.cc:1240] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:57.743144 13870 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 23.
I20260709 13:06:57.748910 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:06:57.751451 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.795761 14019 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:06:57.736436 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:06:57.728036 20279 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.002s
W20260709 13:06:57.802578 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.803014 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.726917 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:57.803301 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.742388 20248 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.006s	sys 0.014s
W20260709 13:06:57.787966 20447 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.516s	user 0.013s	sys 0.000s
I20260709 13:06:57.778407 14559 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
W20260709 13:06:57.777940 13946 rpcz_store.cc:269] 0709 13:06:55.676274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.513593 (+837319us) service_pool.cc:224] Handling call
0709 13:06:57.708133 (+1194540us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b54001c7854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:57.708287 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1590919552}
I20260709 13:06:57.725981 20413 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:57.728096 20444 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.744529 20355 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:57.791056 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.748926 20409 raft_consensus.cc:2804] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:06:57.797935 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 61566.4us
W20260709 13:06:57.859177 13942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 2948) took 2183 ms (client timeout 2173 ms). Trace:
I20260709 13:06:57.869321 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:57.694831 20324 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:57.715317 20091 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.802014 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:06:57.873939 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 72033.5us
I20260709 13:06:57.875123 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:06:57.807619 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.773869 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 87635.2us
I20260709 13:06:57.877743 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:06:57.878942 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.806491 20353 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:57.808674 20367 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:57.807296 20248 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.965s	user 0.007s	sys 0.014s
I20260709 13:06:57.800894 20291 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 22. Current term is 23.
I20260709 13:06:57.814929 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:57.921463 13946 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:57.921623 20376 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:57.825139 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.925158 20353 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:57.925812 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.926978 13950 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:57.926386 20353 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.928179 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.785091 20348 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.785480 20298 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.786108 20431 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:57.786669 20354 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:57.770907 20458 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:57.769934 20417 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 13:06:57.793510 20388 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:57.859462 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.866642 13941 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:57.866937 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1179) took 1087 ms. Trace:
W20260709 13:06:57.868775 13942 rpcz_store.cc:269] 0709 13:06:55.676018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.594467 (+918449us) service_pool.cc:224] Handling call
0709 13:06:57.816567 (+1222100us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b54001c6e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:57.859068 (+ 42501us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1921235200}
I20260709 13:06:57.929575 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.986928 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.872299 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:06:57.988641 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 116468us
I20260709 13:06:57.989107 13950 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:57.881228 20313 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:57.881229 20442 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:57.991806 13942 raft_consensus.cc:2410] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:06:57.882647 20223 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.992873 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:57.993692 13941 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 14: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:57.859481 20158 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:57.986316 14551 rpcz_store.cc:276] 0709 13:06:56.779452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.780141 (+   689us) service_pool.cc:224] Handling call
0709 13:06:56.780557 (+   416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:56.780973 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.781005 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:56.790928 (+  9923us) log.cc:844] Serialized 52 byte log entry
0709 13:06:56.917942 (+127014us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:56.918004 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.918142 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:57.522814 (+604672us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:57.827259 (+304445us) raft_consensus.cc:1685] finished
0709 13:06:57.827293 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.866838 (+ 39545us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32378,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":11203,"spinlock_wait_cycles":60075264,"thread_start_us":126510,"threads_started":1,"wal-append.queue_time_us":169056}
I20260709 13:06:57.992882 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.885571 19983 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:57.848868 20447 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.892232 20091 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:57.894761 20359 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:57.897902 20409 raft_consensus.cc:697] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:57.903466 20451 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:57.904366 20449 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:57.904735 20448 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:58.003695 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 362392us
I20260709 13:06:57.913666 20319 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:57.869058 20444 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:57.869277 20455 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.389s	user 0.001s	sys 0.000s
I20260709 13:06:57.919381 20355 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:57.930859 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 50471.3us
I20260709 13:06:57.924015 20428 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:57.865012 20445 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:57.935202 20367 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:58.018692 14558 raft_consensus.cc:2393] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 19: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:57.935650 20334 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:57.989547 14524 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:57.884218 20325 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:57.943058 20436 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.103s	user 0.007s	sys 0.026s
I20260709 13:06:57.885142 20407 raft_consensus.cc:2764] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:06:57.996316 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.997551 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:58.033133 13941 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:57.943248 20354 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.942498 20413 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:58.009681 20448 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:57.999920 13987 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:57.999609 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:57.935691 20376 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
W20260709 13:06:58.001837 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 278888us
I20260709 13:06:58.040520 13950 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:06:57.987522 20388 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:57.942860 20348 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:58.027930 20436 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.188s	user 0.007s	sys 0.028s
I20260709 13:06:58.008534 13915 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:57.994590 20228 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:57.994896 20158 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.016254 13946 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:58.023535 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:58.003455 20442 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.048821 19782 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.049998 20428 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:06:58.037518 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 331589us
I20260709 13:06:58.037792 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:06:58.000203 14525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.014s	sys 0.044s
I20260709 13:06:57.911840 20097 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:58.044999 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:58.103292 13950 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:58.044591 13987 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:58.104554 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 60105.4us
I20260709 13:06:58.104770 13950 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:58.105597 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3220) took 1529 ms. Trace:
I20260709 13:06:58.021535 20334 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:58.048694 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:58.107818 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 59198.1us
I20260709 13:06:58.042119 20413 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:58.041749 20415 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.035770 20354 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:58.046761 20376 consensus_queue.cc:1243] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:06:58.033169 20445 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:58.033592 20309 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:58.045403 20430 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:58.006471 20330 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:58.032397 13942 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:06:58.129822 13942 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []
W20260709 13:06:58.099026 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 75637.3us
I20260709 13:06:58.131064 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:06:58.039566 20355 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:58.134553 20316 raft_consensus.cc:2764] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:06:58.101064 14525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.014s	sys 0.044s
I20260709 13:06:58.136564 20447 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:58.101117 13941 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:06:58.138449 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2956) took 1410 ms. Trace:
I20260709 13:06:58.138806 13941 rpcz_store.cc:276] 0709 13:06:56.728590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.137419 (+1408829us) service_pool.cc:224] Handling call
0709 13:06:58.137782 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.137985 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:58.138010 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:58.138163 (+   153us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:58.138200 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.138258 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:58.138378 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:58.101807 20442 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:58.101228 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:58.141477 20442 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:58.142203 20472 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:58.142357 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:58.142786 20472 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.143167 20472 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:58.086614 20158 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:58.146082 20248 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:58.105484 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:58.150489 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 45100.4us
W20260709 13:06:58.151437 20456 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.650s	user 0.004s	sys 0.000s
I20260709 13:06:57.943598 20298 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:58.088085 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 78967.7us
I20260709 13:06:58.154569 20298 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:58.156450 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:58.156800 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.157308 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3223) took 1580 ms. Trace:
I20260709 13:06:58.105504 20319 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:58.162781 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.029752 20431 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:58.112044 20334 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:58.127969 20430 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:58.151317 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:58.177109 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:58.116191 20428 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:06:58.182996 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.185164 14019 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:06:58.106387 13950 rpcz_store.cc:276] 0709 13:06:56.576296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.104142 (+1527846us) service_pool.cc:224] Handling call
0709 13:06:58.104528 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.105301 (+   773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.105537 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:58.157728 13942 rpcz_store.cc:276] 0709 13:06:56.577147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.129275 (+1552128us) service_pool.cc:224] Handling call
0709 13:06:58.129629 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.157015 (+ 27386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.157238 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:58.158563 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3217) took 1583 ms. Trace:
I20260709 13:06:58.187613 13947 rpcz_store.cc:276] 0709 13:06:56.575332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.380839 (+805507us) service_pool.cc:224] Handling call
0709 13:06:57.381171 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.801213 (+420042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.127293 (+326080us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b540022a954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:58.158500 (+ 31207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948734848}
I20260709 13:06:58.158821 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:58.188270 13950 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:58.192615 13942 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:58.155211 20451 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:58.160562 13988 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:58.160593 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.124841 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:58.054755 20228 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:58.041829 20448 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:58.165863 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:58.165771 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.200763 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:58.200924 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:58.166776 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.116469 20449 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:58.159145 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.210721 13867 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 21: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:58.161399 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.211792 13873 raft_consensus.cc:2393] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:58.197999 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2961) took 1389 ms. Trace:
I20260709 13:06:58.103204 19782 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:58.202762 13875 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:06:58.180069 20470 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
W20260709 13:06:58.216616 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 56187.5us
I20260709 13:06:58.218281 13988 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:58.219787 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:58.220530 14020 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:06:58.220894 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:06:58.222796 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 98058.4us
I20260709 13:06:58.223001 14551 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:58.223516 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:58.203770 14555 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:58.203197 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1116) took 1348 ms. Trace:
I20260709 13:06:58.212373 13942 rpcz_store.cc:276] 0709 13:06:56.808885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.187601 (+1378716us) service_pool.cc:224] Handling call
0709 13:06:58.192344 (+  4743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.193046 (+   702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.197932 (+  4886us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29056}
I20260709 13:06:58.224771 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.225495 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.238184 13874 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 23: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:58.226217 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.227352 14020 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:58.180925 20430 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:58.190996 20279 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:58.190348 20313 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:58.231232 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:58.098366 20409 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:58.195335 20431 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:58.232813 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.235644 14024 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:06:58.260694 14024 pending_rounds.cc:85] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 15
I20260709 13:06:58.261004 14024 pending_rounds.cc:107] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:06:58.261320 14024 raft_consensus.cc:2889] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:58.165752 20319 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.262585 20319 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:58.262666 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1114) took 1499 ms. Trace:
I20260709 13:06:58.263106 13941 rpcz_store.cc:276] 0709 13:06:56.762794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.140078 (+1377284us) service_pool.cc:224] Handling call
0709 13:06:58.140362 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.140583 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:06:58.171807 (+ 31224us) log.cc:844] Serialized 14 byte log entry
0709 13:06:58.262288 (+ 90481us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:58.262350 (+    62us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:58.262399 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.262480 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:58.262601 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6212,"prepare.queue_time_us":33047,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"thread_start_us":83777,"threads_started":1,"wal-append.queue_time_us":84260}
I20260709 13:06:58.265295 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:58.265939 13941 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:06:58.239073 13950 rpcz_store.cc:276] 0709 13:06:56.854785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.187606 (+1332821us) service_pool.cc:224] Handling call
0709 13:06:58.188027 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.194603 (+  6576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.203128 (+  8525us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14080}
I20260709 13:06:58.240877 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:58.121523 20415 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:58.207803 20228 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:58.245039 14559 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:58.191447 20428 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.245230 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:58.247795 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:58.208719 19997 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:58.250005 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:58.209335 20449 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:58.253957 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 92008us
I20260709 13:06:58.218439 20309 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:58.268816 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.290494 13874 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:58.179751 20469 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
W20260709 13:06:58.271013 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 52787.6us
I20260709 13:06:58.232153 20298 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:58.228574 20244 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:58.286291 20428 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:58.249295 20313 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.215392 20324 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:58.309551 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 61868.2us
I20260709 13:06:58.311926 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:58.252821 20279 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.268316 19983 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:58.313715 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.284152 20316 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:58.320943 20316 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:58.321583 20316 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:58.315079 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:58.315629 20451 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:58.322486 13872 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:58.323181 20451 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:58.319291 20279 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:58.268626 20330 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
W20260709 13:06:58.330935 20475 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:06:58.332134 20413 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:58.333035 20337 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:58.333469 20337 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:58.334450 20337 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:58.335561 20337 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:58.336258 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:58.309116 20486 consensus_queue.cc:1059] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:06:58.339476 13946 raft_consensus.cc:1217] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Deduplicated request from leader. Original: 10.9->[12.10-12.10]   Dedup: 12.10->[]
I20260709 13:06:58.340054 13946 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:06:58.340677 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3218) took 1765 ms. Trace:
I20260709 13:06:58.341008 13946 rpcz_store.cc:276] 0709 13:06:56.575696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.052353 (+1476657us) service_pool.cc:224] Handling call
0709 13:06:58.096978 (+ 44625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.339387 (+242409us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b54000e8ba0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33aadc0 00007fb2c34351c0 00007fb2c3436231 00007fb2c3432d8f 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c
0709 13:06:58.340391 (+  1004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.340618 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52029312}
I20260709 13:06:58.251931 20325 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:06:58.344645 13948 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:06:58.345364 13943 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:58.349179 20477 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:06:58.355995 20454 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.812s	user 0.002s	sys 0.000s
I20260709 13:06:58.358201 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1108) took 2391 ms. Trace:
I20260709 13:06:58.358685 13944 rpcz_store.cc:276] 0709 13:06:55.967281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.313414 (+1346133us) service_pool.cc:224] Handling call
0709 13:06:57.546590 (+233176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.670700 (+124110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.357949 (+687249us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54000b9d50. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:58.358132 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535190912}
I20260709 13:06:58.359962 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:58.360589 13944 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:06:58.363328 13945 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:58.364778 13945 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:58.365087 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.312251 20309 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:58.366955 20481 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:58.367326 20309 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:58.368369 20478 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:06:58.375298 13867 raft_consensus.cc:2410] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:58.375407 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:06:58.361631 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:58.377025 13944 raft_consensus.cc:2410] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:06:58.377267 13948 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:06:58.384347 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.385550 13871 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:58.294137 20449 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:58.393128 14019 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:58.394508 13872 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:06:58.397105 14524 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:58.400491 13872 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:06:58.297215 19997 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:58.423210 14018 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:06:58.389674 20410 consensus_meta.cc:287] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.646s	user 0.005s	sys 0.009s
I20260709 13:06:58.312471 20444 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:58.440780 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:58.441977 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 66699.2us
I20260709 13:06:58.313223 20313 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:58.381338 20298 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.374854 20460 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:58.469579 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.463070 14019 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:06:58.396610 20436 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:58.433121 19997 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.479056 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2970) took 1048 ms. Trace:
I20260709 13:06:58.475075 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:58.435565 20353 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:06:58.451480 20484 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.195s	user 0.001s	sys 0.001s
I20260709 13:06:58.454200 20180 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:58.492410 13838 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:58.479525 13942 rpcz_store.cc:276] 0709 13:06:57.430559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.236977 (+806418us) service_pool.cc:224] Handling call
0709 13:06:58.237350 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.237594 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:58.267794 (+ 30200us) log.cc:844] Serialized 14 byte log entry
0709 13:06:58.478550 (+210756us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:58.478628 (+    78us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:58.478698 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.478803 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:58.478960 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99153,"prepare.queue_time_us":71249,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"thread_start_us":127259,"threads_started":1}
I20260709 13:06:58.419064 20381 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:06:58.502198 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.506520 13867 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:06:58.501544 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.468914 20298 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:58.514300 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 202473us
I20260709 13:06:58.515311 13915 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:58.501436 13872 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:06:58.516405 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:58.520862 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:58.522017 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:06:58.468215 20460 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:58.533834 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.492229 19997 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:58.537992 13871 raft_consensus.cc:2368] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18.
I20260709 13:06:58.543627 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.544028 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.451741 20410 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:58.500312 20180 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:06:58.536801 20460 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:58.505651 20353 raft_consensus.cc:697] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:58.555644 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.566900 19782 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:58.566093 20180 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:58.582517 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.697094 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.591406 20325 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:58.438474 20461 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 13:06:58.715137 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:06:58.681077 20354 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.001s
I20260709 13:06:58.754015 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:58.523221 20485 consensus_queue.cc:1059] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.215s
I20260709 13:06:58.720010 20490 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:58.748914 13874 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:58.529076 20348 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:58.717819 20091 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:58.777565 20091 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.778038 20091 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:58.784000 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:58.784642 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:58.785167 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:58.785530 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:58.785849 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.790680 20485 raft_consensus.cc:3055] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:58.791254 20485 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:06:58.791566 20244 raft_consensus.cc:3055] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:58.792214 20244 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:06:58.793768 20244 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:58.775596 20445 consensus_meta.cc:287] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.626s	user 0.016s	sys 0.002s
W20260709 13:06:58.777146 20458 consensus_meta.cc:287] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.785s	user 0.007s	sys 0.003s
I20260709 13:06:58.795585 20446 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:58.795686 20458 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:58.796098 20446 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:58.797163 20446 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:58.798575 13873 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:58.800328 14553 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:06:58.800745 14553 pending_rounds.cc:85] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting all ops after (but not including) 9
I20260709 13:06:58.801021 14553 pending_rounds.cc:107] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:06:58.801260 14553 raft_consensus.cc:2889] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:58.792258 20485 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:58.802732 20485 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:58.808017 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.758538 20354 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:58.767243 20490 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:58.794668 20462 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.774072 20410 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:58.815716 14558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59630 (request call id 866) took 1322 ms. Trace:
I20260709 13:06:58.829295 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:58.833148 14558 rpcz_store.cc:276] 0709 13:06:57.493209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.495304 (+  2095us) service_pool.cc:224] Handling call
0709 13:06:58.815384 (+1320080us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b5400710f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:58.815617 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1006763264}
I20260709 13:06:58.809995 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.829573 13868 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:58.834321 13870 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:06:58.835512 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.835644 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.836279 13870 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:58.769479 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.837818 13872 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:58.838945 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:58.830458 20462 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:58.840193 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:58.841019 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 348673us
I20260709 13:06:58.841269 20462 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:58.756577 20325 raft_consensus.cc:697] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:58.838598 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.850719 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.857458 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:58.859892 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.860551 13874 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
W20260709 13:06:58.861913 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.862027 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.885691 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.887006 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:58.890688 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48648.2us
I20260709 13:06:58.891716 13870 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:58.892663 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:58.830633 20348 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:58.843240 20388 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.757s	user 0.022s	sys 0.013s
I20260709 13:06:58.892745 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.913605 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:58.826993 20490 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:58.810307 20415 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:58.816504 20354 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:58.915441 14558 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:58.916608 13915 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:06:58.917722 13950 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.006s	sys 0.012s
W20260709 13:06:58.863332 20319 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.591s	user 0.009s	sys 0.002s
W20260709 13:06:58.890025 20355 consensus_meta.cc:287] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.757s	user 0.021s	sys 0.000s
W20260709 13:06:58.923725 13947 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.005s	sys 0.016s
I20260709 13:06:58.853858 20495 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.852376 20436 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:58.927775 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:58.889926 20286 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.890271 20446 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:58.937233 13989 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.541s (SENT) [suppressed 38 similar messages]
W20260709 13:06:58.921396 20367 consensus_meta.cc:287] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.898s	user 0.011s	sys 0.002s
I20260709 13:06:58.956478 14558 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:06:58.955039 20279 consensus_meta.cc:287] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.628s	user 0.003s	sys 0.010s
W20260709 13:06:58.962420 20472 consensus_meta.cc:287] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.819s	user 0.016s	sys 0.008s
I20260709 13:06:58.957791 20495 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:58.922583 19983 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:58.914506 19975 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.049s
W20260709 13:06:58.963361 20337 consensus_meta.cc:287] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.627s	user 0.012s	sys 0.003s
I20260709 13:06:58.954495 19997 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:58.924656 20324 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:58.965241 20316 consensus_meta.cc:287] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.639s	user 0.019s	sys 0.001s
I20260709 13:06:58.826287 20353 consensus_queue.cc:237] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:58.949448 20359 consensus_meta.cc:287] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.948s	user 0.013s	sys 0.000s
I20260709 13:06:58.920778 20461 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:58.966172 20286 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:58.959095 20334 consensus_meta.cc:287] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.763s	user 0.005s	sys 0.006s
I20260709 13:06:58.921737 20388 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:58.925949 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:58.914865 13870 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
W20260709 13:06:58.945960 13947 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.005s	sys 0.020s
W20260709 13:06:58.945960 13950 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.007s	sys 0.013s
I20260709 13:06:58.983577 14560 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:06:58.983155 19997 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:59.277721 14017 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:06:59.196448 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.267733 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 1033) took 1677 ms. Trace:
I20260709 13:06:59.306154 19997 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:59.278362 14525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.000s	sys 0.008s
I20260709 13:06:59.306746 14552 rpcz_store.cc:276] 0709 13:06:57.590042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.592060 (+  2018us) service_pool.cc:224] Handling call
0709 13:06:59.267014 (+1674954us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400748454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:59.267371 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3395198208}
W20260709 13:06:59.308197 14525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.000s	sys 0.010s
W20260709 13:06:59.199730 13946 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.010s	sys 0.018s
I20260709 13:06:59.307634 19997 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:59.309165 13946 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.010s	sys 0.036s
W20260709 13:06:59.294559 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.276684 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.439915 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:59.271335 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 763) took 1544 ms. Trace:
I20260709 13:06:59.314751 14018 rpcz_store.cc:276] 0709 13:06:57.726504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.726632 (+   128us) service_pool.cc:224] Handling call
0709 13:06:59.270638 (+1544006us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54003bbfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:59.271005 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1424346496}
I20260709 13:06:59.315624 19997 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:59.198602 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:59.189291 13841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.000s	sys 0.036s
I20260709 13:06:59.318441 13868 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:06:59.318871 13841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.000s	sys 0.054s
I20260709 13:06:58.979868 19983 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:59.266505 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.264994 13987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.020s	user 0.024s	sys 0.029s
W20260709 13:06:59.322606 13987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.078s	user 0.030s	sys 0.034s
W20260709 13:06:59.323179 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 1.07843e+06us
W20260709 13:06:59.309665 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.326292 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 400983us
W20260709 13:06:59.306813 14017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 761) took 1582 ms (client timeout 2074 ms). Trace:
I20260709 13:06:59.230681 13949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1105) took 3568 ms. Trace:
I20260709 13:06:59.281347 13768 catalog_manager.cc:5697] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:06:59.272574 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 765) took 1085 ms. Trace:
I20260709 13:06:59.327737 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:06:59.062870 20431 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.789s	user 0.023s	sys 0.004s
W20260709 13:06:59.071600 20430 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.801s	user 0.026s	sys 0.009s
I20260709 13:06:59.379035 20430 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.380200 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:58.989980 20316 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:58.961102 20319 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.389145 14551 raft_consensus.cc:2393] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:59.027607 20436 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:58.975452 20279 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.392446 20436 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:59.074697 20448 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.846s	user 0.005s	sys 0.010s
I20260709 13:06:59.395612 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:59.395427 20448 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:59.104936 20449 consensus_meta.cc:287] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.664s	user 0.012s	sys 0.000s
I20260709 13:06:59.399201 14559 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:06:59.400408 14553 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:59.401976 14561 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:58.984721 20158 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.840s	user 0.007s	sys 0.007s
I20260709 13:06:59.409373 20499 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.410920 20281 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:06:59.412662 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.445111 20499 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.445300 20281 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.445986 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:59.447345 13871 raft_consensus.cc:2410] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 13:06:59.371519 14017 rpcz_store.cc:269] 0709 13:06:57.724375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.724510 (+   135us) service_pool.cc:224] Handling call
0709 13:06:59.306467 (+1581957us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b540023ad54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:59.306729 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3419009920}
I20260709 13:06:59.449056 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.449186 20281 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:59.450114 13942 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:06:59.450595 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.414117 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:59.414191 20461 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.415645 20430 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:58.978664 20495 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:59.192149 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 300464us
I20260709 13:06:59.397532 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.417513 20513 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:06:59.418236 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 92469.8us
I20260709 13:06:59.419317 20498 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:06:59.057907 20428 consensus_meta.cc:287] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.684s	user 0.023s	sys 0.013s
W20260709 13:06:58.984083 20313 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.004s
I20260709 13:06:59.034950 20325 consensus_queue.cc:237] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.424898 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 1036) took 1089 ms. Trace:
I20260709 13:06:59.426509 14558 raft_consensus.cc:1240] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:59.428005 20388 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:59.429426 14023 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 23: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:59.430727 20359 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:59.431833 20334 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.320438 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.435163 13768 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 10 to 11, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 11 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:59.438494 20286 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.442977 20449 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.375625 14019 rpcz_store.cc:276] 0709 13:06:58.186799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.193835 (+  7036us) service_pool.cc:224] Handling call
0709 13:06:59.272107 (+1078272us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540004b354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:59.272357 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394588032}
I20260709 13:06:59.455127 20502 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.478842 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:59.210978 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.372728 13949 rpcz_store.cc:276] 0709 13:06:55.662426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.271104 (+1608678us) service_pool.cc:224] Handling call
0709 13:06:57.271514 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.601128 (+329614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.229890 (+1628762us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54001c6454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:06:59.230386 (+   496us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":755419392}
I20260709 13:06:59.474452 13868 raft_consensus.cc:2410] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:06:59.287967 19782 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.173s
I20260709 13:06:59.463161 14551 rpcz_store.cc:276] 0709 13:06:58.335692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.335883 (+   191us) service_pool.cc:224] Handling call
0709 13:06:59.424546 (+1088663us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400750154. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:59.424814 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2418000768}
I20260709 13:06:59.459592 20498 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.494452 20492 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:59.496379 20502 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:06:59.497959 20248 consensus_meta.cc:287] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.294s	user 0.021s	sys 0.004s
I20260709 13:06:59.498174 20381 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:06:59.498622 20248 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:59.498765 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:06:59.499811 20223 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
W20260709 13:06:59.500808 13943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.011s	sys 0.006s
W20260709 13:06:59.501310 13943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.011s	sys 0.007s
I20260709 13:06:59.501536 20223 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:59.502321 13873 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:59.502470 13874 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:06:59.503153 20495 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:59.504901 20376 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:59.506527 20324 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:59.507675 20097 raft_consensus.cc:3055] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:59.512882 20279 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:59.514463 20428 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.118410 20460 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.120646 20490 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.519876 13871 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:59.523310 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:59.524840 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:58.966996 20494 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:06:59.541220 13875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3241) took 1304 ms (client timeout 1641 ms). Trace:
I20260709 13:06:59.545228 20313 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.545291 14016 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:59.548465 20248 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.549156 20359 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.571517 20359 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.550304 20158 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:59.567369 13875 rpcz_store.cc:269] 0709 13:06:58.237099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.210216 (+973117us) service_pool.cc:224] Handling call
0709 13:06:59.541145 (+330929us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:59.486330 20499 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.487581 20498 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:59.501755 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:59.559840 20463 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.570524 20486 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.570082 20248 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.551807 20513 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.576862 20503 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.577621 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:59.577698 20359 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.579797 20286 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:59.559909 20334 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:59.543957 20097 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:59.580128 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:06:59.012632 20501 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.558s
I20260709 13:06:59.596199 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:58.976099 20472 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.589648 20286 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:06:59.604558 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 103017us
I20260709 13:06:59.604902 13869 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:59.594256 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.605612 13866 raft_consensus.cc:2368] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18.
I20260709 13:06:59.623751 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:59.636914 13945 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 14: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:59.638841 20337 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 13:06:59.640400 20490 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.615602 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2966) took 2186 ms. Trace:
I20260709 13:06:59.610133 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2971) took 1949 ms. Trace:
W20260709 13:06:59.618516 14023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 762) took 1893 ms (client timeout 2152 ms). Trace:
I20260709 13:06:59.623103 20463 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:59.647668 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.648792 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.650648 20430 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:59.650913 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71003.9us
I20260709 13:06:59.651502 20430 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.652123 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
W20260709 13:06:59.656694 20517 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:06:59.650389 20337 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:59.660884 20337 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:59.661391 20337 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:06:59.661854 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.661883 20337 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:59.606288 13988 leader_election.cc:336] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851): Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 2.333s (SENT)
W20260709 13:06:59.602006 20228 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.304s	user 0.008s	sys 0.038s
I20260709 13:06:59.612116 20316 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:06:59.644375 14023 rpcz_store.cc:269] 0709 13:06:57.725560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.725688 (+   128us) service_pool.cc:224] Handling call
0709 13:06:59.602043 (+1876355us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b540023bc54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:59.618426 (+ 16383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3890366848}
I20260709 13:06:59.697026 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2974) took 1690 ms. Trace:
I20260709 13:06:59.696784 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:59.697901 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.694321 13989 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.010s	sys 0.015s
I20260709 13:06:59.605990 20507 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:59.610002 20502 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.606948 20449 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:59.609943 20448 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.604423 20158 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:59.649886 20463 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.713989 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:06:59.703909 20324 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.604333 20472 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.610673 20494 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:59.712235 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.709024 20279 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.614260 20513 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.622480 20461 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:59.615859 19975 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:59.705628 13873 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
I20260709 13:06:59.718149 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.612032 20248 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:59.718544 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:59.674793 20430 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:06:59.709779 13989 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.010s	sys 0.015s
W20260709 13:06:59.721001 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 125424us
I20260709 13:06:59.732345 20244 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:59.712954 20334 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.641819 13947 rpcz_store.cc:276] 0709 13:06:57.429363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.214426 (+785063us) service_pool.cc:224] Handling call
0709 13:06:58.241615 (+ 27189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.241913 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:59.326611 (+1084698us) log.cc:844] Serialized 14 byte log entry
0709 13:06:59.594238 (+267627us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:59.614872 (+ 20634us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:59.615003 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.615156 (+   153us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.615408 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25662,"prepare.queue_time_us":100936,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":166,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":55190,"thread_start_us":1289521,"threads_started":2,"wal-append.queue_time_us":287646}
I20260709 13:06:59.737064 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:59.752207 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.752358 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:59.623329 20442 consensus_queue.cc:1059] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.108s
I20260709 13:06:59.753180 13873 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:06:59.622232 20486 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.752606 13949 raft_consensus.cc:2410] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:06:59.617267 20286 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.622886 20097 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.642539 13950 rpcz_store.cc:276] 0709 13:06:57.537000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.266692 (+729692us) service_pool.cc:224] Handling call
0709 13:06:58.267104 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.286604 (+ 19500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:58.286638 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:58.287172 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:06:59.371362 (+1084190us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:59.451208 (+ 79846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.457652 (+  6444us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:59.457721 (+    69us) raft_consensus.cc:1685] finished
0709 13:06:59.457731 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.486681 (+ 28950us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6441,"raft.queue_time_us":20102,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":6523,"thread_start_us":927558,"threads_started":1,"wal-append.queue_time_us":726236}
I20260709 13:06:59.753770 20442 raft_consensus.cc:3055] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:59.756313 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.705889 20228 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.761039 13868 raft_consensus.cc:2393] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:59.623540 20503 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.759948 13870 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:59.699295 13948 rpcz_store.cc:276] 0709 13:06:58.006107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.378513 (+372406us) service_pool.cc:224] Handling call
0709 13:06:58.378883 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.928173 (+549290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:58.928225 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:59.230143 (+301918us) log.cc:844] Serialized 26 byte log entry
0709 13:06:59.483685 (+253542us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:59.492557 (+  8872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.537071 (+ 44514us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:59.655649 (+118578us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:59.673477 (+ 17828us) raft_consensus.cc:1685] finished
0709 13:06:59.673521 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.696813 (+ 23292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73976,"raft.queue_time_us":141585,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":49,"spinlock_wait_cycles":4218112,"thread_start_us":688750,"threads_started":2,"wal-append.queue_time_us":347972}
I20260709 13:06:59.748719 20448 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:06:59.784078 20180 consensus_meta.cc:287] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.035s	user 0.015s	sys 0.004s
I20260709 13:06:59.784642 20180 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:06:59.785262 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 131205us
W20260709 13:06:59.786500 20091 consensus_meta.cc:287] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.008s	user 0.011s	sys 0.017s
I20260709 13:06:59.786396 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:59.787089 20091 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.788511 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:59.760677 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:59.758227 20279 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.783910 13875 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:06:59.792711 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:06:59.793195 20298 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.186s	user 0.013s	sys 0.006s
I20260709 13:06:59.793756 20498 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.758647 20442 raft_consensus.cc:740] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:06:59.750826 20445 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.077s
I20260709 13:06:59.798086 20445 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:06:59.798182 20334 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.789649 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.800482 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:59.789790 20448 raft_consensus.cc:697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:59.798566 14555 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:59.792910 13872 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:59.793691 13875 raft_consensus.cc:2410] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:06:59.782582 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 845276us
I20260709 13:06:59.803926 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:59.805388 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:59.806464 14019 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:59.806253 20430 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:59.808058 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.809402 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:59.801522 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:59.810580 14018 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:59.811093 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:59.815647 14023 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:06:59.798586 20445 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:06:59.812228 20091 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.813306 20472 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:06:59.756758 20324 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.810668 20428 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.817785 20445 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.793674 20298 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.811009 20448 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.820868 20298 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.822969 20279 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:59.824232 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.825057 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.825335 13868 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 23: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:59.826004 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3259) took 1054 ms. Trace:
I20260709 13:06:59.811935 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:59.826321 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.826516 13871 rpcz_store.cc:276] 0709 13:06:58.771614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.824671 (+1053057us) service_pool.cc:224] Handling call
0709 13:06:59.825940 (+  1269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:59.826997 13942 raft_consensus.cc:2368] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17.
I20260709 13:06:59.827611 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.827971 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:59.828150 13949 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:59.828617 13870 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:06:59.829170 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:59.829432 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.816290 13950 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:59.831081 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:59.831550 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:59.832016 20503 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:59.832399 14018 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:59.832274 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:59.833302 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:59.833621 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:59.833966 20180 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:59.834122 14551 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:06:59.786312 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:59.833303 20460 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.835403 20228 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.834980 14017 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:59.837103 20511 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.289s	user 0.000s	sys 0.004s
I20260709 13:06:59.836910 20337 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.837778 13875 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:59.838464 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3358) took 1252 ms. Trace:
I20260709 13:06:59.839123 13914 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:06:59.840569 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:59.841518 14024 raft_consensus.cc:2393] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:06:59.843851 14018 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:59.843957 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:59.844686 14024 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
W20260709 13:06:59.845533 13866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3234) took 1683 ms (client timeout 1641 ms). Trace:
I20260709 13:06:59.845655 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:59.846510 14023 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:59.846643 20337 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:06:59.845886 13867 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:06:59.846693 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:06:59.800376 20158 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.848449 13868 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:06:59.811208 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:59.829931 13942 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:59.838939 13875 rpcz_store.cc:276] 0709 13:06:58.585622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.837086 (+1251464us) service_pool.cc:224] Handling call
0709 13:06:59.837494 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.838193 (+   699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.838388 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:59.833606 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:59.827657 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:06:59.849910 14016 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
W20260709 13:06:59.846089 13866 rpcz_store.cc:269] 0709 13:06:58.162079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.162206 (+   127us) service_pool.cc:224] Handling call
0709 13:06:59.845229 (+1683023us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54000a8054. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:59.845425 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3315310976}
I20260709 13:06:59.851413 13948 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:06:59.856668 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3354) took 1307 ms. Trace:
W20260709 13:06:59.860842 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 60660.6us
I20260709 13:06:59.860559 13867 rpcz_store.cc:276] 0709 13:06:58.542782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.542933 (+   151us) service_pool.cc:224] Handling call
0709 13:06:59.845460 (+1302527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:59.847386 (+  1926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.849010 (+  1624us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54000a8050. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:06:59.850279 (+  1269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2940694784}
I20260709 13:06:59.859093 13870 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:06:59.866499 14019 raft_consensus.cc:2410] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:06:59.855195 20472 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.857095 20490 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:59.812506 20313 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:59.877097 13867 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:59.869234 20460 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:59.882423 20460 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:59.882660 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 94289.6us
I20260709 13:06:59.882941 20460 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:59.883257 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1127) took 1502 ms. Trace:
I20260709 13:06:59.884071 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:06:59.884248 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:59.885107 14015 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:59.796214 20494 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.887001 20324 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:06:59.887984 20486 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:59.889691 20428 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:06:59.890829 20158 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.894586 13873 raft_consensus.cc:2393] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 21: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:06:59.895534 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:59.896936 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:59.898026 13941 raft_consensus.cc:2410] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:06:59.898520 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:59.881518 20325 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:59.885702 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:59.907274 13871 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:59.907862 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:06:59.909087 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:59.870329 20337 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:59.915181 13765 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57648 (request call id 282) took 21032 ms (client timeout 15000 ms). Trace:
I20260709 13:06:59.914201 14017 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
W20260709 13:06:59.908491 20330 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.311s	user 0.060s	sys 0.066s
W20260709 13:06:59.911793 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47326us
I20260709 13:06:59.913980 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3362) took 1151 ms. Trace:
I20260709 13:06:59.926409 13871 rpcz_store.cc:276] 0709 13:06:58.758735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.856466 (+1097731us) service_pool.cc:224] Handling call
0709 13:06:59.876845 (+ 20379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.909515 (+ 32670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.909780 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:59.928995 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:59.930364 14019 raft_consensus.cc:2410] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:06:59.935268 20469 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.936026 20469 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:06:59.900906 20492 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:06:59.939332 20492 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:59.940681 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:59.942845 20490 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:59.943249 20490 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.943640 20490 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:59.944298 20097 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:59.944803 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:59.945619 14018 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:06:59.947880 14021 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:59.949014 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:59.949856 14021 raft_consensus.cc:2410] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:06:59.946655 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:59.950743 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:59.951488 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:59.959611 20428 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:59.961110 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:06:59.896848 13768 catalog_manager.cc:5697] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 10 to 11, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 11 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.954568 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3363) took 1192 ms. Trace:
I20260709 13:06:59.950264 20337 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.883678 13943 rpcz_store.cc:276] 0709 13:06:58.381090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.420897 (+ 39807us) service_pool.cc:224] Handling call
0709 13:06:58.421404 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.856146 (+434742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:58.856186 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:58.889074 (+ 32888us) log.cc:844] Serialized 26 byte log entry
0709 13:06:59.501952 (+612878us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:59.502043 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.502198 (+   155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:59.836203 (+334005us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:06:59.879046 (+ 42843us) raft_consensus.cc:1685] finished
0709 13:06:59.879070 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.883175 (+  4105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91171,"raft.queue_time_us":117909,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":119300,"spinlock_wait_cycles":27364736,"thread_start_us":930219,"threads_started":2,"wal-append.queue_time_us":630595}
I20260709 13:06:59.957710 13942 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 11.
I20260709 13:06:59.927798 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:59.956959 20499 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.966516 13866 rpcz_store.cc:276] 0709 13:06:58.761978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.860557 (+1098579us) service_pool.cc:224] Handling call
0709 13:06:59.908161 (+ 47604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.954132 (+ 45971us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:59.954165 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:59.954223 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:59.954304 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.954378 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.954492 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:59.908748 20445 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:59.983898 14022 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:06:59.925554 20330 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.501s	user 0.061s	sys 0.067s
I20260709 13:06:59.984515 14020 raft_consensus.cc:2368] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19.
I20260709 13:06:59.984598 13949 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
W20260709 13:06:59.987293 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 61253.2us
I20260709 13:06:59.906337 20472 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.989184 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:06:59.989941 14018 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:59.990401 20481 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.990583 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:06:59.991037 20481 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:06:59.992247 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:59.993626 20481 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:06:59.994553 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:06:59.994760 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:06:59.996026 20481 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:00.000506 14015 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:06:59.986549 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:06:59.996284 14020 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:59.998085 14017 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:06:59.998713 14023 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:00.000617 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:00.008436 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:00.009603 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:00.012418 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:00.010499 14021 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:00.018915 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:00.023044 14559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 871) took 1111 ms. Trace:
I20260709 13:07:00.023522 14559 rpcz_store.cc:276] 0709 13:06:58.911736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.911916 (+   180us) service_pool.cc:224] Handling call
0709 13:06:58.912417 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:59.467204 (+554787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.022671 (+555467us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b54007102d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:00.022959 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1117621504}
I20260709 13:07:00.025133 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:00.026139 13945 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:59.998728 20158 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:00.026968 20158 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:00.027904 20337 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:00.028052 20158 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:00.028589 20158 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:07:00.030728 13989 leader_election.cc:336] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.613s (SENT)
I20260709 13:07:00.031436 14019 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:00.032457 13948 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 13: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:07:00.035617 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 72774.2us
I20260709 13:07:00.034477 20499 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:00.039675 20472 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:00.039978 20499 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:00.005002 20481 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:00.041412 20481 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:00.041797 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3365) took 1190 ms. Trace:
I20260709 13:07:00.042268 13874 rpcz_store.cc:276] 0709 13:06:58.851765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.861676 (+  9911us) service_pool.cc:224] Handling call
0709 13:07:00.041603 (+1179927us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54003ac5d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:00.041735 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469912320}
I20260709 13:07:00.042660 20223 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:59.915627 13765 rpcz_store.cc:269] 0709 13:06:38.883209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.889870 (+  6661us) service_pool.cc:224] Handling call
0709 13:06:59.914889 (+21025019us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:55.737476 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:55.738267 (+   791us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:55.738351 (+    84us) write_op.cc:435] Acquired schema lock
0709 13:06:55.738371 (+    20us) tablet.cc:662] Decoding operations
0709 13:06:55.749008 (+ 10637us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:55.749138 (+   130us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:55.749170 (+    32us) tablet.cc:685] Acquiring locks for 30 operations
0709 13:06:55.751675 (+  2505us) tablet.cc:701] Row locks acquired
0709 13:06:55.751732 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:06:55.751953 (+   221us) write_op.cc:270] Start()
0709 13:06:55.752118 (+   165us) write_op.cc:276] Timestamp: P: 1783602415751867 usec, L: 0
0709 13:06:55.752161 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 13:06:55.770153 (+ 17992us) log.cc:844] Serialized 11798 byte log entry
0709 13:06:55.901539 (+131386us) op_driver.cc:464] REPLICATION: finished
0709 13:06:55.903175 (+  1636us) write_op.cc:301] APPLY: starting
0709 13:06:55.903285 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:55.982322 (+ 79037us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:55.982468 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:56.075955 (+ 93487us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:56.076457 (+   502us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=60,key_file_lookups=60,delta_file_lookups=60,mrs_lookups=0
0709 13:06:56.076505 (+    48us) write_op.cc:312] APPLY: finished
0709 13:06:56.091242 (+ 14737us) log.cc:844] Serialized 260 byte log entry
0709 13:06:56.091508 (+   266us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:56.091798 (+   290us) write_op.cc:454] Released schema lock
0709 13:06:56.091912 (+   114us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:56.092439 (+   527us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:56.092475 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":114932,"reported_tablets":52,"spinlock_wait_cycles":896,"child_traces":[["op",{"apply.queue_time_us":1385,"apply.run_cpu_time_us":45158,"apply.run_wall_time_us":189240,"cfile_cache_hit":122,"cfile_cache_hit_bytes":129347,"num_ops":30,"prepare.queue_time_us":25614,"prepare.run_cpu_time_us":10921,"prepare.run_wall_time_us":57003,"raft.queue_time_us":52391,"raft.run_cpu_time_us":1318,"raft.run_wall_time_us":1546,"replication_time_us":149162,"thread_start_us":92410,"threads_started":4,"wal-append.queue_time_us":51809,"wal-append.run_cpu_time_us":39297,"wal-append.run_wall_time_us":1276836}]]}
I20260709 13:07:00.044560 20442 consensus_queue.cc:260] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:00.048875 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 924) took 1733 ms. Trace:
I20260709 13:07:00.049357 13869 rpcz_store.cc:276] 0709 13:06:58.314994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.315580 (+   586us) service_pool.cc:224] Handling call
0709 13:06:58.315938 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.697737 (+1381799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.048529 (+350792us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540026bbd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:00.048787 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2963131264}
I20260709 13:07:00.051769 20091 raft_consensus.cc:3055] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:00.052397 20091 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:00.054419 20436 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:00.055060 20436 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.055302 20469 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:00.055490 20436 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:00.058359 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:00.055060 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:00.060492 13941 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:00.029932 20158 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:00.063740 20158 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:00.063802 14555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 869) took 1772 ms. Trace:
I20260709 13:07:00.064287 14555 rpcz_store.cc:276] 0709 13:06:58.291372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.291583 (+   211us) service_pool.cc:224] Handling call
0709 13:06:58.292503 (+   920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.835223 (+1542720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.063458 (+228235us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54006b2e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:00.063729 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3447039872}
I20260709 13:07:00.009228 20428 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:00.067359 20428 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:00.053851 20091 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:00.061838 13941 raft_consensus.cc:1240] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:00.069768 20091 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:00.073379 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.077579 14018 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:00.082247 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:00.070384 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:00.091128 14023 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
W20260709 13:07:00.099599 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 87074.6us
W20260709 13:07:00.100633 13989 leader_election.cc:336] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.641s (SENT)
I20260709 13:07:00.101061 20319 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:00.062036 20334 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:00.106599 20334 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:00.111454 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:00.112866 14018 raft_consensus.cc:2410] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:07:00.113054 13949 raft_consensus.cc:1240] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:00.115494 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:00.116115 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:00.116506 14015 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:00.117038 14017 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:00.117405 14019 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:07:00.115576 14021 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:00.118891 14022 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:00.119954 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:00.120777 14023 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:00.101101 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:00.124650 13989 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.641s (SENT)
W20260709 13:07:00.122558 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.127123 13989 leader_election.cc:336] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.642s (SENT)
I20260709 13:07:00.126387 13768 catalog_manager.cc:5697] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 8 to 9, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 9 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:00.070952 20498 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:00.148123 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:00.148491 20494 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:00.149362 20469 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:07:00.149796 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:00.150494 14018 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:00.150882 20469 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:00.151304 20469 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:00.152367 20469 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:00.157083 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 214457us
I20260709 13:07:00.148201 13768 catalog_manager.cc:5697] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 12 to 13, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 13 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:00.089538 20486 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.162021 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:00.162420 20486 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:00.163043 13871 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:00.164788 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:00.165115 20486 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:00.165824 13866 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:00.145357 20460 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:00.167479 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:00.168453 13874 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:00.168674 20460 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:00.118309 20442 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:00.183236 13949 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:00.153811 20469 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:00.070354 20330 consensus_queue.cc:1243] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Peer 4dd265d61fc141a48ca9d3d5c8f52921 log is divergent from this leader: its last log entry 19.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:07:00.188896 14018 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:07:00.190266 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:00.233932 13867 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:00.192034 14522 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:00.194662 20286 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:00.178592 20097 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.197773 13943 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:00.198593 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 71525.8us
I20260709 13:07:00.199191 13941 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:00.202478 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:00.255206 14015 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:00.206493 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:00.206280 13768 catalog_manager.cc:5697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:00.207541 20494 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:00.208595 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:00.209431 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:00.201691 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:00.269150 13871 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:00.267344 14020 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:00.186446 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:00.272460 13866 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:00.275703 14023 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:07:00.278690 13945 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
W20260709 13:07:00.283362 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 135732us
I20260709 13:07:00.284498 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:00.256624 20324 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:00.335678 13989 leader_election.cc:336] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.645s (SENT)
I20260709 13:07:00.295933 19782 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:00.338651 13989 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:00.338948 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:00.267938 20494 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:00.360889 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:00.296200 20472 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:00.270825 20097 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:00.353048 19782 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.373696 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:00.378762 14020 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:00.379271 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:00.380414 13867 raft_consensus.cc:2410] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:07:00.378535 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:00.389842 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:00.391070 14015 raft_consensus.cc:2410] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:07:00.371585 20498 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:00.404974 14023 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:00.420115 13768 catalog_manager.cc:5697] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:00.387223 19782 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:00.426210 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:00.427768 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 141844us
I20260709 13:07:00.478499 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:00.428283 20309 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:00.429369 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:00.373677 20330 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:00.432101 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.443506 20494 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:00.445127 20500 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:00.448190 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:00.491200 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:00.448836 19983 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:00.450659 13944 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:00.515312 13941 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:07:00.452971 20492 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.000s
W20260709 13:07:00.457491 14016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 768) took 2237 ms (client timeout 2176 ms). Trace:
I20260709 13:07:00.459273 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1197) took 1090 ms. Trace:
I20260709 13:07:00.515529 13944 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:00.488276 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.520323 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.477012 13874 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:07:00.497226 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:00.497659 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:00.499817 13950 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:00.504293 20463 consensus_queue.cc:1059] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:00.509415 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:00.524821 20097 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:00.525441 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 245755us
I20260709 13:07:00.525503 13943 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
W20260709 13:07:00.526626 13989 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.648s (SENT)
I20260709 13:07:00.526876 20158 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:00.527554 14020 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:00.506163 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:00.527859 20330 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:00.523926 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:00.525635 20472 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:00.527277 13989 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:00.505466 20337 consensus_queue.cc:1059] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:00.511785 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:07:00.517293 14016 rpcz_store.cc:269] 0709 13:06:58.219670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.226096 (+  6426us) service_pool.cc:224] Handling call
0709 13:07:00.457226 (+2231130us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54003255d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:00.457386 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2850219264}
I20260709 13:07:00.541451 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:00.557299 14556 raft_consensus.cc:2410] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:07:00.529776 13944 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:00.568418 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:00.519100 14552 rpcz_store.cc:276] 0709 13:06:59.371943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.387686 (+ 15743us) service_pool.cc:224] Handling call
0709 13:06:59.388347 (+   661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.454926 (+ 66579us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:06:59.587209 (+132283us) log.cc:844] Serialized 14 byte log entry
0709 13:06:59.865094 (+277885us) log.cc:844] Serialized 14 byte log entry
0709 13:06:59.865256 (+   162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:59.865307 (+    51us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:06:59.869031 (+  3724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.110152 (+241121us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b1800390520. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:07:00.123012 (+ 12860us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b1800390520. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:07:00.458984 (+335972us) spinlock_profiling.cc:243] Waited 66.1 ms on lock 0x7b54004b0fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:00.459043 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:00.459206 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5634,"prepare.queue_time_us":329591,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":85,"raft.queue_time_us":15377,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":3721,"spinlock_wait_cycles":176780928,"thread_start_us":355049,"threads_started":2,"wal-append.queue_time_us":316101}
I20260709 13:07:00.613692 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:00.615221 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:07:00.637403 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44871.4us
I20260709 13:07:00.640676 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:00.616298 20158 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:00.663709 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 137141us
W20260709 13:07:00.664768 13989 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.668s (SENT)
I20260709 13:07:00.665194 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:00.627111 20544 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:07:00.669761 13874 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:00.670096 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:00.670833 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:00.671607 13874 raft_consensus.cc:2368] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 18. Current term is 19.
I20260709 13:07:00.672539 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:00.673210 13874 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:07:00.674367 20324 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:00.614900 20498 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:00.676666 20309 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:00.677192 20309 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:00.677793 20091 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.000s
I20260709 13:07:00.532310 20492 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:00.525671 19983 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:00.615823 20472 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:00.680215 13944 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:00.683507 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:00.620241 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:00.688349 20436 consensus_meta.cc:287] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.632s	user 0.009s	sys 0.004s
I20260709 13:07:00.702284 13871 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
W20260709 13:07:00.707216 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 66670.6us
I20260709 13:07:00.712136 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:00.716773 14020 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:00.709676 20494 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:00.724146 13873 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:00.709501 20180 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:00.738428 20518 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.220s	user 0.008s	sys 0.000s
I20260709 13:07:00.740795 20472 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:00.743784 19983 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:00.777253 20490 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.833s	user 0.012s	sys 0.004s
I20260709 13:07:00.781080 20485 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:07:00.781381 20490 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:00.714282 20359 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:00.757230 13873 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:07:00.811045 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:07:00.793867 20540 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.366s	user 0.000s	sys 0.001s
I20260709 13:07:00.717782 20436 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:00.838188 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.832897 13873 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:07:00.856801 20532 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.412s	user 0.001s	sys 0.000s
I20260709 13:07:00.824851 20472 raft_consensus.cc:3055] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:07:00.895877 14024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.038s	sys 0.037s
I20260709 13:07:00.904557 20445 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
W20260709 13:07:00.906267 13872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3367) took 2047 ms (client timeout 1516 ms). Trace:
I20260709 13:07:00.909922 14522 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:00.758400 20180 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:00.933048 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.913877 20481 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.872s	user 0.007s	sys 0.008s
W20260709 13:07:00.938700 20539 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.110s	user 0.001s	sys 0.001s
W20260709 13:07:00.973554 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 112956us
W20260709 13:07:00.954823 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.962697 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.935720 20511 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:07:00.914006 20330 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:00.972330 14024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.040s	sys 0.038s
I20260709 13:07:00.952538 13871 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:00.956342 20442 consensus_meta.cc:287] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.762s	user 0.013s	sys 0.001s
W20260709 13:07:00.971222 13872 rpcz_store.cc:269] 0709 13:06:58.858766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.859647 (+   881us) service_pool.cc:224] Handling call
0709 13:07:00.858364 (+1998717us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b54000ea954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:00.905478 (+ 47114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2136228736}
W20260709 13:07:00.994177 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 510737us
I20260709 13:07:00.998848 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2975) took 2980 ms. Trace:
I20260709 13:07:01.006430 13874 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:00.986987 19983 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.018648 13871 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:00.979179 20472 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:01.004184 13946 rpcz_store.cc:276] 0709 13:06:58.007523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.378999 (+371476us) service_pool.cc:224] Handling call
0709 13:06:58.379314 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:59.310112 (+930798us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:59.310170 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:59.311355 (+  1185us) log.cc:844] Serialized 26 byte log entry
0709 13:06:59.374023 (+ 62668us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:59.374102 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.374301 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:00.737443 (+1363142us) spinlock_profiling.cc:243] Waited 49.9 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:00.960467 (+223024us) raft_consensus.cc:1685] finished
0709 13:07:00.960493 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:00.988108 (+ 27615us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74188,"raft.queue_time_us":142092,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":52,"spinlock_wait_cycles":114818816,"thread_start_us":917657,"threads_started":2,"wal-append.queue_time_us":156913}
I20260709 13:07:01.031476 20490 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:00.986675 20481 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:01.046777 20543 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:07:01.053081 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.054157 20557 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:01.056914 20557 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.057255 20557 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:01.058637 20550 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:01.061084 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 2982) took 1539 ms. Trace:
I20260709 13:07:01.062398 20436 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:01.062676 20498 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.063436 20498 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:01.061651 13948 rpcz_store.cc:276] 0709 13:06:59.521348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.056498 (+535150us) service_pool.cc:224] Handling call
0709 13:07:00.056924 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.057352 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.057389 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.058018 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 13:07:00.286281 (+228263us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:00.286366 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.286472 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:00.792952 (+506480us) spinlock_profiling.cc:243] Waited 64.7 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:01.060622 (+267670us) raft_consensus.cc:1685] finished
0709 13:07:01.060664 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.060978 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148910976,"thread_start_us":227766,"threads_started":1,"wal-append.queue_time_us":329891}
W20260709 13:07:01.071775 13915 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.593s (SENT)
I20260709 13:07:01.072566 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:01.029016 20492 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:01.081794 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 407721us
I20260709 13:07:01.025264 19983 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:00.930344 13768 catalog_manager.cc:5697] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 9 to 10, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 10 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:01.091316 14553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 1044) took 1093 ms. Trace:
W20260709 13:07:01.098990 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.099436 19782 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.001s
I20260709 13:07:01.107388 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:01.108402 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.113071 13868 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:01.116353 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3361) took 2529 ms. Trace:
I20260709 13:07:01.116863 13868 rpcz_store.cc:276] 0709 13:06:58.755436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.860213 (+1104777us) service_pool.cc:224] Handling call
0709 13:07:01.109224 (+1249011us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b5800003698. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:07:01.109496 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.113607 (+  4111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.116262 (+  2655us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3393408}
I20260709 13:07:01.110708 19782 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:01.122726 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.086378 20498 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:01.103308 14553 rpcz_store.cc:276] 0709 13:06:59.997920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.998071 (+   151us) service_pool.cc:224] Handling call
0709 13:06:59.998445 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:59.998857 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:59.998888 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:59.999539 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 13:07:00.279518 (+279979us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:00.279590 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.279700 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:01.090935 (+811235us) raft_consensus.cc:1685] finished
0709 13:07:01.090973 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.091218 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1344,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":24853,"thread_start_us":279449,"threads_started":1,"wal-append.queue_time_us":714135}
W20260709 13:07:01.129452 13947 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.018s	sys 0.029s
I20260709 13:07:01.130827 13869 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:07:01.132762 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.133798 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 420078us
W20260709 13:07:01.134333 13947 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.018s	sys 0.029s
I20260709 13:07:01.111446 20492 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:01.145424 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:01.162700 13838 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:01.164402 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 96122.7us
W20260709 13:07:01.173218 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 284098us
I20260709 13:07:01.115276 20472 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:01.166821 20500 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:01.142021 20359 consensus_queue.cc:1059] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:07:01.176134 13944 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:07:01.173242 20091 consensus_queue.cc:1059] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:07:01.182994 20474 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:01.171098 20498 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.173557 20158 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:01.189461 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1134) took 1678 ms. Trace:
I20260709 13:07:01.191073 13944 rpcz_store.cc:276] 0709 13:06:59.511121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.511352 (+   231us) service_pool.cc:224] Handling call
0709 13:06:59.511788 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:00.450407 (+938619us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b54004603a0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33aadc0 00007fb2c34351c0 00007fb2c3436231 00007fb2c3432d8f 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c
0709 13:07:01.187571 (+737164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.189376 (+  1805us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1735553920}
W20260709 13:07:01.188446 20469 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.935s	user 0.008s	sys 0.007s
W20260709 13:07:01.205824 20407 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
W20260709 13:07:01.216331 20553 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:07:01.187904 20492 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.217849 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2689) took 1155 ms. Trace:
I20260709 13:07:01.171415 20564 consensus_queue.cc:1059] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:07:01.199786 20472 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:01.238675 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.222615 14024 rpcz_store.cc:276] 0709 13:07:00.062475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.067688 (+  5213us) service_pool.cc:224] Handling call
0709 13:07:00.068164 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.069041 (+   877us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.069075 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.069671 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 13:07:01.014292 (+944621us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:01.014339 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.014419 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:01.217510 (+203091us) raft_consensus.cc:1685] finished
0709 13:07:01.217545 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.217732 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":158,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":943722,"threads_started":1,"wal-append.queue_time_us":999780}
I20260709 13:07:01.239832 14023 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:01.242291 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.243199 14018 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:01.251009 14016 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: []
W20260709 13:07:01.253213 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.259618 14015 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:07:01.226584 20158 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:07:01.257709 19975 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.025s	sys 0.046s
I20260709 13:07:01.220820 20498 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:01.277073 19975 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.026s	sys 0.047s
I20260709 13:07:01.218673 20560 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.067s
I20260709 13:07:01.263871 20461 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:01.232555 20492 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:01.268956 19782 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:01.294761 20557 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:01.201901 13768 catalog_manager.cc:5697] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:01.308087 20542 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.372s	user 0.000s	sys 0.001s
I20260709 13:07:01.288470 14015 pending_rounds.cc:85] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 9
W20260709 13:07:01.296326 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.287470 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:01.289697 20461 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:01.314316 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1141) took 1044 ms. Trace:
I20260709 13:07:01.319036 13947 rpcz_store.cc:276] 0709 13:07:00.269507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.464135 (+194628us) service_pool.cc:224] Handling call
0709 13:07:00.489427 (+ 25292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:00.489733 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:07:01.135243 (+645510us) log.cc:844] Serialized 14 byte log entry
0709 13:07:01.313888 (+178645us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:01.313947 (+    59us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:07:01.314013 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.314101 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.314239 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1355,"prepare.queue_time_us":172830,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":53,"raft.queue_time_us":14956,"thread_start_us":769876,"threads_started":2}
W20260709 13:07:01.299295 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 108729us
I20260709 13:07:01.301831 19983 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:01.300462 20566 raft_consensus.cc:3055] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:01.315246 14015 pending_rounds.cc:107] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:07:01.330943 14015 raft_consensus.cc:2889] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:01.332751 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.333786 14016 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:01.326640 20324 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:01.344653 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 65839.7us
I20260709 13:07:01.348667 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.349666 14016 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:01.351105 20481 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:01.352499 20530 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.925s	user 0.004s	sys 0.000s
I20260709 13:07:01.353307 20469 raft_consensus.cc:3055] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:01.358882 13944 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:01.360529 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:01.362565 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.363178 14016 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:01.363395 14023 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
W20260709 13:07:01.342082 20494 consensus_meta.cc:287] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.004s
I20260709 13:07:01.345903 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:01.353778 20469 raft_consensus.cc:740] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:01.369894 20469 consensus_queue.cc:260] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:01.375308 14024 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:01.378301 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.379071 14023 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:01.378886 20461 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:01.379431 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:01.379338 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.380637 14024 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:01.380231 14022 raft_consensus.cc:2393] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:01.380185 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.383162 14016 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:01.385257 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:01.386885 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.342906 20158 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:01.386035 14016 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:01.389001 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:01.389737 13869 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
W20260709 13:07:01.390316 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 49993us
I20260709 13:07:01.392109 20494 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:01.392714 20494 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:01.393522 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:01.395797 14023 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:07:01.395938 20569 log.cc:927] Time spent T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:07:01.396242 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:01.395608 20494 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:01.397183 20428 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:01.398365 20428 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:01.398727 20428 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:01.399444 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.399813 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.400316 14021 raft_consensus.cc:2410] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:07:01.400507 14017 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:07:01.398877 14024 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:07:01.401150 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.401337 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.401451 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.402077 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:01.402411 20428 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:01.403826 20428 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:01.418120 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:01.404084 13866 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:01.404140 20500 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:01.404491 14017 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:01.333698 20566 raft_consensus.cc:740] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:01.401197 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:01.424612 14021 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:01.406342 13874 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:01.393620 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:01.424995 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:01.407111 20337 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:01.425740 13871 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:01.407150 20309 consensus_meta.cc:287] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.729s	user 0.008s	sys 0.007s
I20260709 13:07:01.379096 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:01.426653 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:01.427189 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:01.426671 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:01.428624 14024 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:01.401893 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.411489 13875 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
W20260709 13:07:01.429461 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.431388 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:01.405093 20324 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:01.420912 14022 raft_consensus.cc:1275] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:01.418920 14556 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:07:01.426532 14018 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:01.438059 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:01.458753 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:01.459630 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.459550 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:01.460330 13944 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:01.380334 20461 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:01.461009 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:07:01.463346 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 69844.1us
I20260709 13:07:01.457652 13948 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:07:01.465234 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:01.468861 14017 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:01.470278 13869 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:01.474339 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:01.480984 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:01.482570 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:01.475603 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 930) took 2068 ms. Trace:
I20260709 13:07:01.486526 13946 raft_consensus.cc:2393] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 24: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:07:01.487793 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.482002 14555 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 24.
W20260709 13:07:01.491899 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 146076us
I20260709 13:07:01.422353 20500 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:01.500803 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:01.502640 14553 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:01.503784 20570 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
W20260709 13:07:01.506343 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.427637 20228 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:01.506412 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:01.426777 20337 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:01.508388 13942 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:01.509959 14016 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:01.507855 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:01.491017 13840 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.063s (SENT)
I20260709 13:07:01.509505 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:01.516532 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.484477 13870 rpcz_store.cc:276] 0709 13:06:59.407500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.871714 (+464214us) service_pool.cc:224] Handling call
0709 13:07:00.556162 (+684448us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b5800003698. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:07:00.566351 (+ 10189us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.822283 (+255932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.822357 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.908382 (+ 86025us) log.cc:844] Serialized 26 byte log entry
0709 13:07:01.176103 (+267721us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:01.176155 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.475168 (+299013us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54003ac0d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:01.475265 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:01.475335 (+    70us) raft_consensus.cc:1685] finished
0709 13:07:01.475356 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.475520 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7983,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":33698,"spinlock_wait_cycles":545307136,"thread_start_us":242043,"threads_started":1,"wal-append.queue_time_us":351637}
I20260709 13:07:01.482154 19983 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:01.524729 13875 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:07:01.520278 13874 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:01.527634 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:01.530694 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:01.531381 13866 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:01.535580 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:01.518368 14018 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:01.536253 13945 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:01.544773 13871 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:01.537038 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:01.550688 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.553745 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:01.437822 20309 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:01.552814 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:01.559976 13946 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:01.563782 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:01.530383 19983 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.530577 20557 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:01.535501 20494 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:01.572037 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:01.577801 13944 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:01.557109 20442 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:01.555810 20492 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:01.518651 20228 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:01.591133 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:01.593899 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 97987us
W20260709 13:07:01.595561 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 118463us
I20260709 13:07:01.578359 19983 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:01.607959 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.609776 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:01.622113 13945 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:07:01.579679 20494 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.625427 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:01.549130 20407 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:07:01.628667 13944 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:01.582788 20436 raft_consensus.cc:2764] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:07:01.631179 13945 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:07:01.596067 20442 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.594606 20498 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:01.651634 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:01.594911 20584 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:01.660825 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:01.665395 13987 leader_election.cc:400] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17.
I20260709 13:07:01.665947 13987 leader_election.cc:403] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:07:01.667487 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:07:01.673363 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 44384.4us
I20260709 13:07:01.637742 20494 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:01.557403 20566 consensus_queue.cc:260] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:01.682577 13944 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:01.684698 13945 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:01.653441 20442 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:01.631070 20180 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:01.688709 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 119650us
I20260709 13:07:01.726392 13989 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:01.612726 20500 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:01.596947 20337 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:01.731465 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:01.739861 13866 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:01.727813 20492 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.689531 20585 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17.
W20260709 13:07:01.783025 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 56826.2us
I20260709 13:07:01.770857 13840 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:01.737838 20578 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:01.756484 20180 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:07:01.798635 14024 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:01.746379 20584 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:01.803580 14021 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:01.806455 13942 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:01.804188 13988 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:01.805106 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:01.814114 20407 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.813690 20492 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:01.818806 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:01.828568 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.747741 20472 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:07:01.809397 20337 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:01.846312 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 150828us
W20260709 13:07:01.849911 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 182557us
W20260709 13:07:01.850479 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.850170 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:01.851337 13942 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:01.868104 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:01.831935 20091 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:01.847899 20474 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:01.871304 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:01.897657 13941 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:01.890533 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 161230us
I20260709 13:07:01.856740 20481 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:01.868017 20584 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:01.957111 20584 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.957005 20481 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:01.874470 20309 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:01.924629 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.904491 13987 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:01.899855 20474 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:01.902648 20578 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:07:01.916236 20575 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.361s	user 0.002s	sys 0.000s
I20260709 13:07:01.887574 20598 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:01.955544 20597 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:01.989671 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.902218 20589 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:01.992772 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 88388.3us
I20260709 13:07:01.993427 20585 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:07:01.995096 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.904368 20594 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:07:01.904002 20583 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:07:01.957665 20584 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:01.903254 20587 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 13:07:01.903777 20582 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
W20260709 13:07:01.998484 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 179742us
I20260709 13:07:02.012941 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:02.092587 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 79778.2us
I20260709 13:07:02.032243 20485 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.867240 20407 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:02.024267 20324 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:01.995975 20474 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:02.025290 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:02.029073 20469 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:02.121474 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 96274.3us
I20260709 13:07:02.122342 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:02.123443 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:02.026104 20580 consensus_queue.cc:1059] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:07:02.024859 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 284821us
W20260709 13:07:02.028178 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.125466 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.024737 14555 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:02.032994 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:01.995326 20578 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.097358 20485 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:02.099781 20436 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:02.096658 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:02.108067 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.109855 14023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.009s	sys 0.028s
W20260709 13:07:02.131976 14023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.010s	sys 0.028s
I20260709 13:07:02.019852 20359 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:02.125988 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.128198 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 324105us
I20260709 13:07:02.141582 13838 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:02.146816 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 50359.7us
I20260709 13:07:02.149531 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:02.089035 20585 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:02.116173 20309 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:02.158706 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:02.164846 13874 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:02.155258 13942 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
W20260709 13:07:02.130234 20593 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:07:02.137553 20578 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:02.136742 20598 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:02.160181 20587 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:07:02.082607 20091 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:02.191794 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:02.082666 20589 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:02.205994 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.156075 20560 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:02.152408 20565 consensus_queue.cc:1059] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.068s
W20260709 13:07:02.248625 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.250520 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 101064us
I20260709 13:07:02.152487 20359 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:02.235657 20472 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:02.264017 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.264695 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.265446 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 73788.9us
I20260709 13:07:02.249672 20598 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:07:02.268447 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 145051us
I20260709 13:07:02.245687 20602 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:02.289067 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:07:02.293751 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.294911 14553 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:02.289201 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:02.226985 20091 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:02.251660 20600 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:02.301326 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 163052us
W20260709 13:07:02.300649 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 311470us
I20260709 13:07:02.253206 20606 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:02.264640 20492 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:02.288795 20598 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.266563 20472 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:02.329816 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.267277 20560 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:02.320438 13987 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:07:02.376111 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 55800.6us
I20260709 13:07:02.379019 13987 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:07:02.381012 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 63732.8us
I20260709 13:07:02.294166 20597 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:02.252569 20610 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:07:02.381966 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:02.303830 20609 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:02.410542 20609 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.411046 20609 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.413192 20609 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:02.413720 20609 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:02.388600 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:02.420634 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:02.423508 20359 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:02.391022 20607 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.088s
W20260709 13:07:02.425696 20612 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:07:02.392899 20589 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:02.394960 20603 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:07:02.395080 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:02.427160 14553 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:02.336289 20600 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:02.395884 20614 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:02.303925 20585 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.375964 20598 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:02.397992 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:02.437170 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:02.398275 20436 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:02.398663 20586 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:07:02.398880 20492 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:07:02.399222 20613 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
W20260709 13:07:02.406737 19983 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.763s	user 0.014s	sys 0.010s
I20260709 13:07:02.438972 13914 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:02.374927 20472 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:02.454104 13941 consensus_meta.cc:287] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.537s	user 0.012s	sys 0.012s
I20260709 13:07:02.454574 13941 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
W20260709 13:07:02.460326 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 204868us
I20260709 13:07:02.442417 20436 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.447623 20589 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:02.449863 20614 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:02.304627 20602 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:02.473133 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.448374 20600 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:02.476958 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:02.448115 20609 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:02.478260 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:02.477447 20607 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.485189 20621 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:02.449832 20580 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:07:02.486267 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:02.487298 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:02.522946 13872 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:02.469381 20481 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:02.534071 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:02.535264 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:02.455253 19983 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:02.476555 20599 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.382s	user 0.002s	sys 0.000s
W20260709 13:07:02.541661 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 275244us
I20260709 13:07:02.488662 20566 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:02.598877 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.473759 20436 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:02.480880 20619 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:02.471663 20565 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:02.576184 13874 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:07:02.603083 13873 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:02.584350 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 416687us
W20260709 13:07:02.611483 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.488008 20589 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:02.614028 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:02.616425 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 237529us
I20260709 13:07:02.547065 20564 raft_consensus.cc:3055] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:07:02.586905 20461 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.109s	user 0.018s	sys 0.007s
I20260709 13:07:02.523460 20407 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:02.550159 20180 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:02.629294 13875 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:02.583220 20606 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:02.541975 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:02.630295 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:02.630577 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:02.630625 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:02.633036 14016 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:07:02.634037 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:02.635843 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:02.636276 20428 consensus_meta.cc:287] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.214s	user 0.015s	sys 0.009s
I20260709 13:07:02.547551 20629 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:02.538736 20600 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.649847 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:02.546226 20624 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:02.546478 20632 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:02.534137 20607 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:02.652920 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.654793 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:02.636641 20620 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:02.663035 14017 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:02.662073 14560 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 24.
I20260709 13:07:02.656239 14524 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:02.667698 14021 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:02.647826 20564 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:07:02.672861 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.627195 20565 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:02.680610 14022 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:02.651155 14556 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 22.
I20260709 13:07:02.681389 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:02.688058 14019 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 21: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:02.655335 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 168140us
I20260709 13:07:02.693220 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:02.699437 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:02.649544 20180 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:02.653095 20359 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:02.745314 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:02.578011 20597 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.481482 20560 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:02.721012 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:02.633913 20474 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:02.636950 20584 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:02.776047 14017 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:07:02.685261 20629 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:02.780181 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:02.700480 20609 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.683813 20614 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:02.705696 20616 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.699139 20620 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.786513 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:02.688905 20600 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.768607 13914 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:02.756183 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:02.816704 14021 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 23: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:02.704715 20309 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:02.787966 13867 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:07:02.736008 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.796247 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 165750us
I20260709 13:07:02.796594 14017 raft_consensus.cc:1275] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:07:02.759265 20494 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.032s	user 0.017s	sys 0.010s
I20260709 13:07:02.785830 14018 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
W20260709 13:07:02.804510 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 311959us
W20260709 13:07:02.790187 20583 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:07:02.776113 20630 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:02.699461 20622 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:02.813642 13875 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 24.
I20260709 13:07:02.816126 14016 raft_consensus.cc:2368] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14.
I20260709 13:07:02.764449 20091 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:02.816569 14015 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:02.701882 20621 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:02.823400 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:02.825403 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 56928.3us
W20260709 13:07:02.826670 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.837284 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.805361 20407 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:02.684876 20624 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:02.841882 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:02.856791 14021 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:02.861789 20618 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:02.863996 19983 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:02.630188 13860 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 with delete_type TABLET_DATA_TOMBSTONED (TS e1649107c781428692458f10597c5742 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:46040
I20260709 13:07:02.815729 20620 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:02.870386 14560 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: []
W20260709 13:07:02.873317 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 45186.4us
I20260709 13:07:02.830895 20494 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.858521 20461 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.903628 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:02.820557 20606 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.901222 14022 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:07:02.901891 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:02.815541 20616 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.814865 20589 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:02.895661 20337 consensus_meta.cc:287] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.034s	user 0.020s	sys 0.004s
I20260709 13:07:02.892156 19983 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:02.919945 13950 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:02.860634 20091 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:07:02.867659 20407 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.914621 19975 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 13:07:02.938637 14553 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:02.941794 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:02.932832 20645 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.858503 20180 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.851991 20614 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.920159 20428 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:02.957688 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 66948.6us
I20260709 13:07:02.852805 20565 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.906453 20158 consensus_queue.cc:1059] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:07:02.965378 20622 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:02.965898 20640 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:07:02.966337 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.966500 20640 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.966885 20640 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:02.967268 20607 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:02.971783 20469 consensus_meta.cc:287] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.832s	user 0.009s	sys 0.004s
I20260709 13:07:02.960542 20636 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.976049 20636 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:02.978494 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 62526.1us
I20260709 13:07:02.978904 20630 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.979477 20630 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.981266 20584 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:02.982621 20474 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:02.963611 20621 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:02.983382 20474 raft_consensus.cc:3055] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:02.983762 20474 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:02.984755 20474 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:02.987973 20228 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:02.992672 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3237) took 1595 ms. Trace:
I20260709 13:07:02.998287 20500 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:03.002534 20588 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:03.003033 20629 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:02.982623 20158 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:03.007547 20158 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:02.927014 20634 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:02.981752 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:02.993172 14023 rpcz_store.cc:276] 0709 13:07:01.396905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.397010 (+   105us) service_pool.cc:224] Handling call
0709 13:07:01.397300 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.397750 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:01.519237 (+121487us) log.cc:844] Serialized 14 byte log entry
0709 13:07:02.992102 (+1472865us) spinlock_profiling.cc:243] Waited 7.32 ms on lock 0x7b18004760e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287
0709 13:07:02.992224 (+   122us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:02.992300 (+    76us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:02.992366 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.992456 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.992601 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21591,"prepare.queue_time_us":181608,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":242120,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":7683,"spinlock_wait_cycles":16840064,"thread_start_us":821116,"threads_started":3,"wal-append.queue_time_us":732501}
I20260709 13:07:03.002614 20324 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:02.947078 20633 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:02.992748 20442 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.253s	user 0.010s	sys 0.013s
I20260709 13:07:03.009312 20637 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:02.928367 20461 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:03.008682 20158 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.003093 20588 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:03.020444 20324 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:03.054243 20309 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:03.030236 20442 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.064924 20324 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:03.065157 20309 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:03.066452 20324 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:03.066885 20442 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:03.070982 20485 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:03.078426 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.079720 20600 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:03.082484 14522 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:03.087548 20337 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:03.088160 20337 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:02.961576 20645 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:03.025916 20474 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:03.082691 20636 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:03.100860 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.110545 14023 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:03.104974 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.111653 14016 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 23.
I20260709 13:07:03.101897 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.103745 14017 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:03.092103 14019 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:07:03.090967 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3214) took 1853 ms. Trace:
I20260709 13:07:03.114171 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:03.115593 14021 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:03.148826 13941 raft_consensus.cc:2393] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:03.148818 14022 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:03.150028 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.150491 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:03.073750 20461 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:03.094935 20472 consensus_meta.cc:287] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.615s	user 0.012s	sys 0.000s
I20260709 13:07:03.027992 20584 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:03.101544 20498 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:03.159473 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:03.167052 14022 raft_consensus.cc:2376] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 14. Re-sending same reply.
I20260709 13:07:03.093297 20645 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:03.044821 20629 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:03.164685 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:03.174592 13949 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:07:03.095804 20632 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.095705 20616 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:03.170679 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 188991us
I20260709 13:07:03.113240 14020 rpcz_store.cc:276] 0709 13:07:01.237676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.249755 (+ 12079us) service_pool.cc:224] Handling call
0709 13:07:01.271222 (+ 21467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.286521 (+ 15299us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:07:01.420372 (+133851us) log.cc:844] Serialized 14 byte log entry
0709 13:07:02.469083 (+1048711us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b180023da00. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd323a
0709 13:07:02.665190 (+196107us) log.cc:844] Serialized 14 byte log entry
0709 13:07:03.043603 (+378413us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b180023da00. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:07:03.090446 (+ 46843us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b180023da00. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287
0709 13:07:03.090565 (+   119us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:03.090617 (+    52us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:03.090675 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.090761 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.090898 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28605,"prepare.queue_time_us":456154,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":4099,"raft.queue_time_us":449992,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":14408,"spinlock_wait_cycles":66304896,"thread_start_us":482599,"threads_started":5,"wal-append.queue_time_us":160385}
I20260709 13:07:03.180179 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:03.025398 20633 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:03.186510 20588 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:03.189456 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.169763 20619 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:03.162838 19983 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:03.190186 20629 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:03.198722 20649 log.cc:927] Time spent T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
W20260709 13:07:03.199257 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 50504.8us
W20260709 13:07:03.202504 20627 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.258s	user 0.001s	sys 0.001s
I20260709 13:07:03.203860 20655 tablet_replica.cc:333] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:03.207173 14022 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:03.032043 20428 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.125s
I20260709 13:07:03.228176 13949 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:07:03.089121 20337 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:03.212878 20578 consensus_meta.cc:287] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.967s	user 0.003s	sys 0.012s
I20260709 13:07:03.243605 13941 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:03.246893 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:03.248271 14020 raft_consensus.cc:2376] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 19. Re-sending same reply.
I20260709 13:07:03.249634 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:03.248500 20598 consensus_meta.cc:287] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.804s	user 0.009s	sys 0.007s
I20260709 13:07:03.254369 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:03.255152 14560 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:03.254710 20598 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.257964 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:03.258826 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:03.258412 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:03.168735 20472 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.261209 14522 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:03.266757 13839 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:03.267831 20661 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:03.268298 20661 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:03.269616 13988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.006s	sys 0.041s
W20260709 13:07:03.276240 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:03.271906 13988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.006s	sys 0.041s
I20260709 13:07:03.282655 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:03.283661 14555 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:03.283859 13941 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:07:03.272097 14524 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.152s (SENT)
I20260709 13:07:03.257759 20598 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:03.271121 20657 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:03.290349 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:03.266810 20588 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:03.266099 20485 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.349112 20485 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:03.299649 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:03.287216 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:03.218930 20632 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.299338 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:03.264516 20494 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:03.302855 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.361883 14015 raft_consensus.cc:2393] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:03.312800 14022 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:03.241365 20578 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.321996 13989 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:03.323022 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:03.326397 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:03.328696 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 56633.7us
W20260709 13:07:03.392587 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 117484us
I20260709 13:07:03.393106 14021 raft_consensus.cc:2376] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 16. Re-sending same reply.
W20260709 13:07:03.393337 14524 leader_election.cc:336] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.176s (SENT)
I20260709 13:07:03.330461 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.394145 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.394276 14024 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:03.394801 14021 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:03.331235 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:03.395766 14016 raft_consensus.cc:2368] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 22. Current term is 23.
I20260709 13:07:03.388002 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:03.395920 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.397338 14016 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:07:03.397811 14016 pending_rounds.cc:85] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 9
I20260709 13:07:03.331992 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.398535 14023 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:03.332877 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.399749 14018 raft_consensus.cc:2376] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 14. Re-sending same reply.
I20260709 13:07:03.400888 14016 pending_rounds.cc:107] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:07:03.401607 14016 raft_consensus.cc:2889] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:03.346235 20619 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.347865 20584 consensus_queue.cc:1059] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:03.273845 20661 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:03.245708 20620 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:03.317690 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:03.386984 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.399616 13947 raft_consensus.cc:2410] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:07:03.405228 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:03.410787 14020 raft_consensus.cc:2376] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 16. Re-sending same reply.
I20260709 13:07:03.320315 20657 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.299266 20651 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 13:07:03.433601 14021 raft_consensus.cc:2393] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 17: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:03.409548 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:03.411242 14022 raft_consensus.cc:2368] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19.
I20260709 13:07:03.440603 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.413045 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.425848 20645 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.423146 14015 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:07:03.457373 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 64078.8us
I20260709 13:07:03.423714 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.423887 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:03.356655 20407 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:03.436869 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.299494 20359 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.107s
I20260709 13:07:03.458590 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:03.458830 14018 raft_consensus.cc:2376] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 17. Re-sending same reply.
I20260709 13:07:03.411132 13945 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:03.414361 20598 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:03.415479 20602 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:03.408077 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 504513us
I20260709 13:07:03.445387 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:03.423995 20500 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.457453 14016 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:03.436868 20588 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:03.428900 20657 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:03.461863 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 139971us
I20260709 13:07:03.463759 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:03.458822 14020 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
W20260709 13:07:03.462144 20653 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.412s	user 0.000s	sys 0.002s
I20260709 13:07:03.469300 14552 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 24.
I20260709 13:07:03.410926 20091 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:07:03.480393 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:03.489066 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:03.490511 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:03.492537 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.494275 14015 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:03.465700 20461 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:03.503949 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:03.475906 20609 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.001s	sys 0.016s
W20260709 13:07:03.505942 20609 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.001s	sys 0.017s
I20260709 13:07:03.505708 20448 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:03.507405 14024 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:03.508623 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:03.509598 14020 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 21: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:03.509517 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:03.513424 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:03.513582 13915 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:03.514936 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:03.515233 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:03.483590 20485 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:03.485020 20602 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:03.490936 20407 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:03.522164 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:03.523823 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:03.528740 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:03.472391 20645 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.482538 20500 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.528535 20589 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:03.586309 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62607.6us
I20260709 13:07:03.493469 20494 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.090s
W20260709 13:07:03.518956 20650 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.346s	user 0.000s	sys 0.002s
I20260709 13:07:03.522042 20606 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:03.490934 20560 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:07:03.590137 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.642022 14015 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
W20260709 13:07:03.642088 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 136865us
W20260709 13:07:03.645116 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 116440us
I20260709 13:07:03.584987 20472 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:03.535006 20359 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:03.655479 20500 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:03.641477 20606 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:03.656337 20606 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:07:03.661767 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 139781us
I20260709 13:07:03.664273 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:03.663259 20359 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:03.666013 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3399) took 1033 ms. Trace:
W20260709 13:07:03.666842 20665 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
W20260709 13:07:03.622852 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 114868us
I20260709 13:07:03.521462 20609 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:03.635644 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:03.540777 20607 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:03.671360 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:03.521853 20669 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:03.647823 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:03.522392 20309 consensus_queue.cc:1048] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.112s
I20260709 13:07:03.519467 20600 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:03.649735 13869 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:07:03.626164 20623 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.009s	sys 0.020s
W20260709 13:07:03.707877 20623 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.009s	sys 0.021s
I20260709 13:07:03.666522 13874 rpcz_store.cc:276] 0709 13:07:02.634246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.635139 (+   893us) service_pool.cc:224] Handling call
0709 13:07:02.635511 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:02.653584 (+ 18073us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:02.653626 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:02.665524 (+ 11898us) log.cc:844] Serialized 26 byte log entry
0709 13:07:02.859347 (+193823us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:02.859398 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.859475 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:03.665727 (+806252us) raft_consensus.cc:1685] finished
0709 13:07:03.665755 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.665935 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30715,"raft.queue_time_us":15184,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":11615,"thread_start_us":92017,"threads_started":1,"wal-append.queue_time_us":338104}
I20260709 13:07:03.712082 20645 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:03.541738 20428 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:03.718200 20428 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:03.718449 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:03.643581 20640 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.676s	user 0.012s	sys 0.002s
I20260709 13:07:03.718711 20428 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:03.719209 20640 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:03.720404 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:03.721016 13945 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:03.721514 20640 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 25 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:03.540243 20474 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 13:07:03.722704 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 782) took 1087 ms. Trace:
I20260709 13:07:03.723327 13944 raft_consensus.cc:2368] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 24. Current term is 25.
I20260709 13:07:03.675083 13989 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:03.679188 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:03.724071 20647 raft_consensus.cc:3055] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:03.724306 14552 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:03.724516 20647 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:03.724815 20645 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:03.541083 19983 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.108s
I20260709 13:07:03.687345 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:03.646971 20158 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:03.726650 14561 raft_consensus.cc:2410] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:07:03.726760 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:03.728353 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.729563 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:03.691435 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 56498.6us
I20260709 13:07:03.693745 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:03.737007 20620 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:03.738631 14018 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:07:03.738180 20564 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.739667 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.695317 20442 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:03.638182 20602 consensus_queue.cc:1048] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:07:03.705008 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:03.718331 20682 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:03.723133 14019 rpcz_store.cc:276] 0709 13:07:02.635794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.786182 (+150388us) service_pool.cc:224] Handling call
0709 13:07:03.722480 (+936298us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b540023a354. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:03.722600 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641883392}
I20260709 13:07:03.743947 20588 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:03.744830 20588 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:03.745482 14018 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:03.747642 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:03.750660 14018 raft_consensus.cc:2368] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14.
I20260709 13:07:03.744084 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:03.682245 20621 raft_consensus.cc:2764] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:07:03.752516 14019 raft_consensus.cc:2376] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16. Re-sending same reply.
W20260709 13:07:03.753219 20436 consensus_meta.cc:287] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.105s	user 0.014s	sys 0.001s
I20260709 13:07:03.753542 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.753676 20436 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.666311 20359 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:03.757182 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:03.758023 14555 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:03.758500 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:03.759229 14559 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:03.760115 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 85173.9us
I20260709 13:07:03.740175 20564 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:03.760936 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:03.762557 14553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 1214) took 1298 ms. Trace:
I20260709 13:07:03.762946 14553 rpcz_store.cc:276] 0709 13:07:02.464572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.465114 (+   542us) service_pool.cc:224] Handling call
0709 13:07:03.762389 (+1297275us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b5400584b54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:03.762497 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1037614464}
I20260709 13:07:03.763425 20602 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:03.764003 20602 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:03.764765 13946 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:07:03.765789 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:03.766369 13946 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:03.766427 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:03.768114 20632 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:03.769440 20633 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:03.770550 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3400) took 1117 ms. Trace:
I20260709 13:07:03.770983 20602 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:03.770927 13875 rpcz_store.cc:276] 0709 13:07:02.653380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.664695 (+ 11315us) service_pool.cc:224] Handling call
0709 13:07:03.770381 (+1105686us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54003ab6d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:03.770487 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":243836288}
I20260709 13:07:03.774158 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:03.774287 20436 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:03.774961 13948 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:03.775452 20560 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:03.638466 20659 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:03.778648 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:03.779400 13943 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:07:03.689329 20669 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:03.780920 20669 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:03.780387 13946 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
W20260709 13:07:03.782568 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 48721us
I20260709 13:07:03.739805 20578 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:03.791628 13914 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:03.783399 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:07:03.751904 20658 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 13:07:03.691628 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:03.784832 13946 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:03.786266 14552 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:07:03.797535 14015 raft_consensus.cc:2376] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17. Re-sending same reply.
W20260709 13:07:03.799582 13950 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3262) took 2221 ms (client timeout 1733 ms). Trace:
W20260709 13:07:03.799983 13950 rpcz_store.cc:269] 0709 13:07:01.563731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.572654 (+  8923us) service_pool.cc:224] Handling call
0709 13:07:03.757905 (+2185251us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400092754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:03.758045 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3039073024}
I20260709 13:07:03.789686 14555 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 25.
W20260709 13:07:03.804064 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 99131.2us
W20260709 13:07:03.805483 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.812772 13914 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:07:03.814738 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 168677us
I20260709 13:07:03.810271 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:03.816776 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:03.819720 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 58874.6us
I20260709 13:07:03.820538 13989 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:03.821580 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:03.823030 13989 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:03.825042 20428 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.795490 19782 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:03.833825 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:03.816419 20091 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:03.803805 20158 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:03.808143 20659 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:03.819434 19983 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:03.819355 20620 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:03.725502 20647 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:03.827116 20640 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:03.818920 20500 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.830818 20645 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:03.849186 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2701) took 1125 ms. Trace:
W20260709 13:07:03.874046 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:03.888190 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 43891.5us
I20260709 13:07:03.893803 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:03.896656 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3403) took 1196 ms. Trace:
I20260709 13:07:03.897100 13866 rpcz_store.cc:276] 0709 13:07:02.700578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.700717 (+   139us) service_pool.cc:224] Handling call
0709 13:07:02.763157 (+ 62440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:02.887059 (+123902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:02.887095 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:02.906855 (+ 19760us) log.cc:844] Serialized 26 byte log entry
0709 13:07:03.148614 (+241759us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:03.148654 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.148724 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:03.896376 (+747652us) raft_consensus.cc:1685] finished
0709 13:07:03.896401 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.896585 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13324,"raft.queue_time_us":16801,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":8146,"thread_start_us":242898,"threads_started":2,"wal-append.queue_time_us":525696}
I20260709 13:07:03.836519 20621 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:03.896726 14017 rpcz_store.cc:276] 0709 13:07:02.723555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.830090 (+106535us) service_pool.cc:224] Handling call
0709 13:07:02.830446 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:03.112714 (+282268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.848793 (+736079us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b540022d654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:03.849095 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":627462400}
I20260709 13:07:03.837144 20091 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:03.856218 20598 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:03.845737 19983 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:03.912437 19983 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:03.912781 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:03.914674 20600 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:03.915099 20600 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.915475 20600 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:03.919332 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71392.4us
I20260709 13:07:03.920341 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:03.923454 20647 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.907857 20598 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:03.926600 20598 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:03.926265 19983 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:03.910913 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 77175.3us
I20260709 13:07:03.849298 19782 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:03.930855 13989 leader_election.cc:400] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 18. Current term is 19.
I20260709 13:07:03.931227 13989 leader_election.cc:403] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:07:03.931273 19782 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.934369 20621 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 18. Current term is 19.
I20260709 13:07:03.935573 20442 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:03.911583 20618 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:03.937829 20607 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:03.938503 20618 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:03.938966 20618 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:03.940207 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:03.941356 20442 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:03.941771 20442 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.941810 19983 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:03.942276 20442 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:03.942755 20564 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:03.944195 20428 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:03.947168 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:03.948164 20359 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.921854 13840 leader_election.cc:400] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 22. Current term is 23.
I20260709 13:07:03.950912 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:03.896427 20620 raft_consensus.cc:697] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:03.893924 20645 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.906446 20091 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:03.902177 20640 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 25 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:03.962389 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:03.898685 20500 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:03.950394 20682 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:03.964861 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 231561us
W20260709 13:07:03.975868 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 282211us
I20260709 13:07:03.977485 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:03.985941 13945 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:03.990374 13943 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
W20260709 13:07:03.991747 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.999091 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.995893 14559 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:03.999868 14017 raft_consensus.cc:2376] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 16. Re-sending same reply.
I20260709 13:07:04.001780 13914 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:03.972124 20645 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:03.979179 20614 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:04.006872 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:03.977437 20578 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:03.978842 20621 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:04.015662 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:04.016451 14017 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:04.027189 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:04.028311 14015 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:04.039752 14017 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:04.041705 20702 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:04.028362 13915 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:07:04.056187 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 78726.3us
W20260709 13:07:04.075706 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 49040.7us
I20260709 13:07:04.076294 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:07:04.062216 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 140500us
I20260709 13:07:04.077435 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:07:04.042671 20657 consensus_meta.cc:287] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.549s	user 0.013s	sys 0.002s
I20260709 13:07:04.015808 20614 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:04.070011 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 94132.7us
I20260709 13:07:04.006093 20588 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.077512 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:04.083014 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:04.046262 20597 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:04.088251 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:04.089193 13945 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:04.089002 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:04.089936 13947 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:04.091372 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:04.090955 14524 leader_election.cc:400] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14.
I20260709 13:07:04.092046 14015 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:04.092059 14524 leader_election.cc:403] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:07:04.094013 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:03.995266 20158 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.099025 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:04.114758 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:04.081284 19983 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:04.117095 13950 raft_consensus.cc:2410] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:07:04.091533 20560 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:04.086292 20614 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:04.085228 20702 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:07:04.120013 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.120100 20621 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:04.124593 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:04.129220 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:04.092537 20632 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:04.133126 13946 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:04.113122 20659 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:04.113307 20609 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:04.095495 20655 raft_consensus.cc:2243] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:04.141165 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:04.098881 20607 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:04.146828 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:07:04.147148 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 69803.6us
I20260709 13:07:04.110522 20597 raft_consensus.cc:697] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:04.156701 14015 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:07:04.138981 13945 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:04.158763 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 72039.9us
I20260709 13:07:04.148036 13838 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:04.160565 13838 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:04.085938 20657 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.159207 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:04.120611 20619 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:07:04.135593 19782 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:04.164083 13915 leader_election.cc:336] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.666s (SENT)
I20260709 13:07:04.127375 20702 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:04.184264 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:04.148973 20659 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:04.160241 20606 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.000s
I20260709 13:07:04.183599 13946 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:04.140312 20632 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:04.190763 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:04.191393 13945 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:07:04.192571 14522 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:04.184942 14020 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 17
W20260709 13:07:04.174950 20690 log.cc:927] Time spent T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:07:04.186147 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:04.192687 13945 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:07:04.187093 20498 consensus_meta.cc:287] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.006s	user 0.013s	sys 0.003s
I20260709 13:07:04.194749 20588 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:04.188405 20560 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:04.196491 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:04.084147 20620 consensus_queue.cc:237] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.231971 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.198622 20657 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:04.158409 20500 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:04.226532 20698 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 13:07:04.210494 20632 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:04.237330 14015 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:04.211699 20655 raft_consensus.cc:2272] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:04.239063 20614 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.212265 20597 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.191867 20359 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:04.209760 14020 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:04.169589 20494 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:04.246924 20560 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.247288 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.139065 20669 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.248821 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:04.249454 14020 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:04.250839 20614 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:04.250960 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:04.253419 20700 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.254071 20700 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:04.255028 20661 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.821s	user 0.013s	sys 0.000s
I20260709 13:07:04.184432 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:04.274501 20661 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.223542 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:04.275888 13867 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:07:04.175347 20626 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:04.169166 20474 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:04.171813 20697 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 13. Current term is 14.
I20260709 13:07:04.228314 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:04.226521 20498 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:04.281062 20474 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:04.185629 20472 consensus_queue.cc:1059] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:07:04.189422 19782 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.282186 20472 raft_consensus.cc:3055] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:04.282658 20472 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:04.282194 20474 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.283296 19782 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:04.210625 20442 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.283602 20472 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.284369 14020 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:04.285001 20472 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:04.256455 20700 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:04.177801 19983 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:07:04.257299 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 102105us
I20260709 13:07:04.254448 20655 ts_tablet_manager.cc:1916] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:04.189688 20659 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:04.305783 14022 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
I20260709 13:07:04.259184 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.123764 20640 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.267093 14559 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 22.
I20260709 13:07:04.215540 20606 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:04.275007 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 110966us
I20260709 13:07:04.331022 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:04.278857 13945 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:04.278587 20626 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.285835 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.332540 14020 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:04.288784 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:04.285456 20442 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:04.302570 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:07:04.309646 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.312037 13914 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:04.221212 14553 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
W20260709 13:07:04.314417 14524 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.528s (SENT)
I20260709 13:07:04.333498 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:04.338971 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:04.343268 13915 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:04.349726 20693 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:04.335861 20630 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:04.370319 20565 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:04.371155 20651 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:04.371816 20651 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:04.372761 20651 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:04.371176 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 68738.1us
I20260709 13:07:04.375754 13840 leader_election.cc:400] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19.
I20260709 13:07:04.376061 13840 leader_election.cc:403] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:04.374676 20180 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:04.379473 20694 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.327s	user 0.003s	sys 0.003s
I20260709 13:07:04.381426 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:04.382301 20359 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:07:04.345505 20701 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:04.384298 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:04.348050 13914 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:04.388569 13947 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:04.388890 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:04.389734 13946 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
W20260709 13:07:04.399760 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 51849.2us
W20260709 13:07:04.400713 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.400791 13838 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:04.402220 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 59105.4us
I20260709 13:07:04.402977 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:04.397361 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.404722 14016 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
I20260709 13:07:04.412820 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:04.412897 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 3008) took 1105 ms. Trace:
I20260709 13:07:04.414819 13944 rpcz_store.cc:276] 0709 13:07:03.278978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.279649 (+   671us) service_pool.cc:224] Handling call
0709 13:07:04.361971 (+1082322us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54001c6454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:04.384123 (+ 22152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":835324288}
I20260709 13:07:04.387987 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:04.414016 13866 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:04.310339 19983 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:04.336551 20626 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:04.439417 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.440464 14021 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:04.341017 20588 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:04.447585 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:04.448570 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:04.449388 14015 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:04.384018 20494 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:04.452770 13838 leader_election.cc:400] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 24. Current term is 25.
I20260709 13:07:04.453231 13838 leader_election.cc:403] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:07:04.398025 20436 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:04.450173 13867 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:04.397893 20630 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.402070 20614 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:04.403589 20609 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:04.462541 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:04.431252 20442 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:07:04.466539 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 233409us
I20260709 13:07:04.431757 20428 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:04.455662 20618 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.002s
I20260709 13:07:04.469973 14019 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:04.466230 20584 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:04.462173 20647 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:04.462723 20500 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.465994 20621 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:04.473170 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.473544 20469 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.508487 13945 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
W20260709 13:07:04.504998 14524 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.805s (SENT) [suppressed 43 similar messages]
I20260709 13:07:04.468984 20597 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:04.478587 19983 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.482923 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.522699 14020 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:04.458401 20624 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:04.430997 20091 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.486981 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 790) took 1068 ms. Trace:
I20260709 13:07:04.470429 20630 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.386981 20632 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.488641 13987 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:04.458081 19782 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:04.512339 14015 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:04.533548 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.466466 20494 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 22 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:04.496618 20600 consensus_meta.cc:287] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.581s	user 0.007s	sys 0.006s
I20260709 13:07:04.491130 20158 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:04.504268 20442 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:04.524952 14022 rpcz_store.cc:276] 0709 13:07:03.419134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.463469 (+ 44335us) service_pool.cc:224] Handling call
0709 13:07:04.486775 (+1023306us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b54002eb654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:04.486913 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1849188608}
I20260709 13:07:04.545131 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.477119 20702 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.500283 20436 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:04.535823 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3401) took 1871 ms. Trace:
I20260709 13:07:04.535202 14015 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:04.542342 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:04.542677 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 139777us
I20260709 13:07:04.569375 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:04.570996 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:04.512040 20428 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.546216 13874 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:07:04.546643 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.698601 14022 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:07:04.699648 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:04.521108 20597 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:04.498905 20693 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:07:04.532593 19782 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.572338 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:04.755455 13949 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:07:04.534853 20705 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:04.567535 13873 rpcz_store.cc:276] 0709 13:07:02.664116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.664275 (+   159us) service_pool.cc:224] Handling call
0709 13:07:02.684448 (+ 20173us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:02.775640 (+ 91192us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:02.775676 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:02.799593 (+ 23917us) log.cc:844] Serialized 26 byte log entry
0709 13:07:02.927169 (+127576us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:02.927219 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.513895 (+1586676us) spinlock_profiling.cc:243] Waited 66.4 ms on lock 0x7b54002e77d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:04.513969 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:04.514016 (+    47us) raft_consensus.cc:1685] finished
0709 13:07:04.514027 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:04.535729 (+ 21702us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24205,"raft.queue_time_us":24462,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":19265,"spinlock_wait_cycles":152641920,"thread_start_us":96719,"threads_started":1,"wal-append.queue_time_us":348389,"wal-append.run_cpu_time_us":2764,"wal-append.run_wall_time_us":1075744}
I20260709 13:07:04.684674 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.762486 14015 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:04.762374 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:04.536150 20624 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:04.513118 20621 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.519125 20557 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.048s
I20260709 13:07:04.625355 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.511837 20500 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.509660 20647 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.508936 20584 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.515769 20469 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.508380 20461 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:07:04.599929 20640 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:04.693221 13945 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:04.693646 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.785593 14020 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
W20260709 13:07:04.700189 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 195170us
I20260709 13:07:04.700392 14022 raft_consensus.cc:2376] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 16. Re-sending same reply.
I20260709 13:07:04.545729 20600 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.763952 19782 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:04.763482 13869 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:04.673362 20661 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:04.767160 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:04.771319 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:04.774895 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:04.775831 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:04.891989 13872 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:07:04.780079 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:04.780455 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:04.893234 13873 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:07:04.741102 20597 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:04.768445 20606 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:04.626974 20609 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:04.507289 20618 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.918817 13949 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:04.921391 13871 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:04.796134 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.781482 20705 raft_consensus.cc:697] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:04.923878 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:04.925597 13947 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:04.926409 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.670447 20428 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.887396 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:04.927508 14020 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:04.927933 13945 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:07:04.887261 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.888253 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:04.932312 14023 raft_consensus.cc:2368] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17.
I20260709 13:07:04.660851 20442 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:04.784111 20598 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.857s	user 0.015s	sys 0.000s
I20260709 13:07:04.896970 13867 raft_consensus.cc:2393] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:04.758751 20474 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.896239 20564 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.912546 13870 raft_consensus.cc:2368] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 18. Current term is 19.
I20260709 13:07:04.787498 20700 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:04.790436 20717 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:04.984201 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:04.985320 14015 raft_consensus.cc:2410] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:07:04.963841 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:04.976595 13868 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:05.008711 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:04.784948 20624 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.022974 20624 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:04.784978 20436 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:05.026755 20716 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
I20260709 13:07:05.027536 20647 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:05.027563 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:05.029398 14020 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:04.983839 20717 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.031213 20717 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:05.019093 20632 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:05.019232 14016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 787) took 1615 ms (client timeout 1805 ms). Trace:
W20260709 13:07:05.032526 14016 rpcz_store.cc:269] 0709 13:07:03.403853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.726491 (+322638us) service_pool.cc:224] Handling call
0709 13:07:05.018998 (+1292507us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b540029aed4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:05.019155 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1496613632}
W20260709 13:07:05.034781 20706 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.336s	user 0.000s	sys 0.001s
W20260709 13:07:05.037317 20712 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.350s	user 0.000s	sys 0.001s
I20260709 13:07:05.028395 20647 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.038836 20647 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:05.038673 20705 consensus_queue.cc:237] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.916309 20707 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.124s
I20260709 13:07:05.030380 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:05.043506 14020 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:05.043548 20621 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:05.046809 20713 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.280s	user 0.000s	sys 0.000s
I20260709 13:07:05.047755 20630 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:05.048679 20647 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:05.050130 20500 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.051950 20584 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:05.055271 20469 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:04.984298 20474 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.063439 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 800) took 1384 ms. Trace:
I20260709 13:07:05.066526 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:05.069303 20600 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.070179 20600 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:05.070257 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:05.071017 20600 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.071133 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:05.071569 20600 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:05.071924 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:05.030275 20618 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.072890 14559 raft_consensus.cc:2410] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:07:05.074162 20474 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.075929 14020 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:07:05.064019 14024 rpcz_store.cc:276] 0709 13:07:03.673062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.802382 (+129320us) service_pool.cc:224] Handling call
0709 13:07:03.802874 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:03.819743 (+ 16869us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:03.819785 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:04.203954 (+384169us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b2c000e8788. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:07:04.204488 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:07:04.412955 (+208467us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:04.458985 (+ 46030us) log.cc:844] Serialized 14 byte log entry
0709 13:07:04.462979 (+  3994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.463081 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.057225 (+594144us) raft_consensus.cc:1685] finished
0709 13:07:05.057377 (+   152us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.057588 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10367,"prepare.queue_time_us":46068,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":20587,"raft.queue_time_us":41928,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":2280,"spinlock_wait_cycles":44351104,"thread_start_us":207985,"threads_started":1,"wal-append.queue_time_us":337418}
I20260709 13:07:05.079149 20702 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:04.982909 20564 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.770269 20494 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:05.096982 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:05.103408 13869 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:07:05.110780 20621 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:05.111594 20428 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:05.112752 20711 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.539s	user 0.001s	sys 0.000s
I20260709 13:07:05.111548 20621 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.115106 20621 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:05.115705 20621 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.116184 20621 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:04.975862 20598 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.121803 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:05.122838 14023 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:07:05.182569 13873 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:05.184298 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:05.186002 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:05.188346 13873 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:05.195945 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 3015) took 1478 ms. Trace:
I20260709 13:07:05.028280 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:05.204555 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:05.196571 13942 rpcz_store.cc:276] 0709 13:07:03.717056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.717190 (+   134us) service_pool.cc:224] Handling call
0709 13:07:03.717603 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:03.741052 (+ 23449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:03.741107 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:03.741752 (+   645us) log.cc:844] Serialized 26 byte log entry
0709 13:07:04.149570 (+407818us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:04.150602 (+  1032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.125643 (+975041us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b54001c7854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:05.125709 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.125766 (+    57us) raft_consensus.cc:1685] finished
0709 13:07:05.125788 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.195822 (+ 70034us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":660,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":52668672,"thread_start_us":407266,"threads_started":1,"wal-append.queue_time_us":474290}
I20260709 13:07:04.975951 20700 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:07:05.211354 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 639074us
W20260709 13:07:05.025911 20659 consensus_meta.cc:287] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.690s	user 0.012s	sys 0.000s
I20260709 13:07:05.124706 20584 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.233459 13875 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:07:05.234025 13875 pending_rounds.cc:85] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 7
I20260709 13:07:05.234308 13875 pending_rounds.cc:107] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:07:05.234617 13875 raft_consensus.cc:2889] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:05.205387 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 416361us
I20260709 13:07:05.010648 14015 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:05.209015 20472 consensus_meta.cc:287] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.924s	user 0.010s	sys 0.006s
I20260709 13:07:05.278014 13873 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:07:05.289883 13914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.008s	sys 0.000s
I20260709 13:07:05.270764 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3272) took 1246 ms. Trace:
W20260709 13:07:05.210269 20565 consensus_meta.cc:287] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.834s	user 0.007s	sys 0.006s
W20260709 13:07:05.272203 13839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.000s	sys 0.009s
W20260709 13:07:05.304714 14019 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.796s	user 0.013s	sys 0.000s
W20260709 13:07:05.329941 13839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.001s	sys 0.009s
I20260709 13:07:05.330271 14019 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:05.331612 14024 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:07:05.333488 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:05.334596 14019 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:05.303243 14015 rpcz_store.cc:276] 0709 13:07:04.024214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.008299 (+984085us) service_pool.cc:224] Handling call
0709 13:07:05.270668 (+262369us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:05.337589 20731 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.338375 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:05.338223 20731 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.339016 14015 raft_consensus.cc:2376] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12. Re-sending same reply.
I20260709 13:07:05.340133 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:05.340646 14015 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:05.341562 20700 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:05.342283 20727 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:05.342872 20700 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:05.343199 20704 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:05.346712 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.349182 13872 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:05.350368 13867 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:05.223698 20659 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.387092 14019 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:05.395763 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:05.273087 20584 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.406731 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:05.284198 20647 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:07:05.415944 13873 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 22: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:05.434607 14015 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:05.436218 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:05.356572 20731 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.354426 20472 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:05.458686 20472 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.461243 20472 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.343010 20727 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.463588 20659 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:05.463829 20727 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.464442 20727 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:05.464558 20722 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.359539 20721 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.466213 20721 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.465035 20727 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.468590 20721 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.469900 20598 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:05.471891 20727 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.355768 20704 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:05.478082 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:05.479450 20584 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:05.479900 13838 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:05.480429 20685 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.399s	user 0.000s	sys 0.012s
I20260709 13:07:05.480741 20629 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:05.452804 20709 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:05.481878 20709 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:05.454437 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3270) took 1689 ms. Trace:
I20260709 13:07:05.482824 13943 rpcz_store.cc:276] 0709 13:07:03.764617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.780080 (+ 15463us) service_pool.cc:224] Handling call
0709 13:07:03.987133 (+207053us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:03.990930 (+  3797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.454174 (+1463244us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b5400092250. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:05.454364 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475080576}
I20260709 13:07:05.405972 20640 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:05.482268 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:05.483765 20621 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:05.484232 13866 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
W20260709 13:07:05.453646 20684 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 1.375s	user 0.002s	sys 0.020s
I20260709 13:07:05.484563 20640 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.485284 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:05.473047 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:05.486616 13868 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:05.482556 20597 consensus_meta.cc:287] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.000s
I20260709 13:07:05.488579 20597 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:05.441457 20359 consensus_meta.cc:287] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.059s	user 0.013s	sys 0.002s
I20260709 13:07:05.490533 20359 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19.
I20260709 13:07:05.490952 20597 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.491263 13944 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:05.495585 20626 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.033s	user 0.010s	sys 0.002s
I20260709 13:07:05.496044 20626 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.496533 20655 ts_tablet_manager.cc:1929] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 12.9
I20260709 13:07:05.497131 20655 log.cc:1199] T 6b098cc2d51e471cb2c9c1b17eebcb57 P e1649107c781428692458f10597c5742: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/wals/6b098cc2d51e471cb2c9c1b17eebcb57
I20260709 13:07:05.497326 13870 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:07:05.499341 20645 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 1.444s	user 0.012s	sys 0.003s
I20260709 13:07:05.499841 20645 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:05.502267 13841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.978s	user 0.041s	sys 0.098s
W20260709 13:07:05.502687 13841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.978s	user 0.041s	sys 0.098s
I20260709 13:07:05.502342 14016 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:07:05.377581 20565 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.476723 20704 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:05.484386 20709 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:05.519623 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.472889 20600 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:05.542533 13868 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:05.515615 13871 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:05.548867 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:05.549443 20687 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.497s	user 0.000s	sys 0.012s
I20260709 13:07:05.549618 14021 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:07:05.550910 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:05.551848 14015 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:07:05.554471 14524 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:05.555658 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:05.510743 20474 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:05.551478 14021 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:05.589856 14524 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:05.512251 20645 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:05.592685 14523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.007s	sys 0.027s
I20260709 13:07:05.597694 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:07:05.611012 14523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.008s	sys 0.027s
I20260709 13:07:05.611424 13871 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:05.597821 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:05.600919 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:05.613197 13869 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:05.612653 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 799) took 1940 ms. Trace:
I20260709 13:07:05.519415 20647 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.465246 20722 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.614328 14018 rpcz_store.cc:276] 0709 13:07:03.672133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.756352 (+ 84219us) service_pool.cc:224] Handling call
0709 13:07:03.756653 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:03.783950 (+ 27297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:03.783987 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:03.803939 (+ 19952us) log.cc:844] Serialized 26 byte log entry
0709 13:07:03.964239 (+160300us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:04.083687 (+119448us) log.cc:844] Serialized 14 byte log entry
0709 13:07:04.084580 (+   893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.085323 (+   743us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.548477 (+1463154us) spinlock_profiling.cc:243] Waited 60.7 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:05.592370 (+ 43893us) raft_consensus.cc:1685] finished
0709 13:07:05.612410 (+ 20040us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.612589 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15225,"prepare.queue_time_us":193883,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":65,"raft.queue_time_us":31927,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":6424,"spinlock_wait_cycles":139678080,"thread_start_us":244517,"threads_started":2,"wal-append.queue_time_us":228228}
I20260709 13:07:05.619427 20727 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.621697 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:05.511149 20359 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 13:07:05.624528 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:05.624665 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:05.630859 13941 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:05.631261 13947 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:05.623288 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:05.592886 20702 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:05.518712 20565 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.602823 20474 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.635259 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:05.658265 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:05.643940 13868 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:07:05.645532 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:05.592761 20626 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.661758 13948 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:05.675406 13989 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.602s (SENT)
I20260709 13:07:05.586632 20472 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 13:07:05.690501 20472 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:05.690912 20472 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:05.691813 20472 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.693140 20472 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:05.630347 20717 consensus_meta.cc:287] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.598s	user 0.004s	sys 0.010s
I20260709 13:07:05.676088 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:05.538147 20629 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.087s
I20260709 13:07:05.677507 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:05.678457 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:05.628418 20469 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:05.683480 20158 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.137s	user 0.011s	sys 0.001s
I20260709 13:07:05.627512 20655 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46040 (request call id 0) took 3141 ms. Trace:
I20260709 13:07:05.689412 20728 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.630591 20560 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:05.743875 20158 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 24. Current term is 25.
I20260709 13:07:05.682209 20702 raft_consensus.cc:697] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:05.755381 20560 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:05.756007 20560 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:05.756140 20702 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:05.758468 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 88587us
I20260709 13:07:05.760450 20669 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.628194 20570 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:05.761174 20570 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:05.761632 20570 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:05.770599 20629 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:05.771283 20629 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:05.772641 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:05.714226 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:05.685755 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:05.677243 20359 consensus_queue.cc:1243] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:05.717124 13947 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:05.717841 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:07:05.774761 19782 consensus_meta.cc:287] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.859s	user 0.010s	sys 0.001s
W20260709 13:07:05.775224 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 97185.4us
I20260709 13:07:05.775429 19782 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.711846 13948 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:05.748337 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:05.626533 20584 consensus_queue.cc:1059] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:07:05.673055 20474 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:05.752946 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:05.716262 14018 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:07:05.776387 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:05.782910 13875 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:05.793499 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:07:05.782847 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 97178.1us
I20260709 13:07:05.794922 13839 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:05.795821 14559 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:05.629128 20647 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.802760 13869 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:05.803674 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:05.636315 20727 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.807017 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:05.808172 19782 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.808458 13839 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:05.813184 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:05.813357 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:05.815071 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:05.815732 14018 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:05.818939 20565 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:05.820811 20719 log.cc:927] Time spent T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.378s	user 0.001s	sys 0.000s
I20260709 13:07:05.821406 20739 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:05.746068 20655 rpcz_store.cc:276] 0709 13:07:02.485450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.538160 (+ 52710us) service_pool.cc:224] Handling call
0709 13:07:03.203073 (+664913us) ts_tablet_manager.cc:1151] Deleting tablet 6b098cc2d51e471cb2c9c1b17eebcb57
0709 13:07:03.203202 (+   129us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:07:05.074232 (+1871030us) tablet_metadata.cc:665] Metadata flushed
0709 13:07:05.496371 (+422139us) tablet_metadata.cc:665] Metadata flushed
0709 13:07:05.606693 (+110322us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":585950,"tablet-delete.queue_time_us":336699,"thread_start_us":79615,"threads_started":1}
I20260709 13:07:05.776022 20728 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:05.824590 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:05.826655 13839 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:05.793220 20693 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.834594 13987 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:05.832736 19782 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:05.802680 20359 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:05.742924 20717 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.802237 20640 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:05.845224 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:05.838021 20565 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:05.852591 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3270) took 1829 ms. Trace:
I20260709 13:07:05.854892 20727 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.826596 20726 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:05.858708 20722 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.861917 20728 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:05.862881 20714 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.869s	user 0.002s	sys 0.012s
I20260709 13:07:05.865222 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 961) took 1596 ms. Trace:
I20260709 13:07:05.748564 20469 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:05.827919 20158 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:05.867733 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:05.868460 20701 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.772333 20629 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:05.874400 20693 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:05.825518 20609 raft_consensus.cc:3055] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:05.876168 20584 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:07:05.827936 20710 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.800s	user 0.001s	sys 0.012s
I20260709 13:07:05.880702 20618 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:05.883451 13875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3319) took 1098 ms. Trace:
I20260709 13:07:05.832758 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:05.892562 14023 rpcz_store.cc:276] 0709 13:07:04.023066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.123751 (+1100685us) service_pool.cc:224] Handling call
0709 13:07:05.209508 (+ 85757us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.210077 (+   569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.210112 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.230759 (+ 20647us) log.cc:844] Serialized 26 byte log entry
0709 13:07:05.786130 (+555371us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:05.786187 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.786284 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.852274 (+ 65990us) raft_consensus.cc:1685] finished
0709 13:07:05.852314 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.852507 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":141,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":17650,"thread_start_us":497612,"threads_started":1,"wal-append.queue_time_us":523619}
I20260709 13:07:05.894035 14018 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
W20260709 13:07:05.897030 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 51906.5us
I20260709 13:07:05.906211 13871 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:05.914639 20686 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:05.946107 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 356318us
I20260709 13:07:05.945408 20727 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:05.948371 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 151335us
I20260709 13:07:05.959487 13874 rpcz_store.cc:276] 0709 13:07:04.268777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.268944 (+   167us) service_pool.cc:224] Handling call
0709 13:07:05.865027 (+1596083us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54002e6dd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:05.865154 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489092864}
I20260709 13:07:05.883324 19983 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:05.891270 20633 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:05.896582 20707 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:06.029511 20686 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:06.032203 20686 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:06.033988 20717 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:05.895071 20620 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.118s
I20260709 13:07:05.972319 20158 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.898418 20704 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.020129 13869 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 22: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:06.016572 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:06.017547 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:05.954375 20739 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:06.023898 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:06.066931 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 307621us
I20260709 13:07:06.015175 13875 rpcz_store.cc:276] 0709 13:07:04.784847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.236084 (+451237us) service_pool.cc:224] Handling call
0709 13:07:05.288523 (+ 52439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.793102 (+504579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.883145 (+ 90043us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54003ac5d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:05.883379 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1122700544}
I20260709 13:07:05.899173 20640 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:06.064687 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48196.1us
I20260709 13:07:06.065726 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:06.075331 13874 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:07:06.076426 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:06.077162 13874 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:06.080607 20686 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:06.081765 20686 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:06.021884 19983 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:06.073040 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 55595us
I20260709 13:07:06.089605 19983 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:06.092680 20750 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:06.074174 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 50393.1us
I20260709 13:07:06.093580 20750 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:06.094548 14553 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:06.097097 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:06.099108 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:06.100435 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:06.101042 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:06.101300 14555 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 19: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:06.102360 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:06.103047 14561 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:07:06.115851 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:06.117411 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:05.980355 20701 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:06.020804 20736 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.982185 20609 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:06.018091 20618 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:06.127513 14558 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:07:06.062021 20158 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:06.131013 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:06.021497 20584 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:06.039813 20727 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.058441 20633 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:06.182789 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.163009 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.204198 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:06.205189 13948 raft_consensus.cc:2410] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:07:06.162672 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:06.066728 20629 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:06.206260 14023 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:06.070389 20621 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:06.220484 19983 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.228669 13874 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:06.237362 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:06.241756 13874 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:06.115389 20645 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:06.195976 20744 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:07:06.298815 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:06.299908 13875 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:06.125131 20442 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:06.308456 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 203938us
I20260709 13:07:06.311338 19983 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:06.083767 20686 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:06.075989 20704 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:06.216101 20584 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:06.331188 20750 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:06.331557 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:06.348776 13874 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:06.332535 14555 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:07:06.333719 20751 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 13:07:06.167430 20762 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:06.316903 20474 consensus_meta.cc:287] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.506s	user 0.012s	sys 0.000s
I20260709 13:07:06.308427 20620 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:06.315127 20442 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:06.368752 20746 log.cc:927] Time spent T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.322s	user 0.001s	sys 0.001s
W20260709 13:07:06.368498 20729 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:07:06.438580 20469 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:06.438908 20570 consensus_meta.cc:287] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.677s	user 0.013s	sys 0.000s
I20260709 13:07:06.439373 20570 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:06.440202 20589 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:06.444558 20704 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:06.392479 20474 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.445180 20704 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.447432 20633 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:06.424469 13874 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:06.315582 20609 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.450341 20609 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:06.450534 13874 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:06.452020 20564 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:06.428951 20752 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:07:06.157160 20640 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.193878 20739 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.416203 20762 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:06.417187 20771 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
W20260709 13:07:06.430652 20759 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:07:06.434685 20693 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:06.416510 13875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76071b78766a43e0b5097f615de4c569"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
W20260709 13:07:06.417673 20560 consensus_meta.cc:287] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.661s	user 0.008s	sys 0.005s
I20260709 13:07:06.439751 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3405) took 1081 ms. Trace:
I20260709 13:07:06.407976 13869 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 25.
I20260709 13:07:06.451988 13874 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:07:06.457315 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.459008 13989 leader_election.cc:336] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.864s (SENT)
I20260709 13:07:06.132711 20736 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:06.466867 13875 raft_consensus.cc:606] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:06.445736 20704 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.423784 20442 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:06.484193 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3274) took 2459 ms. Trace:
I20260709 13:07:06.479202 20570 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:06.483136 20688 log.cc:927] Time spent T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 2.394s	user 0.025s	sys 0.003s
W20260709 13:07:06.468545 20700 consensus_meta.cc:287] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 1.125s	user 0.010s	sys 0.005s
I20260709 13:07:06.483912 13867 raft_consensus.cc:2410] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:07:06.485985 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3406) took 1127 ms. Trace:
I20260709 13:07:06.509789 13873 rpcz_store.cc:276] 0709 13:07:05.358285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.443536 (+ 85251us) service_pool.cc:224] Handling call
0709 13:07:05.444084 (+   548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.517706 (+ 73622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.517758 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.539495 (+ 21737us) log.cc:844] Serialized 26 byte log entry
0709 13:07:05.948779 (+409284us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:05.948836 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.948925 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.485723 (+536798us) raft_consensus.cc:1685] finished
0709 13:07:06.485751 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.485909 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":138750,"raft.queue_time_us":93812,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":65258,"thread_start_us":292641,"threads_started":1,"wal-append.queue_time_us":527068}
I20260709 13:07:06.479144 20762 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.471122 13872 rpcz_store.cc:276] 0709 13:07:05.357990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.409994 (+ 52004us) service_pool.cc:224] Handling call
0709 13:07:05.410531 (+   537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.429102 (+ 18571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.429159 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.448018 (+ 18859us) log.cc:844] Serialized 26 byte log entry
0709 13:07:05.784297 (+336279us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:05.784359 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.784465 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.439540 (+655075us) raft_consensus.cc:1685] finished
0709 13:07:06.439562 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.439692 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17101,"raft.queue_time_us":121,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":20427,"thread_start_us":276586,"threads_started":1,"wal-append.queue_time_us":531194}
I20260709 13:07:06.491204 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:06.495338 13875 raft_consensus.cc:614] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:06.466427 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:06.499214 20693 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.504802 14021 rpcz_store.cc:276] 0709 13:07:04.025002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.612750 (+1587748us) service_pool.cc:224] Handling call
0709 13:07:05.613166 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.618295 (+  5129us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:05.618350 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.629372 (+ 11022us) log.cc:844] Serialized 26 byte log entry
0709 13:07:06.102733 (+473361us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:06.102782 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.102855 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.483923 (+381068us) raft_consensus.cc:1685] finished
0709 13:07:06.483946 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.484112 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4679,"raft.queue_time_us":4863,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":4489,"thread_start_us":405638,"threads_started":1,"wal-append.queue_time_us":707526}
I20260709 13:07:06.385833 20618 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.514449 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3424) took 1012 ms. Trace:
I20260709 13:07:06.521804 13874 rpcz_store.cc:276] 0709 13:07:05.501986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.499032 (+997046us) service_pool.cc:224] Handling call
0709 13:07:06.513848 (+ 14816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.514074 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:06.514105 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:06.514152 (+    47us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:06.514189 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.514246 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.514386 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.530239 20693 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:06.534435 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3408) took 1124 ms. Trace:
I20260709 13:07:06.535710 20704 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:06.534957 13866 rpcz_store.cc:276] 0709 13:07:05.410140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.488604 (+ 78464us) service_pool.cc:224] Handling call
0709 13:07:05.488944 (+   340us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:05.505242 (+ 16298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.505276 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:05.506040 (+   764us) log.cc:844] Serialized 52 byte log entry
0709 13:07:05.895882 (+389842us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:05.895934 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.896460 (+   526us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.534131 (+637671us) raft_consensus.cc:1685] finished
0709 13:07:06.534158 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.534361 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85748,"raft.queue_time_us":48303,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":65833,"thread_start_us":304532,"threads_started":1,"wal-append.queue_time_us":982987}
I20260709 13:07:06.536164 20736 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:06.536396 20704 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:06.536887 20736 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.537365 13866 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:06.539798 13875 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:07:06.540710 20474 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:06.542157 13873 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:06.544191 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3273) took 2519 ms. Trace:
I20260709 13:07:06.544607 13872 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:06.544564 14015 rpcz_store.cc:276] 0709 13:07:04.024616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.597679 (+1573063us) service_pool.cc:224] Handling call
0709 13:07:05.598153 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.598583 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.598629 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.599260 (+   631us) log.cc:844] Serialized 26 byte log entry
0709 13:07:05.899544 (+300284us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:05.899600 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.899680 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.543969 (+644289us) raft_consensus.cc:1685] finished
0709 13:07:06.543991 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.544133 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1040,"raft.queue_time_us":4733,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":12591,"thread_start_us":223857,"threads_started":1,"wal-append.queue_time_us":901748}
I20260709 13:07:06.545765 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:06.553228 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:06.554345 14015 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:06.488850 20560 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:06.537868 20704 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:06.610841 14552 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 24.
I20260709 13:07:06.623064 14015 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:06.624639 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:06.549448 20727 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:06.642388 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:06.643762 14015 raft_consensus.cc:2410] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:07:06.645748 14556 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:06.585260 20618 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:06.672118 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.683890 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.537376 20736 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.707041 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:06.707981 14015 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:06.770212 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:06.771101 14015 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:06.606241 20621 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:06.778859 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:06.779969 14017 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 21.
W20260709 13:07:06.782644 13913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.027s	sys 0.081s
W20260709 13:07:06.783046 13913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.027s	sys 0.082s
I20260709 13:07:06.781571 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:06.644393 20704 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:06.789597 14017 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:07:06.790433 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 801) took 2628 ms. Trace:
I20260709 13:07:06.790791 14017 rpcz_store.cc:276] 0709 13:07:04.161640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.787505 (+2625865us) service_pool.cc:224] Handling call
0709 13:07:06.788868 (+  1363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.790117 (+  1249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.790337 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.807600 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3268) took 2785 ms. Trace:
I20260709 13:07:06.808147 14020 rpcz_store.cc:276] 0709 13:07:04.022453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.076794 (+1054341us) service_pool.cc:224] Handling call
0709 13:07:05.077199 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:05.077720 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.077755 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:05.877080 (+799325us) spinlock_profiling.cc:243] Waited 41.4 ms on lock 0x7b2c000e8788. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c1d2ca11 00007fb2c1d3c372 00007fb2c346ea01 00007fb2c343343a 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:07:05.877943 (+   863us) log.cc:844] Serialized 52 byte log entry
0709 13:07:06.098473 (+220530us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:06.098523 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.098621 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.793296 (+694675us) raft_consensus.cc:1685] finished
0709 13:07:06.793324 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.807503 (+ 14179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":875,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":142,"spinlock_wait_cycles":95181568,"thread_start_us":219982,"threads_started":1,"wal-append.queue_time_us":799813}
I20260709 13:07:06.794932 20700 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:06.783138 20787 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:06.794450 20727 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:06.867790 14020 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:07:06.807250 20731 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.874744 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 805) took 2638 ms. Trace:
I20260709 13:07:06.877177 14020 rpcz_store.cc:276] 0709 13:07:04.236520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.866951 (+2630431us) service_pool.cc:224] Handling call
0709 13:07:06.867345 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.874433 (+  7088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.874683 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.814153 20763 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.813340 20762 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.885291 14019 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:07:06.887650 20779 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:07:06.882767 20763 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.895486 14019 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 24.
W20260709 13:07:06.895643 20158 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.601s	user 0.012s	sys 0.000s
I20260709 13:07:06.864833 20700 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.938004 13944 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:06.938256 20700 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:06.942546 14558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1230) took 1154 ms. Trace:
I20260709 13:07:06.942993 14558 rpcz_store.cc:276] 0709 13:07:05.788293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.788547 (+   254us) service_pool.cc:224] Handling call
0709 13:07:05.822294 (+ 33747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.205035 (+382741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.929992 (+724957us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b54005622d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:06.942473 (+ 12481us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":694468864}
I20260709 13:07:06.944876 20693 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:06.947342 20704 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:06.953438 20701 consensus_meta.cc:287] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.808s	user 0.007s	sys 0.009s
I20260709 13:07:06.953644 20091 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:06.933985 20762 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.956418 20762 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:06.957113 20560 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:06.880494 20727 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:06.959823 20727 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:06.963896 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3291) took 1215 ms. Trace:
I20260709 13:07:06.968009 20442 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.964522 13943 rpcz_store.cc:276] 0709 13:07:05.748193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.749651 (+  1458us) service_pool.cc:224] Handling call
0709 13:07:05.750145 (+   494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.885409 (+135264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.885459 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.958627 (+ 73168us) log.cc:844] Serialized 26 byte log entry
0709 13:07:06.206790 (+248163us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:06.206851 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.963269 (+756418us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54003309d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:06.963352 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.963421 (+    69us) raft_consensus.cc:1685] finished
0709 13:07:06.963443 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.963807 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3380,"raft.queue_time_us":88388,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":6829,"spinlock_wait_cycles":307049856,"thread_start_us":227938,"threads_started":1,"wal-append.queue_time_us":488966}
I20260709 13:07:06.947220 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:06.970794 20647 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:06.976295 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:06.977272 13868 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:06.897871 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 965) took 1267 ms. Trace:
W20260709 13:07:06.979066 14019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2743) took 1733 ms (client timeout 2088 ms). Trace:
W20260709 13:07:06.979736 14019 rpcz_store.cc:269] 0709 13:07:05.245773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.395353 (+149580us) service_pool.cc:224] Handling call
0709 13:07:06.978827 (+1583474us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b540023bc54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:06.978989 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3412805888}
I20260709 13:07:06.899181 20621 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:06.901851 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3278) took 2571 ms. Trace:
I20260709 13:07:06.982348 14020 rpcz_store.cc:276] 0709 13:07:04.330210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.900884 (+2570674us) service_pool.cc:224] Handling call
0709 13:07:06.901272 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.901493 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:06.901522 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:06.901569 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:06.901604 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.901663 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.901778 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.881786 20731 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:06.983712 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:06.906733 20776 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 13:07:06.909121 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:06.893733 20763 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:06.986662 20620 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:06.987182 20620 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:06.986888 20763 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:07:06.912003 20736 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:06.913225 20584 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.527s	user 0.005s	sys 0.006s
I20260709 13:07:06.990419 20584 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:06.977900 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3419) took 1477 ms. Trace:
I20260709 13:07:06.914554 20607 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.913954 14015 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
W20260709 13:07:07.001492 14023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.032s	sys 0.074s
W20260709 13:07:07.001837 14023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.032s	sys 0.075s
I20260709 13:07:06.805037 20570 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
W20260709 13:07:06.921168 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 462142us
W20260709 13:07:07.005050 13989 leader_election.cc:336] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.874s (SENT)
W20260709 13:07:07.005359 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 58279us
I20260709 13:07:07.005506 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:06.928537 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:07.006333 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.953994 20701 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.010229 20701 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:07.010497 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:06.928745 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:06.929037 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:06.931341 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3266) took 2909 ms. Trace:
I20260709 13:07:07.012260 14016 rpcz_store.cc:276] 0709 13:07:04.021718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.033710 (+1011992us) service_pool.cc:224] Handling call
0709 13:07:05.034118 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.503790 (+469672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.931066 (+1427276us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b54002eac54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:06.931278 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1076129664}
W20260709 13:07:06.935586 13870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3316) took 2180 ms (client timeout 2105 ms). Trace:
I20260709 13:07:06.937242 20158 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.013541 13874 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:07.013970 13870 rpcz_store.cc:269] 0709 13:07:04.755397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.977082 (+221685us) service_pool.cc:224] Handling call
0709 13:07:06.935377 (+1958295us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54003abbd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:06.935513 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":917687680}
I20260709 13:07:06.985438 20640 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.991405 13868 rpcz_store.cc:276] 0709 13:07:05.500524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.665517 (+164993us) service_pool.cc:224] Handling call
0709 13:07:06.976074 (+1310557us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b5800003698. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:07:06.977818 (+  1744us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21012736}
I20260709 13:07:07.015225 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:07.015448 20640 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.015821 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:07.013703 14016 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:07.016484 13868 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 25.
I20260709 13:07:06.981827 14019 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:07:06.992432 14017 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:06.994704 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:07.017767 20640 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.018045 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:07.018941 13866 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 22.
I20260709 13:07:06.979255 13869 rpcz_store.cc:276] 0709 13:07:05.630669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.490895 (+860226us) service_pool.cc:224] Handling call
0709 13:07:06.492200 (+  1305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:06.492823 (+   623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:06.492874 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:06.502380 (+  9506us) log.cc:844] Serialized 26 byte log entry
0709 13:07:06.794661 (+292281us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:06.794743 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.795008 (+   265us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.897570 (+102562us) raft_consensus.cc:1685] finished
0709 13:07:06.897597 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.897790 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":8200,"thread_start_us":243984,"threads_started":1,"wal-append.queue_time_us":375910}
W20260709 13:07:06.997128 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.020267 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:07.002755 14015 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 13:07:06.997468 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 65498.5us
I20260709 13:07:07.021312 13875 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 22: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:07.021746 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2733) took 2627 ms. Trace:
I20260709 13:07:07.022392 14559 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:07.022863 20647 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:07.024572 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.025717 13941 raft_consensus.cc:2410] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:07:07.003391 14020 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:07:07.003253 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3276) took 2883 ms. Trace:
I20260709 13:07:07.016888 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2734) took 2622 ms. Trace:
I20260709 13:07:07.026561 14016 rpcz_store.cc:276] 0709 13:07:04.394256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.013227 (+2618971us) service_pool.cc:224] Handling call
0709 13:07:07.013517 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.016656 (+  3139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.016835 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.022123 14015 rpcz_store.cc:276] 0709 13:07:04.393981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.002007 (+2608026us) service_pool.cc:224] Handling call
0709 13:07:07.002419 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:07.021464 (+ 19045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.021690 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.026437 14023 rpcz_store.cc:276] 0709 13:07:04.119865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.281324 (+2161459us) service_pool.cc:224] Handling call
0709 13:07:06.316337 (+ 35013us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.316610 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:06.474837 (+158227us) log.cc:844] Serialized 14 byte log entry
0709 13:07:07.002889 (+528052us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:07.002945 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:07.003005 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.003082 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:07.003200 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":644,"prepare.queue_time_us":497035,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":42,"raft.queue_time_us":1578,"thread_start_us":623836,"threads_started":2}
I20260709 13:07:07.028713 14016 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:07:07.029174 13946 raft_consensus.cc:1240] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:07.029448 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2736) took 2634 ms. Trace:
I20260709 13:07:07.029956 14016 rpcz_store.cc:276] 0709 13:07:04.394712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.028013 (+2633301us) service_pool.cc:224] Handling call
0709 13:07:07.028416 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:07.029142 (+   726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.029379 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.985502 20600 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:07.033674 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.034396 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:07.034997 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:07.037283 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:07.037899 13869 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:07.041474 14016 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:07.042145 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2741) took 1797 ms. Trace:
I20260709 13:07:07.042518 14016 rpcz_store.cc:276] 0709 13:07:05.245088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.040824 (+1795736us) service_pool.cc:224] Handling call
0709 13:07:07.041139 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.041878 (+   739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.042090 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.044883 13866 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "765348f2471a493d92e998132c0c746b"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:07:07.045750 13866 raft_consensus.cc:606] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:07:07.046205 13866 raft_consensus.cc:614] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:06.993268 20773 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.048025 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.050510 14016 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:07.051141 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:06.985442 20731 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.051690 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3282) took 1404 ms. Trace:
I20260709 13:07:07.052543 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.053572 14016 rpcz_store.cc:276] 0709 13:07:05.647355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.049840 (+1402485us) service_pool.cc:224] Handling call
0709 13:07:07.050172 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.051436 (+  1264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.051637 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.055052 14016 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:07.055246 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 808) took 2266 ms. Trace:
I20260709 13:07:07.033655 20600 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.055608 14023 rpcz_store.cc:276] 0709 13:07:04.788675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.041076 (+2252401us) service_pool.cc:224] Handling call
0709 13:07:07.054755 (+ 13679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.054940 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:07.054965 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:07.055006 (+    41us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:07.055037 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.055091 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:07.055192 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.053159 20498 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:07.063175 20442 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:07.063628 20498 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:07.066910 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:07.070309 20693 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:07.072039 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.072724 13941 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:07:07.078352 20158 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.079073 20600 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.079748 20584 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.050912 20662 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 13:07:07.080636 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 75636.2us
I20260709 13:07:07.066866 13946 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:07.083192 14023 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:07.083838 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3285) took 1434 ms. Trace:
I20260709 13:07:07.083916 20774 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.084717 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:07.084525 20774 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.083912 20693 raft_consensus.cc:3055] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:07.086542 20693 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:07.087497 20693 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.088723 20693 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:07.084209 14023 rpcz_store.cc:276] 0709 13:07:05.649586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.062046 (+1412460us) service_pool.cc:224] Handling call
0709 13:07:07.082791 (+ 20745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.083578 (+   787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.083766 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.090668 14021 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:07.091768 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:07.092059 13989 leader_election.cc:336] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.000s (SENT)
I20260709 13:07:07.093408 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:07.046816 20560 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:07:07.123333 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.097071 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3287) took 1445 ms. Trace:
I20260709 13:07:07.124418 14021 rpcz_store.cc:276] 0709 13:07:05.651658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.070439 (+1418781us) service_pool.cc:224] Handling call
0709 13:07:07.070774 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.096720 (+ 25946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.096989 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.125881 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:07.126637 14021 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:07.055608 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3283) took 1407 ms. Trace:
I20260709 13:07:07.127815 14016 rpcz_store.cc:276] 0709 13:07:05.648146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.054435 (+1406289us) service_pool.cc:224] Handling call
0709 13:07:07.054755 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.055385 (+   630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.055557 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22784}
I20260709 13:07:07.129349 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.092943 13946 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:07.097718 13869 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:07:07.103556 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:07.138653 14016 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:07:07.139706 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:07.141891 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:07.141059 13946 raft_consensus.cc:1275] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:07:07.108613 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.024214 20607 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:07.121650 20588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.627s	user 0.010s	sys 0.028s
I20260709 13:07:07.094883 20584 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.146831 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:07.121847 20560 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.142846 20620 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:07.147856 13871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3415) took 1648 ms (client timeout 2024 ms). Trace:
I20260709 13:07:07.163750 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:07.140641 20707 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.141518 20775 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.145989 14023 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:07.135402 20498 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:07.171195 13871 rpcz_store.cc:269] 0709 13:07:05.499415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.657082 (+157667us) service_pool.cc:224] Handling call
0709 13:07:07.147633 (+1490551us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54000d9654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:07.147776 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303651584}
I20260709 13:07:07.172513 13946 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:07.173959 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:07:07.162982 20588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.697s	user 0.011s	sys 0.029s
W20260709 13:07:07.147255 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55255.3us
I20260709 13:07:07.174121 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:07.201562 13950 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:07.201537 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.203483 14522 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:07.178705 14016 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:07.160707 20578 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.170648 20560 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:07.191244 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:07.057168 20773 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.192857 13947 raft_consensus.cc:2393] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:07.213429 14021 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:07:07.213780 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.196820 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.154631 20704 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:07.198884 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.219287 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.192574 20701 consensus_queue.cc:1059] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:07:07.201211 20588 raft_consensus.cc:697] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:07.204867 20662 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:07.210562 20774 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.192255 20607 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.234851 20662 consensus_queue.cc:1048] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:07.220753 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.226272 14015 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:07.227082 20588 consensus_queue.cc:237] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:07.236778 13946 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:07:07.202936 20442 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:07.229681 20769 log.cc:927] Time spent T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.861s	user 0.000s	sys 0.010s
I20260709 13:07:07.217717 20228 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:07:07.217800 20578 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.241623 13946 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:07:07.171068 20584 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:07.210228 20663 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:07.248458 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:07.250517 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.252240 13946 pending_rounds.cc:85] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 7
I20260709 13:07:07.277829 13946 pending_rounds.cc:107] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:07:07.278321 13946 raft_consensus.cc:2889] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:07.279276 13949 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:07.197347 20707 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:07.200662 20775 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:07.199877 20700 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.289316 13941 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:07.290449 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.253114 20600 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.298357 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:07.298650 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2728) took 2906 ms. Trace:
I20260709 13:07:07.299052 14020 rpcz_store.cc:276] 0709 13:07:04.392680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.983343 (+2590663us) service_pool.cc:224] Handling call
0709 13:07:07.003154 (+ 19811us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:07.026409 (+ 23255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.298460 (+272051us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b540023a350. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:07.298591 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44678528}
I20260709 13:07:07.300297 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.301115 14020 raft_consensus.cc:2410] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:07:07.298144 20600 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.304072 20600 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.296332 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 803) took 3130 ms. Trace:
I20260709 13:07:07.305542 14017 rpcz_store.cc:276] 0709 13:07:04.166239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.792039 (+2625800us) service_pool.cc:224] Handling call
0709 13:07:06.792436 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.017643 (+225207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.296052 (+278409us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b5400325ad4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:07.296262 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459495168}
I20260709 13:07:07.258586 20578 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.236260 20704 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:07.324138 13949 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:07.263674 20620 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:07.301875 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:07.328930 14020 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:07.320622 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.321759 13946 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:07.301658 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.284286 20663 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.326527 20158 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:07.330581 13947 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:07.331308 13941 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:07.247983 20442 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.341436 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.343212 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.344108 14017 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:07.345414 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:07.346311 14017 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:07.256515 14023 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:07.332867 13942 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:07:07.354089 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3290) took 1557 ms. Trace:
I20260709 13:07:07.354672 14023 rpcz_store.cc:276] 0709 13:07:05.796563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.122613 (+1326050us) service_pool.cc:224] Handling call
0709 13:07:07.145854 (+ 23241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.353704 (+207850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.353988 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.378934 20628 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:07.380834 20091 consensus_queue.cc:1059] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:07:07.389943 13989 leader_election.cc:336] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 3.072s (SENT)
I20260709 13:07:07.458398 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:07.395304 14020 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:07.364776 20682 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:07.423909 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:07.459921 13949 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:07.457543 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.461342 14023 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:07:06.981658 13759 catalog_manager.cc:5028] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): tablet 6b098cc2d51e471cb2c9c1b17eebcb57 (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
W20260709 13:07:07.468220 13759 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 486655us
W20260709 13:07:07.471722 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 260126us
W20260709 13:07:07.474617 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.445612 13944 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:07.475703 20652 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:07.505038 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 295992us
I20260709 13:07:07.470595 20584 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:07.486917 20158 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:07.484849 20091 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.521274 14017 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
W20260709 13:07:07.524155 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.505400 20619 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:07.537488 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:07.538230 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 148332us
I20260709 13:07:07.539263 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.540040 14017 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:07.540463 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:07.517592 20158 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:07.560804 20775 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:07.559365 20774 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:07:07.568707 20686 raft_consensus.cc:3055] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:07.572296 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.573071 13948 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:07:07.574047 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.569329 20686 raft_consensus.cc:740] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:07.534940 20704 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:07.582387 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.546967 20588 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:07.585110 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.559445 20578 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:07:07.527289 20091 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:07.600607 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 806) took 3214 ms. Trace:
I20260709 13:07:07.600862 13949 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:07.631949 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.601606 20732 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:07.601655 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.635665 14021 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
W20260709 13:07:07.602766 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62365us
I20260709 13:07:07.604408 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:07.554764 20700 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:07.598760 13948 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:07:07.618827 13840 connection.cc:256] RPC call timeout handler was delayed by -1.7328585550000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
I20260709 13:07:07.630482 14018 rpcz_store.cc:276] 0709 13:07:04.386650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.971693 (+2585043us) service_pool.cc:224] Handling call
0709 13:07:06.972099 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:06.972499 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:06.972528 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:06.973109 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:07:07.396367 (+423258us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:07.428968 (+ 32601us) log.cc:844] Serialized 14 byte log entry
0709 13:07:07.452592 (+ 23624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.452725 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:07.600345 (+147620us) raft_consensus.cc:1685] finished
0709 13:07:07.600370 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:07.600540 (+   170us) inbound_call.cc:177] Queueing success response
0709 13:07:07.600648 (+   108us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b18003c6fa0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
Metrics: {"mutex_wait_us":5050,"prepare.queue_time_us":29215,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":40056,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":205,"spinlock_wait_cycles":15209216,"thread_start_us":422781,"threads_started":1,"wal-append.queue_time_us":624888}
W20260709 13:07:07.640991 13840 connection.cc:256] RPC call timeout handler was delayed by -1.6590585550000014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
W20260709 13:07:07.641417 13840 connection.cc:256] RPC call timeout handler was delayed by -1.6446585550000918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
I20260709 13:07:07.635311 20732 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.644081 13915 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:07.625746 20773 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.632141 20588 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:07.637415 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:07:07.641876 13840 connection.cc:256] RPC call timeout handler was delayed by -1.4358585550000953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
I20260709 13:07:07.639524 20775 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:07.645978 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:07.646685 19983 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:07.652360 19983 raft_consensus.cc:697] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:07.644223 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.653220 14021 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:07:07.653441 13869 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: []
W20260709 13:07:07.651537 13840 connection.cc:256] RPC call timeout handler was delayed by -1.3953585550000298s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
W20260709 13:07:07.654318 13840 connection.cc:256] RPC call timeout handler was delayed by -1.3827585550000094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260709 13:07:07.654796 13840 connection.cc:256] RPC call timeout handler was delayed by -1.3467585550000649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
W20260709 13:07:07.655182 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2333585549999952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
W20260709 13:07:07.655652 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2261585550000973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2074s for a response
W20260709 13:07:07.656009 13840 connection.cc:256] RPC call timeout handler was delayed by -1.2234585550000929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 13:07:07.656390 13840 connection.cc:256] RPC call timeout handler was delayed by -1.1910585550000405s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21130000000000002s for a response
I20260709 13:07:07.625456 20704 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.655256 13869 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:07.630513 20442 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.658140 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:07.718514 13870 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:07:07.718766 20645 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:07:07.723136 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 63813.6us
I20260709 13:07:07.591826 20774 consensus_queue.cc:1059] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:07:07.731361 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:07.643450 20732 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:07.733639 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:07.728006 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 82130.2us
I20260709 13:07:07.579214 20647 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:07:07.731437 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:07.737149 13868 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:07.732120 13871 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:07.727025 14021 raft_consensus.cc:2368] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19.
I20260709 13:07:07.734283 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:07.734812 13840 connection.cc:256] RPC call timeout handler was delayed by -1.1415585550000742s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
I20260709 13:07:07.734783 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:07.740597 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:07.740893 13840 connection.cc:256] RPC call timeout handler was delayed by -1.1370585550000669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
W20260709 13:07:07.741628 13840 connection.cc:256] RPC call timeout handler was delayed by -1.1172585550000349s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2195s for a response
I20260709 13:07:07.743141 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:07.634684 20663 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.648072 20707 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:07.729108 20645 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.742702 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:07:07.743580 13840 connection.cc:256] RPC call timeout handler was delayed by -1.0994848400000592s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
I20260709 13:07:07.741381 14021 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:07.743961 13868 raft_consensus.cc:2368] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17.
I20260709 13:07:07.744894 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:07.756140 13870 raft_consensus.cc:2376] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 14. Re-sending same reply.
W20260709 13:07:07.756161 13840 connection.cc:256] RPC call timeout handler was delayed by -1.0929585549999956s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
I20260709 13:07:07.756657 13869 raft_consensus.cc:2376] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 17. Re-sending same reply.
W20260709 13:07:07.756853 13840 connection.cc:256] RPC call timeout handler was delayed by -1.0524585550000438s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2267s for a response
I20260709 13:07:07.757143 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:07.757570 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:07.757900 13868 raft_consensus.cc:2376] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 20. Re-sending same reply.
I20260709 13:07:07.758745 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:07.751618 20494 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:07.751945 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:07.665755 19983 consensus_queue.cc:237] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:07.758961 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:07.759754 13871 raft_consensus.cc:2393] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 22: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:07.760087 13868 raft_consensus.cc:2393] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 17: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:07.743028 20774 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.762224 13868 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:07.731158 20442 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:07.657508 20775 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.764631 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.773178 14021 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:07.774483 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.763896 20494 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.776029 20494 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:07.763409 13840 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:07.772735 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.785102 13869 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:07:07.793700 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:07.794569 14021 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 17: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:07.789529 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:07.799700 13870 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:07.726370 20560 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.771020 20645 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:07.801515 20180 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:07.647341 20158 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:07.802680 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.781975 20775 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:07.827237 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3269) took 3787 ms. Trace:
I20260709 13:07:07.827932 14024 rpcz_store.cc:276] 0709 13:07:04.022760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.079917 (+1057157us) service_pool.cc:224] Handling call
0709 13:07:05.331165 (+251248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.332291 (+  1126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.810230 (+2477939us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b540020cbd0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:07.810389 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":561672960}
I20260709 13:07:07.806459 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:07.813741 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.829828 14021 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:07.830945 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:07.832268 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:07.833122 14021 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:07.726639 20686 consensus_queue.cc:260] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.744266 20704 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:07.829713 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:07.837232 14024 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:07.778254 20428 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:07:07.846009 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.783473 20774 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:07.847419 14021 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:07.849239 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.811616 20180 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:07.796808 20793 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.061s
W20260709 13:07:07.842649 20803 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:07:07.861492 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:07.814286 20663 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.810220 20442 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:07:07.877354 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:07.878274 14021 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:07:07.880893 13949 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:07:07.886283 20808 log.cc:927] Time spent T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:07:07.886870 20494 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:07.888074 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 1061) took 1344 ms. Trace:
I20260709 13:07:07.889532 20494 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:07.891052 20804 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 13:07:07.888476 14551 rpcz_store.cc:276] 0709 13:07:06.543917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.546640 (+  2723us) service_pool.cc:224] Handling call
0709 13:07:06.586487 (+ 39847us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:06.588843 (+  2356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:06.588873 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:06.614927 (+ 26054us) log.cc:844] Serialized 26 byte log entry
0709 13:07:07.166037 (+551110us) spinlock_profiling.cc:243] Waited 879 us on lock 0x7b180035fce0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd30d1 00007fb2c33eea0f
0709 13:07:07.466039 (+300002us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:07.466176 (+   137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.466375 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:07.887714 (+421339us) raft_consensus.cc:1685] finished
0709 13:07:07.887743 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:07.888004 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":36895,"spinlock_wait_cycles":2021248,"thread_start_us":299467,"threads_started":1,"wal-append.queue_time_us":1082976}
I20260709 13:07:07.898298 20818 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:07.899129 20818 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:07.899647 20818 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:07.916563 20774 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:07.917263 20774 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.918228 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.919454 20704 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:07.920717 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:07.857117 20158 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:07.923755 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:07.925354 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:07.926676 14017 raft_consensus.cc:2393] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 20: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:07.928107 14017 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:07.928306 20588 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:07.933418 20560 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:07.856238 20722 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:07.934610 20722 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:07.942610 20091 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:07.944180 20801 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.223s	user 0.002s	sys 0.000s
I20260709 13:07:07.944072 20091 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:07.955894 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:07.863762 20180 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:07.917726 20774 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:07.958218 20693 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.869s	user 0.010s	sys 0.009s
I20260709 13:07:07.991803 14024 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
W20260709 13:07:08.009778 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.009502 14024 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:08.013222 14024 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:08.014741 14021 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:07.957021 20663 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:08.022392 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 248532us
I20260709 13:07:07.967974 20091 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.015126 14024 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:08.008024 20722 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.043150 20722 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:08.044023 20091 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:08.044703 20091 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:08.027632 20774 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:08.030104 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:08.030680 20663 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:08.036233 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:08.048632 13949 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:07:08.038457 20645 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.040772 14021 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: []
W20260709 13:07:08.008662 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.052325 13839 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:08.054045 13839 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:08.058219 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 198240us
I20260709 13:07:08.059089 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:08.059787 20645 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:08.061636 20091 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:08.062333 20091 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:08.062803 20816 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
I20260709 13:07:08.063964 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 807) took 3275 ms. Trace:
I20260709 13:07:08.064344 14015 rpcz_store.cc:276] 0709 13:07:04.788330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.041076 (+2252746us) service_pool.cc:224] Handling call
0709 13:07:07.059882 (+ 18806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.236768 (+176886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.063711 (+826943us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b540023bc54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:08.063902 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":382144896}
I20260709 13:07:08.059851 20652 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:08.065292 20652 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:08.067313 14015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99"
dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:41088
I20260709 13:07:08.067410 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:08.068094 14015 raft_consensus.cc:606] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:07:08.068282 13944 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:08.068529 14015 raft_consensus.cc:614] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:08.069360 14022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2724) took 3677 ms (client timeout 2222 ms). Trace:
W20260709 13:07:08.069790 14022 rpcz_store.cc:269] 0709 13:07:04.391570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.886893 (+495323us) service_pool.cc:224] Handling call
0709 13:07:08.069141 (+3182248us) spinlock_profiling.cc:243] Waited 2.52 s on lock 0x7b54000abc54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:08.069291 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5804367232}
I20260709 13:07:08.078613 20663 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:08.080569 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:08.081295 13875 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:08.082157 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:08.083067 20663 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:08.083478 20663 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.083811 20663 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:08.067641 20682 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:08.089623 20774 consensus_queue.cc:1059] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:08.091176 20628 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.091331 20774 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:08.091781 20774 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:08.091748 20628 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.092957 20774 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.095635 19983 consensus_queue.cc:1059] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:07:08.100698 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.101173 13944 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:08.102897 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:08.107586 19983 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:08.109300 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:08.113844 19983 raft_consensus.cc:2764] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:07:08.089906 20682 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:08.094302 20774 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:08.213888 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 97894.4us
I20260709 13:07:08.093874 20732 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.215737 19983 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:08.115046 20704 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:08.112699 20793 raft_consensus.cc:3055] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:08.107352 20645 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:08.214880 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:08.231773 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:08.232975 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:08.115374 20628 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:08.114730 20602 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.106s
I20260709 13:07:08.235107 13915 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:08.236016 20602 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:08.236552 20823 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:08.237067 20588 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:08.237285 20823 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.237715 20588 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:08.237843 20823 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:08.238848 20588 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.240700 20180 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.241854 20620 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:08.243101 20180 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:08.159998 20818 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.226941 20793 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:08.211928 20826 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:08.243645 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:08.245754 20818 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:08.252604 20620 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:08.253208 20620 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.253688 20620 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:08.254436 20827 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:08.254957 20827 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.255311 20827 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:08.256618 20158 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:08.257169 20158 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 24 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:08.258147 20158 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.260244 13944 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:08.261854 20628 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:08.262626 20628 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:08.263118 20628 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.268169 20652 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.284332 20494 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:07:08.287873 20494 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:08.288264 20494 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:08.289243 20494 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.297281 20494 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:08.250016 20793 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.276111 20180 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.299186 20180 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.302438 14559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1233) took 2244 ms. Trace:
I20260709 13:07:08.302870 14559 rpcz_store.cc:276] 0709 13:07:06.058323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.058722 (+   399us) service_pool.cc:224] Handling call
0709 13:07:06.059145 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.022940 (+963795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.302102 (+1279162us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b5400584b54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:08.302318 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2214591360}
W20260709 13:07:08.306551 20815 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.582s	user 0.000s	sys 0.002s
I20260709 13:07:08.306099 20775 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.309007 20775 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:08.310480 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:08.310532 13867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3427) took 2596 ms (client timeout 1884 ms). Trace:
W20260709 13:07:08.312606 13867 rpcz_store.cc:269] 0709 13:07:05.682908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.773759 (+ 90851us) service_pool.cc:224] Handling call
0709 13:07:08.279040 (+2505281us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b540026d9d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:08.279238 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1539826048}
I20260709 13:07:08.236639 20602 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:08.302035 20428 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:08.295643 20652 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:08.315810 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:08.311347 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:08.319780 13871 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:08.248432 20732 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:08.316915 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:08.249346 20773 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.248822 20704 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:08.321468 14524 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
W20260709 13:07:08.330987 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.332019 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 974) took 1322 ms. Trace:
I20260709 13:07:08.329452 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:08.339948 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.339792 13874 rpcz_store.cc:276] 0709 13:07:07.009407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.014913 (+  5506us) service_pool.cc:224] Handling call
0709 13:07:08.316702 (+1301789us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b5800003698. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:07:08.331932 (+ 15230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28355328}
I20260709 13:07:08.333520 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:08.346928 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:08.347597 13949 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:08.350479 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:08.352672 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:08.353617 14018 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:07:08.355549 14022 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:08.285588 20628 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:08.348533 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:08.272457 20645 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.366178 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:08.366592 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:08.366899 14016 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:08.367283 14023 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:07:08.324848 20748 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:08.274045 20707 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
W20260709 13:07:08.368597 13915 connection.cc:256] RPC call timeout handler was delayed by -1.1521359020000546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21800000000000003s for a response
I20260709 13:07:08.374501 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:08.375237 14018 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:07:08.380330 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:08.376750 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 55387.1us
I20260709 13:07:08.325798 20091 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:08.250608 20826 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.323132 20436 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:07:08.396286 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3298) took 1213 ms. Trace:
I20260709 13:07:08.407159 14017 rpcz_store.cc:276] 0709 13:07:07.183101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.929102 (+746001us) service_pool.cc:224] Handling call
0709 13:07:07.929487 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.929790 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:07.935003 (+  5213us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.395677 (+460674us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:08.395757 (+    80us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:08.395872 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.395979 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.396184 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13,"prepare.queue_time_us":4466,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":8391,"thread_start_us":451828,"threads_started":1}
I20260709 13:07:08.411268 13949 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 20.
I20260709 13:07:08.412897 14016 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:08.326702 20731 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:07:08.435554 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:08.443250 14017 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:07:08.395905 20662 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:08.363443 20663 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.371236 20560 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 13:07:08.358112 20773 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.378119 20682 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.478730 20826 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:08.478885 20682 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 26 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:08.444041 20180 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:08.482950 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 423893us
I20260709 13:07:08.484242 13840 leader_election.cc:400] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17.
I20260709 13:07:08.484720 13840 leader_election.cc:403] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:07:08.448572 20775 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:08.507968 20773 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:08.468477 20602 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.473433 20662 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:08.515328 20662 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.460776 20481 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.026s
W20260709 13:07:08.521152 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 467141us
I20260709 13:07:08.524111 14020 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:08.525085 13872 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:08.534427 20620 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.536307 13941 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:08.537572 20620 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:08.539611 20180 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17.
I20260709 13:07:08.542866 20645 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:08.569088 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:08.570430 13870 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
W20260709 13:07:08.547787 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.549501 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:08.571538 13872 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:08.553903 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 69795.4us
I20260709 13:07:08.471165 20704 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.556725 13874 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:07:08.586253 13874 pending_rounds.cc:85] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 13
I20260709 13:07:08.586591 13874 pending_rounds.cc:107] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:07:08.586899 13874 raft_consensus.cc:2889] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:08.557374 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.518532 20775 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.559770 13867 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:08.565917 14017 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:08.543867 20663 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:08.579011 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:08.589885 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:08.582700 13872 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:08.584183 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:08.598012 13866 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:08.557510 20481 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:07:08.589488 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.595758 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:08.603582 13870 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:08.597364 13875 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:08.599377 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:08.555315 20827 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.612200 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:08.612902 13866 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:08.616981 13914 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:08.618225 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:08.623379 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:08.604907 20826 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.609053 14556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1236) took 2220 ms. Trace:
I20260709 13:07:08.610764 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:08.649214 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 267237us
I20260709 13:07:08.652760 14556 rpcz_store.cc:276] 0709 13:07:06.388601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.388801 (+   200us) service_pool.cc:224] Handling call
0709 13:07:06.389275 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.664452 (+275177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.608709 (+1944257us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54006f18d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:08.608976 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":589135104}
W20260709 13:07:08.647568 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 268024us
W20260709 13:07:08.610404 13838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.018s	sys 0.012s
I20260709 13:07:08.602696 20481 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:08.604626 20829 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:08.625768 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 267373us
W20260709 13:07:08.656201 13838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.019s	sys 0.012s
I20260709 13:07:08.659368 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:08.663911 14555 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 20: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:08.625488 20180 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:08.646144 20732 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:08.657290 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 58030.5us
I20260709 13:07:08.669818 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:08.670305 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:08.679311 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:08.681324 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:08.683105 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:08.685220 13866 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:08.688025 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:08.689348 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:08.693197 13838 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:08.693374 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:08.654325 20091 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:08.696209 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:08.702733 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:08.696162 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:08.705355 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:08.705623 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:08.707813 14560 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 24.
I20260709 13:07:08.656904 20607 consensus_queue.cc:1059] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:07:08.656599 20826 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.716686 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:08.713886 13838 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:08.693694 20832 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:08.789379 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.721545 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:08.671315 20436 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:07:08.694247 20830 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:07:08.827855 20828 log.cc:927] Time spent T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.227s	user 0.001s	sys 0.001s
I20260709 13:07:08.695544 20831 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:08.852689 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.711756 20838 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.713562 20560 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.706001 20158 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.130s
I20260709 13:07:08.878552 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:08.731890 20829 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:08.700214 20818 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.133s
I20260709 13:07:08.705494 20091 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.889600 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:08.891166 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:08.893663 14551 raft_consensus.cc:2410] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:07:08.871917 20824 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:07:08.897447 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.908125 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:08.909430 14555 raft_consensus.cc:2410] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:07:08.916970 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:08.918201 13872 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 22: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:08.731500 20833 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.178s
W20260709 13:07:08.923336 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:08.932288 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 269275us
I20260709 13:07:08.955511 14560 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:08.885787 20829 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:08.833935 20832 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:08.819957 20481 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:08.982365 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 326407us
I20260709 13:07:08.899209 20824 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:08.988919 13870 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:08.884411 20560 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:08.975898 20823 consensus_meta.cc:287] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.737s	user 0.008s	sys 0.007s
I20260709 13:07:09.000152 20481 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:08.999928 20824 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:09.001437 20824 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:08.882040 20838 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:08.997781 20180 raft_consensus.cc:2764] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
W20260709 13:07:09.007103 13989 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.512s (SENT)
I20260709 13:07:09.007853 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:09.054955 13950 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5"
dest_uuid: "3779649036364744823fc035dac892c7"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:36586
I20260709 13:07:09.055105 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:09.055737 13950 raft_consensus.cc:606] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:08.974833 20829 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:09.076231 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:07:09.081893 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 133337us
I20260709 13:07:09.104779 19983 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:09.105337 19983 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:09.105726 19983 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:08.987995 20775 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:08.997174 20827 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:08.979693 20602 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.095s
I20260709 13:07:09.111052 14522 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:09.126513 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:09.129307 13947 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 16. Current term is 17. Ops: [15.14-16.15]
W20260709 13:07:09.112319 20722 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.065s	user 0.012s	sys 0.003s
I20260709 13:07:09.109066 20436 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:09.147899 20436 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:09.148027 20722 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:09.148422 20436 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:09.111068 20620 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:09.054785 20823 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:09.158996 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:09.161142 13870 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:09.164273 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:09.164891 13872 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:09.153231 20602 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:07:09.122602 20839 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.176251 13872 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 26.
I20260709 13:07:09.177116 19983 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:09.151232 20827 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:09.183851 13870 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 20.
W20260709 13:07:09.154253 20835 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.436s	user 0.001s	sys 0.000s
I20260709 13:07:09.193634 13946 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 13:07:09.203049 20813 log.cc:927] Time spent T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.483s	user 0.002s	sys 0.013s
I20260709 13:07:09.205084 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3308) took 2023 ms. Trace:
I20260709 13:07:09.205593 13945 rpcz_store.cc:276] 0709 13:07:07.181524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.230530 (+ 49006us) service_pool.cc:224] Handling call
0709 13:07:07.231083 (+   553us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:07.231644 (+   561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:07.231687 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:07.232514 (+   827us) log.cc:844] Serialized 52 byte log entry
0709 13:07:07.578372 (+345858us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:07.578431 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.578536 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.189699 (+1611163us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:09.204630 (+ 14931us) raft_consensus.cc:1685] finished
0709 13:07:09.204665 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.204997 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3812,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":4551424,"thread_start_us":345344,"threads_started":1,"wal-append.queue_time_us":486859}
I20260709 13:07:09.208415 20867 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:09.208843 20838 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:09.209050 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:09.209227 20867 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:09.209882 14018 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 22: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:09.210485 20838 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:09.210813 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:09.211042 20838 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:09.212217 14018 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:09.213165 20838 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:09.217155 19983 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:09.157706 20830 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:07:09.221134 20748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.011s	sys 0.021s
I20260709 13:07:09.189590 20826 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:09.219631 20494 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.922s	user 0.014s	sys 0.001s
I20260709 13:07:09.195302 20158 raft_consensus.cc:2764] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:07:09.169608 20620 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:09.228420 20836 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
I20260709 13:07:09.245288 20865 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:09.245518 14560 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:09.239638 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:07:09.238672 20748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.011s	sys 0.021s
I20260709 13:07:09.227936 20827 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:07:09.269192 20852 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:07:09.279785 20860 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:09.281143 20860 consensus_queue.cc:1243] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [LEADER]: Peer 4dd265d61fc141a48ca9d3d5c8f52921 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:07:09.284165 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:09.293103 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 180288us
W20260709 13:07:09.286258 20834 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.384s	user 0.000s	sys 0.001s
I20260709 13:07:09.299973 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:09.191159 20839 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:09.304597 14560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:09.305073 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.309108 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:09.310654 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:09.311867 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:09.266983 20748 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:09.266672 20865 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:09.340626 14561 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 22: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:09.344475 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:09.344477 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:09.351065 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.352602 19983 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:07:09.354362 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:09.332330 20682 consensus_queue.cc:1059] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
W20260709 13:07:09.376314 13840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.007s	sys 0.019s
W20260709 13:07:09.364636 20853 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
W20260709 13:07:09.359964 20837 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 13:07:09.331848 20578 raft_consensus.cc:3055] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:09.332034 20867 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:09.379245 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3323) took 1291 ms. Trace:
W20260709 13:07:09.403334 13838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.002s	sys 0.018s
W20260709 13:07:09.422094 13838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.003s	sys 0.018s
W20260709 13:07:09.422567 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 701059us
I20260709 13:07:09.421913 13948 rpcz_store.cc:276] 0709 13:07:08.087444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.087930 (+   486us) service_pool.cc:224] Handling call
0709 13:07:08.088325 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.088710 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.088741 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:08.110295 (+ 21554us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.626697 (+516402us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:08.626763 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.627919 (+  1156us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.378964 (+751045us) raft_consensus.cc:1685] finished
0709 13:07:09.378990 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.379158 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":951,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"thread_start_us":410262,"threads_started":1,"wal-append.queue_time_us":675079}
I20260709 13:07:09.363467 20560 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:09.412367 19983 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:09.421784 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:09.371368 20849 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.249s	user 0.002s	sys 0.000s
W20260709 13:07:09.413931 13840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.010s	sys 0.028s
W20260709 13:07:09.427578 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 728370us
I20260709 13:07:09.428233 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:09.422509 20840 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:07:09.362979 20832 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:09.416237 14015 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:09.429046 13840 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:09.379009 14561 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:09.424630 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 49452.9us
I20260709 13:07:09.360951 20860 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
W20260709 13:07:09.378336 20842 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.228s	user 0.001s	sys 0.001s
I20260709 13:07:09.365129 20827 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:09.449472 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2723) took 5058 ms. Trace:
I20260709 13:07:09.456004 13915 leader_election.cc:400] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17.
I20260709 13:07:09.443607 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:09.404230 20494 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:07:09.362322 20829 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:09.432180 20607 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:09.450230 13946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 3048) took 1102 ms. Trace:
I20260709 13:07:09.428088 20823 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:09.439232 13944 raft_consensus.cc:1240] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 13:07:09.440060 13871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3458) took 1633 ms (client timeout 1677 ms). Trace:
I20260709 13:07:09.445940 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:09.444849 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:09.437377 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3325) took 1127 ms. Trace:
W20260709 13:07:09.518282 13871 rpcz_store.cc:269] 0709 13:07:07.807183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.807372 (+   189us) service_pool.cc:224] Handling call
0709 13:07:09.439882 (+1632510us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54000ea954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:09.439999 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":917972224}
I20260709 13:07:09.448645 20741 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:09.519203 13941 rpcz_store.cc:276] 0709 13:07:08.309965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.310163 (+   198us) service_pool.cc:224] Handling call
0709 13:07:09.437154 (+1126991us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b5400086454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:09.437315 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":499579136}
W20260709 13:07:09.521643 20861 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:07:09.431100 14524 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:09.424907 20865 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:09.435020 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 3047) took 1087 ms. Trace:
I20260709 13:07:09.521674 20741 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:09.533917 20741 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:09.436691 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:09.538611 14016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.027s	sys 0.033s
I20260709 13:07:09.539011 13948 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
W20260709 13:07:09.539176 14016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.027s	sys 0.033s
I20260709 13:07:09.435617 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1188) took 1090 ms. Trace:
I20260709 13:07:09.540670 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:09.542306 13874 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
W20260709 13:07:09.543522 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 112531us
I20260709 13:07:09.540208 13942 rpcz_store.cc:276] 0709 13:07:08.345449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.345600 (+   151us) service_pool.cc:224] Handling call
0709 13:07:08.346114 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.346601 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.346630 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:08.347151 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.661270 (+314119us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:08.677860 (+ 16590us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.962331 (+284471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.962455 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.435394 (+472939us) raft_consensus.cc:1685] finished
0709 13:07:09.435418 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.435556 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128871,"prepare.queue_time_us":16393,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":34982,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":24685,"thread_start_us":518934,"threads_started":2,"wal-append.queue_time_us":746423}
I20260709 13:07:09.455916 14019 rpcz_store.cc:276] 0709 13:07:04.391279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.981027 (+2589748us) service_pool.cc:224] Handling call
0709 13:07:06.981473 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:07.017288 (+ 35815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.597713 (+580425us) spinlock_profiling.cc:243] Waited 35.7 ms on lock 0x7b5400293720. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33aadc0 00007fb2c3436a02 00007fb2c3432dd6 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:07:09.449109 (+1851396us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b5400293720. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c33b041d 00007fb2c3436a57 00007fb2c3432dd6 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2
0709 13:07:09.449388 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161919616}
I20260709 13:07:09.448244 20436 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:09.419380 20578 raft_consensus.cc:740] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:09.460330 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:07:09.470440 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:09.504987 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 76011.7us
I20260709 13:07:09.350666 20830 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:09.550454 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 90236.9us
W20260709 13:07:09.550844 13839 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.68:42851 timed out after 1.676s (SENT) [suppressed 30 similar messages]
I20260709 13:07:09.551010 20823 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:09.506696 14561 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:07:09.439232 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:09.515055 13946 rpcz_store.cc:276] 0709 13:07:08.347804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.369451 (+ 21647us) service_pool.cc:224] Handling call
0709 13:07:08.369901 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:09.195316 (+825415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.449907 (+254591us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54000f4e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:09.450160 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1893567360}
I20260709 13:07:09.515416 20832 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:09.503441 20428 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:09.421180 20748 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:09.561098 20876 consensus_queue.cc:1059] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:07:09.541850 20881 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:09.550585 13840 leader_election.cc:400] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19.
I20260709 13:07:09.565354 13840 leader_election.cc:403] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:07:09.566895 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:09.522617 20843 log.cc:927] Time spent T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 13:07:09.521868 20857 log.cc:927] Time spent T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:07:09.533254 13943 rpcz_store.cc:276] 0709 13:07:08.347517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.365006 (+ 17489us) service_pool.cc:224] Handling call
0709 13:07:08.365358 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:08.365733 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.365762 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:08.415370 (+ 49608us) log.cc:844] Serialized 52 byte log entry
0709 13:07:08.842594 (+427224us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:08.842666 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.842783 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.434751 (+591968us) raft_consensus.cc:1685] finished
0709 13:07:09.434775 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.434951 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20481,"raft.queue_time_us":22075,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":16952,"thread_start_us":367235,"threads_started":1,"wal-append.queue_time_us":551208}
W20260709 13:07:09.548206 20869 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 13:07:09.548669 20824 consensus_meta.cc:287] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.547s	user 0.014s	sys 0.000s
I20260709 13:07:09.560554 20856 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:09.533279 20865 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:09.563305 14524 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:09.571722 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:09.555608 20832 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:09.583674 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.586344 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:09.569077 20876 raft_consensus.cc:3055] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:09.576306 20180 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:09.594136 20602 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:07:09.595643 20829 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:09.596171 20832 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:09.576756 20823 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:07:09.576917 20607 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:07:09.603533 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3320) took 1220 ms. Trace:
I20260709 13:07:09.565125 20881 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.580476 20839 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:09.604007 14021 rpcz_store.cc:276] 0709 13:07:08.383484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.383603 (+   119us) service_pool.cc:224] Handling call
0709 13:07:08.383904 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.394745 (+ 10841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.394779 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:08.414683 (+ 19904us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.657861 (+243178us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:08.658424 (+   563us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.660760 (+  2336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.661584 (+   824us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.282501 (+620917us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b1800027be0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:07:09.603153 (+320652us) raft_consensus.cc:1685] finished
0709 13:07:09.603273 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.603453 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16962,"prepare.queue_time_us":159,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":10641,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":25513,"spinlock_wait_cycles":41890688,"thread_start_us":242490,"threads_started":1,"wal-append.queue_time_us":917237}
I20260709 13:07:09.608858 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:09.610592 20881 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:09.611984 20578 consensus_queue.cc:260] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:09.613826 20578 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:09.620639 20481 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.620s	user 0.010s	sys 0.002s
W20260709 13:07:09.621554 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 629303us
I20260709 13:07:09.621855 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3313) took 1263 ms. Trace:
W20260709 13:07:09.623103 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:09.623011 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 72197.6us
I20260709 13:07:09.564028 20428 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:09.580240 20870 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19.
I20260709 13:07:09.589099 20866 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:09.590288 20880 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:09.625523 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 618452us
I20260709 13:07:09.626400 14552 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:09.622252 14022 rpcz_store.cc:276] 0709 13:07:08.358900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.364662 (+  5762us) service_pool.cc:224] Handling call
0709 13:07:08.365032 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.365322 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:08.663719 (+298397us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.976336 (+312617us) log.cc:844] Serialized 14 byte log entry
0709 13:07:09.453155 (+476819us) spinlock_profiling.cc:243] Waited 52 ms on lock 0x7b18002ac2e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:07:09.621292 (+168137us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b18002ac2e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c0348a40 00007fb2c03434e4 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287 00007fb2c344e208
0709 13:07:09.621487 (+   195us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:09.621536 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:09.621586 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.621674 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.621795 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35818,"prepare.queue_time_us":112412,"prepare.run_cpu_time_us":134,"prepare.run_wall_time_us":3485,"raft.queue_time_us":10046,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":17122,"spinlock_wait_cycles":140489216,"thread_start_us":536430,"threads_started":2,"wal-append.queue_time_us":572467}
I20260709 13:07:09.581525 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:09.624645 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:09.635964 13945 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:09.636593 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:09.637508 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:09.638511 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:09.642012 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 78792us
I20260709 13:07:09.622725 20858 consensus_queue.cc:1059] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:07:09.646243 14558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1242) took 2612 ms. Trace:
W20260709 13:07:09.716068 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.641609 20866 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.644553 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3321) took 1260 ms. Trace:
I20260709 13:07:09.715481 14558 rpcz_store.cc:276] 0709 13:07:07.033502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.044415 (+ 10913us) service_pool.cc:224] Handling call
0709 13:07:07.064556 (+ 20141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.064767 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:07:09.587233 (+2522466us) spinlock_profiling.cc:243] Waited 5.48 ms on lock 0x7b18003f6ca0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c6e2b3c7 00007fb2c343d9a7 00007fb2c344d0bf
0709 13:07:09.587419 (+   186us) log.cc:844] Serialized 14 byte log entry
0709 13:07:09.645792 (+ 58373us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:09.645855 (+    63us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:09.645908 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.645999 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.646165 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5842,"prepare.queue_time_us":469662,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":3712,"raft.queue_time_us":5952,"spinlock_wait_cycles":12593536,"thread_start_us":544799,"threads_started":2,"wal-append.queue_time_us":52446}
I20260709 13:07:09.623128 20876 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:09.752665 14556 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 22.
W20260709 13:07:09.753525 13989 leader_election.cc:336] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.733s (SENT)
I20260709 13:07:09.754232 13989 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:09.623013 20856 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:09.628837 20793 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:09.641345 20774 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:09.717016 20698 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:07:09.758867 14524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.641552 20775 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:09.766089 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:09.726258 20866 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:09.763721 20588 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 13:07:09.760422 20885 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:07:09.776089 20428 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:09.776563 20091 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:09.780159 20873 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.364s	user 0.002s	sys 0.000s
I20260709 13:07:09.783747 20686 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:09.784982 20863 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.355s	user 0.000s	sys 0.003s
I20260709 13:07:09.768926 20662 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:09.936690 20663 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:09.802352 20826 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:09.937486 20663 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.937999 20663 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:09.812923 20832 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:09.938850 20619 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.773066 20775 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:09.939996 20619 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:09.869040 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:09.865693 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:09.871665 13839 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:09.791843 20864 raft_consensus.cc:3055] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:07:09.896812 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 143252us
I20260709 13:07:09.900880 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3318) took 1523 ms. Trace:
I20260709 13:07:10.014473 14023 rpcz_store.cc:276] 0709 13:07:08.377738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.377913 (+   175us) service_pool.cc:224] Handling call
0709 13:07:08.414909 (+ 36996us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:08.669492 (+254583us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.669525 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:08.701936 (+ 32411us) log.cc:844] Serialized 52 byte log entry
0709 13:07:09.156042 (+454106us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:09.156103 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.156198 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.781679 (+625481us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:09.900114 (+118435us) raft_consensus.cc:1685] finished
0709 13:07:09.900158 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.900617 (+   459us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8870,"raft.queue_time_us":45192,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":3607,"spinlock_wait_cycles":323258752,"thread_start_us":522606,"threads_started":2,"wal-append.queue_time_us":693003}
I20260709 13:07:09.904088 14551 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:09.904799 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:09.768124 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:10.020478 20775 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:10.021522 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 253345us
I20260709 13:07:09.919318 20091 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:10.024094 20091 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:09.920106 20436 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:09.794034 20494 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:09.772251 20588 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:10.033792 20662 consensus_queue.cc:1059] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:09.787323 20600 raft_consensus.cc:3055] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:10.035248 20662 raft_consensus.cc:3055] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:10.035602 20600 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:10.035813 20662 raft_consensus.cc:740] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:09.930559 13945 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:07:09.931978 20442 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:07:09.933456 20602 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:10.039901 20602 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:09.934744 13943 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:10.042276 13873 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:09.795795 20620 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:09.936852 20704 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:10.043068 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3436) took 2390 ms. Trace:
I20260709 13:07:09.590806 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:10.043452 13873 rpcz_store.cc:276] 0709 13:07:07.652471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.652607 (+   136us) service_pool.cc:224] Handling call
0709 13:07:10.041378 (+2388771us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b5800003698. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:07:10.041822 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.042709 (+   887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.042950 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7884032}
I20260709 13:07:09.866943 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:09.763237 20856 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.048715 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:09.804177 20855 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.509s	user 0.001s	sys 0.001s
I20260709 13:07:10.050634 20158 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:10.051914 14561 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 13:07:09.879875 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 298250us
W20260709 13:07:09.892861 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 253871us
W20260709 13:07:09.894147 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 269402us
I20260709 13:07:09.801497 20871 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:09.755714 20872 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:09.808771 20876 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.063884 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:10.065835 20876 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:10.066244 20871 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:10.067884 20871 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:09.800462 20877 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.248s	user 0.000s	sys 0.000s
W20260709 13:07:09.820380 20879 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:07:09.822624 20881 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:09.977197 13871 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:10.079502 20830 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:10.080560 20830 consensus_queue.cc:1243] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:10.081226 20748 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:07:09.990566 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 119469us
I20260709 13:07:10.083515 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:09.990821 20880 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:10.084805 20880 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:10.081497 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:10.086100 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3322) took 1701 ms. Trace:
I20260709 13:07:10.086896 13988 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:09.991285 20864 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:09.996549 20866 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.090876 20890 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:09.938015 20826 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:10.006299 20619 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:10.007488 20832 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:10.017338 13989 leader_election.cc:336] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.536s (SENT)
I20260709 13:07:09.990604 14552 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:10.037274 20600 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:10.045629 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 454715us
I20260709 13:07:10.052793 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:10.033597 20494 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:10.085728 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:10.041577 20602 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.086508 14016 rpcz_store.cc:276] 0709 13:07:08.384224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.435753 (+ 51529us) service_pool.cc:224] Handling call
0709 13:07:08.436210 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.448692 (+ 12482us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:08.680357 (+231665us) log.cc:844] Serialized 14 byte log entry
0709 13:07:09.899158 (+1218801us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:09.900628 (+  1470us) log.cc:844] Serialized 26 byte log entry
0709 13:07:09.900939 (+   311us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:09.901015 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.901163 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:10.085629 (+184466us) raft_consensus.cc:1685] finished
0709 13:07:10.085651 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:10.085881 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":383636,"prepare.queue_time_us":29153,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":433801,"raft.run_cpu_time_us":468,"raft.run_wall_time_us":470,"thread_start_us":1027940,"threads_started":2,"wal-append.queue_time_us":883636}
I20260709 13:07:10.089541 20872 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:10.098825 20872 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.099093 20494 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.100088 13989 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:10.086611 20091 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:10.098961 20826 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.101495 20620 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:10.069984 20871 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:10.105085 20826 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:10.105258 20830 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:10.106148 20442 raft_consensus.cc:2764] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:07:10.111431 20600 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:10.111543 14020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3309) took 1778 ms (client timeout 2027 ms). Trace:
W20260709 13:07:10.112174 14020 rpcz_store.cc:269] 0709 13:07:08.332983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.333178 (+   195us) service_pool.cc:224] Handling call
0709 13:07:10.111084 (+1777906us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b5400297cd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:10.111271 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":397802368}
I20260709 13:07:10.105461 20619 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:10.117517 20864 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.119007 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:10.119190 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 101973us
I20260709 13:07:10.120342 20628 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:10.120560 20866 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:10.121106 20891 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:10.121315 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 978) took 1210 ms. Trace:
I20260709 13:07:10.121814 13874 rpcz_store.cc:276] 0709 13:07:08.911247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.911460 (+   213us) service_pool.cc:224] Handling call
0709 13:07:08.968292 (+ 56832us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:09.542750 (+574458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.120721 (+577971us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b54000e9554. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:10.121099 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1313687296}
W20260709 13:07:10.124264 20693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.569s	user 0.007s	sys 0.024s
W20260709 13:07:10.124836 20693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.675s	user 0.007s	sys 0.024s
I20260709 13:07:10.125058 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1187) took 1780 ms. Trace:
I20260709 13:07:10.125597 13944 rpcz_store.cc:276] 0709 13:07:08.345032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.345224 (+   192us) service_pool.cc:224] Handling call
0709 13:07:09.438948 (+1093724us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:09.518034 (+ 79086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.124578 (+606544us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400086450. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:10.124829 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2487231872}
I20260709 13:07:10.128192 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:10.129317 20628 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:10.129758 20628 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:10.130236 20628 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:10.132068 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:10.135598 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:10.037278 20662 consensus_queue.cc:260] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.136272 20881 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:10.136556 20494 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:10.138346 20831 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.078s	user 0.011s	sys 0.011s
I20260709 13:07:10.139459 20732 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:10.144512 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 94966.6us
I20260709 13:07:10.119275 20864 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:10.120342 13989 leader_election.cc:336] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.575s (SENT)
I20260709 13:07:10.162366 13989 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:10.163451 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:10.164528 13944 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:10.164839 20891 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:10.162518 20830 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:10.162424 20882 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.373s	user 0.000s	sys 0.003s
I20260709 13:07:10.166874 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 819) took 1253 ms. Trace:
I20260709 13:07:10.167330 14015 rpcz_store.cc:276] 0709 13:07:08.912934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.913086 (+   152us) service_pool.cc:224] Handling call
0709 13:07:08.921162 (+  8076us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:09.429952 (+508790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.166327 (+736375us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b5400020dd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:10.166635 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1137996160}
W20260709 13:07:10.164559 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 44330.5us
W20260709 13:07:10.169792 13989 leader_election.cc:336] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.609s (SENT)
W20260709 13:07:10.176299 20741 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.008s
I20260709 13:07:10.176108 14556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 1063) took 1486 ms. Trace:
I20260709 13:07:10.178427 20741 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.184116 13988 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:10.178989 14556 rpcz_store.cc:276] 0709 13:07:08.689424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.689563 (+   139us) service_pool.cc:224] Handling call
0709 13:07:10.175669 (+1486106us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b540040a6d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:10.175866 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2103523712}
W20260709 13:07:10.182548 20831 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.284s	user 0.011s	sys 0.012s
I20260709 13:07:09.721535 14024 rpcz_store.cc:276] 0709 13:07:08.383820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.386907 (+  3087us) service_pool.cc:224] Handling call
0709 13:07:08.396580 (+  9673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.396881 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:08.692796 (+295915us) log.cc:844] Serialized 14 byte log entry
0709 13:07:09.643974 (+951178us) log.cc:844] Serialized 14 byte log entry
0709 13:07:09.644177 (+   203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:09.644223 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:09.644279 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.644355 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.644491 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":527026,"prepare.queue_time_us":151650,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":78,"raft.queue_time_us":634234,"spinlock_wait_cycles":14440832,"thread_start_us":695089,"threads_started":2,"wal-append.queue_time_us":568886}
I20260709 13:07:10.171460 20693 consensus_queue.cc:1059] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:07:10.202699 20830 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:10.202888 20698 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:10.203590 20698 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:10.206241 20620 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.206911 20647 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.207677 20647 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:10.202592 13989 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.690s (SENT)
I20260709 13:07:10.206988 20620 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.212033 20891 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:10.202477 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:10.213270 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:10.210016 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:10.212087 20732 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:10.227064 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 70762.8us
W20260709 13:07:10.226229 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 144989us
W20260709 13:07:10.236511 13989 leader_election.cc:336] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.775s (SENT)
I20260709 13:07:10.237740 20732 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:10.187464 20831 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:10.242513 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:10.238783 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 103472us
I20260709 13:07:10.243439 13946 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:07:10.241024 14559 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:10.254537 20657 consensus_queue.cc:1059] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:07:10.254441 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:10.255705 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:10.256716 20830 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:10.257174 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:10.255832 20657 raft_consensus.cc:3055] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:10.257647 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:10.237996 13989 leader_election.cc:336] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.784s (SENT)
I20260709 13:07:10.232996 20741 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:10.257843 20867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.008s	sys 0.020s
I20260709 13:07:10.270181 13870 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:10.258388 20748 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.245819 20891 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:10.261765 20657 raft_consensus.cc:740] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:07:10.279661 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 194048us
I20260709 13:07:10.240590 20663 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:10.274264 20867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.009s	sys 0.020s
I20260709 13:07:10.281585 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:10.284453 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:10.281276 20748 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:10.294472 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 56540.5us
I20260709 13:07:10.295486 13839 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:10.297281 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:10.298290 20866 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:10.298938 20866 raft_consensus.cc:697] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:10.300149 20866 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:10.295506 13989 leader_election.cc:336] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.820s (SENT)
I20260709 13:07:10.302306 20832 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:10.302410 13989 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:10.298959 20881 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.005s
I20260709 13:07:10.303694 20698 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:10.305634 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 48144.7us
I20260709 13:07:10.306834 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:10.309697 14554 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 22. Current term is 23. Ops: []
W20260709 13:07:10.309725 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.311424 20831 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.283463 20830 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:10.321162 20844 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:10.326195 20663 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 25 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:10.338644 20881 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:10.339744 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 62881.6us
I20260709 13:07:10.340787 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:10.342289 20793 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:10.342913 20793 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:10.344178 20793 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.344741 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 896) took 1041 ms. Trace:
I20260709 13:07:10.345602 20436 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:10.346246 20436 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:10.346726 14015 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:10.346514 20657 consensus_queue.cc:260] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.348035 20657 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:10.347584 20436 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.345188 14551 rpcz_store.cc:276] 0709 13:07:09.303566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.303745 (+   179us) service_pool.cc:224] Handling call
0709 13:07:09.304100 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.030237 (+726137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.330003 (+299766us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b54004f1854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:10.344646 (+ 14643us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1378013440}
W20260709 13:07:10.349535 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 54115.7us
I20260709 13:07:10.350028 20578 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:10.350255 14019 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:07:10.350544 13989 leader_election.cc:336] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.842s (SENT)
I20260709 13:07:10.351187 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:10.352705 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:10.319463 20647 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:10.353816 13949 raft_consensus.cc:2410] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:07:10.356961 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:10.357645 14552 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:10.358052 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:10.354300 20851 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.359156 20851 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.359848 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.360523 13874 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:10.362264 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.361851 20851 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:10.363166 20903 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:07:10.363353 20856 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:07:10.336901 20787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.007s	sys 0.010s
W20260709 13:07:10.364375 20787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.007s	sys 0.010s
I20260709 13:07:10.366360 20620 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:10.366858 20619 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:10.367368 20686 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:10.367477 20663 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:10.368651 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:10.369479 13944 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:10.371049 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:10.372104 20830 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:10.372790 20830 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 20 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:10.374619 20830 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:10.354096 13989 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.895s (SENT)
I20260709 13:07:10.377110 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:10.354764 20647 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:10.378957 20647 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:10.355455 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:10.380448 14020 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:07:10.355799 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:10.337913 20662 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:10.382436 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 85341.1us
I20260709 13:07:10.383597 20663 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.384384 20663 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:10.385594 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.357949 14553 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:10.387502 20663 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:10.388430 20663 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.389348 20663 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.392773 13950 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:10.387611 14559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59646 (request call id 1247) took 1764 ms. Trace:
I20260709 13:07:10.394539 14559 rpcz_store.cc:276] 0709 13:07:08.622841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.623066 (+   225us) service_pool.cc:224] Handling call
0709 13:07:10.387361 (+1764295us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540018c9d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:10.387539 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3717939072}
W20260709 13:07:10.396149 13989 leader_election.cc:336] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 1.815s (SENT)
I20260709 13:07:10.355944 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:10.396692 13989 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:10.398731 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 92062.4us
I20260709 13:07:10.416726 13942 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:10.417297 20698 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:10.420043 20663 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:10.419193 14552 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 25.
I20260709 13:07:10.421603 14559 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
W20260709 13:07:10.423925 13989 leader_election.cc:336] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.105s (SENT)
I20260709 13:07:10.424556 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:10.427485 13989 leader_election.cc:336] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.602s (SENT)
I20260709 13:07:10.427927 13989 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:10.356846 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:10.434861 14021 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:07:10.353788 20628 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.437199 20619 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:10.437973 20823 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.438206 20619 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:10.438644 20823 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.439145 20619 consensus_queue.cc:1059] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:10.440188 20851 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:10.440248 20619 raft_consensus.cc:3055] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:10.440709 20619 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:10.398032 14551 raft_consensus.cc:2376] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 22. Re-sending same reply.
W20260709 13:07:10.398236 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 94250.7us
I20260709 13:07:10.446961 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:10.356257 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:10.448073 14017 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 25: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:10.448566 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.458359 13874 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 23.
I20260709 13:07:10.460289 13950 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 19. Current term is 20. Ops: []
W20260709 13:07:10.461903 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.464499 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:10.438004 20628 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:10.463860 20647 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:10.474709 20647 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:10.474275 20619 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.476181 20619 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:10.476578 20865 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.477434 20865 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.478259 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:10.480410 20826 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.481027 20826 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.481555 13840 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:10.482339 13944 raft_consensus.cc:1240] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:10.482373 20876 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:10.483080 20876 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 22 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:10.483549 20826 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:10.483693 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:10.484105 20876 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.486514 13871 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:07:10.486867 13873 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:10.488972 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.489621 13871 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:10.490903 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.491709 13871 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:10.492772 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.493646 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.488193 20865 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:10.494510 13867 raft_consensus.cc:2368] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 22. Current term is 23.
I20260709 13:07:10.494783 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:10.494855 20647 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.498333 13869 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:07:10.498665 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:07:10.493660 13871 raft_consensus.cc:2368] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 25. Current term is 26.
I20260709 13:07:10.500651 13988 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:10.448323 20823 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:10.472097 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:10.489666 13868 raft_consensus.cc:2376] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 19. Re-sending same reply.
W20260709 13:07:10.527448 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.488302 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.535092 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:10.535277 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.536966 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:10.539263 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:10.541011 13873 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:07:10.542737 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 61280.9us
I20260709 13:07:10.546310 20865 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:10.549453 20851 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:10.550266 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 85838.1us
I20260709 13:07:10.552043 20663 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:10.553053 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:10.553998 14021 raft_consensus.cc:2393] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:10.554177 13866 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:10.556236 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.557051 13866 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:10.559242 14556 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 15. (term mismatch)
I20260709 13:07:10.559643 14556 pending_rounds.cc:85] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting all ops after (but not including) 14
I20260709 13:07:10.559861 14556 pending_rounds.cc:107] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting uncommitted NO_OP operation due to leader change: 16.15
I20260709 13:07:10.560159 14556 raft_consensus.cc:2889] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:10.570766 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 98783.3us
I20260709 13:07:10.572453 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:10.573432 20657 raft_consensus.cc:2764] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:07:10.516259 20693 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.073s
I20260709 13:07:10.574774 20871 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:10.575047 20905 log.cc:927] Time spent T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.220s	user 0.004s	sys 0.000s
I20260709 13:07:10.575639 20871 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:10.576000 20871 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:10.576833 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:10.577051 20871 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.580075 20871 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:10.557838 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.603130 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:10.605216 14020 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:10.607419 13941 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:10.609290 13875 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:10.610275 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.616339 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.619602 14020 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:10.620621 13866 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:10.623199 13868 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:10.629556 14020 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:07:10.632802 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.591001 20851 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:10.637811 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 181650us
W20260709 13:07:10.640517 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 213042us
I20260709 13:07:10.641672 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.627622 20619 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:10.625960 20867 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:10.616395 20686 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:07:10.646569 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.651343 14019 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:07:10.653195 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.650272 20619 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:10.625834 20657 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:07:10.657239 13875 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:10.663852 13941 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:07:10.665287 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:10.665189 20619 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:10.666196 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.649551 20894 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.666137 14022 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:10.669200 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.636759 20866 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:07:10.664348 13941 pending_rounds.cc:85] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 13
I20260709 13:07:10.670543 20773 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:10.675961 13941 pending_rounds.cc:107] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:07:10.695359 13941 raft_consensus.cc:2889] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:10.644126 20578 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 13:07:10.681352 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.686203 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:10.691493 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.681118 20851 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:10.700718 13870 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:10.658418 20867 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:10.702522 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.704464 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:10.706853 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.707671 13875 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:10.707150 20867 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.708395 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.708772 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.709424 13874 raft_consensus.cc:2410] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:07:10.709877 13875 raft_consensus.cc:2410] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:07:10.707600 13870 raft_consensus.cc:2376] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17. Re-sending same reply.
I20260709 13:07:10.710780 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.710889 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.711544 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.711846 13870 raft_consensus.cc:2393] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:10.712030 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.712873 13875 raft_consensus.cc:2368] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18.
I20260709 13:07:10.712761 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.713402 20584 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:10.713486 13870 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:10.713832 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.714195 20584 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.714807 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.715183 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.715620 13870 raft_consensus.cc:2410] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:07:10.716001 13869 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:10.718915 20787 raft_consensus.cc:3055] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:10.719309 20787 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:10.720383 20787 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.722319 13866 raft_consensus.cc:2376] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19. Re-sending same reply.
I20260709 13:07:10.724004 20787 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:10.712869 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.725049 13871 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
W20260709 13:07:10.726450 20904 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.068s	user 0.000s	sys 0.002s
W20260709 13:07:10.729583 13989 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.456s (SENT)
I20260709 13:07:10.730124 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:10.731019 20844 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:10.731830 20894 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.735419 20588 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.738051 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:10.738834 13868 raft_consensus.cc:2410] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:07:10.747583 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.748821 13867 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 20: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:10.713832 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.753693 13869 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:07:10.756386 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:10.757058 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 54978.2us
I20260709 13:07:10.755013 20787 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.756317 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.751400 13875 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:10.762121 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:10.766853 14020 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:10.765030 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.790961 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:10.794660 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:10.801999 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.803036 13874 raft_consensus.cc:2410] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:07:10.804914 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:10.805608 14015 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:10.805721 20748 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:10.801424 13871 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:07:10.807886 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.808634 13875 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:07:10.813203 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:10.775578 20894 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:10.815500 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.812633 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:10.778508 20787 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.797919 20872 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:10.822443 13875 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 13:07:10.832372 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.833324 13874 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 20: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:10.834201 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.835080 13874 raft_consensus.cc:2410] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:07:10.836496 13867 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:07:10.837587 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.839892 13869 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 22. Current term is 23. Ops: [21.14-22.15]
I20260709 13:07:10.840960 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.841706 13869 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:10.844729 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.845887 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.849385 13867 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 20: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:10.850580 13871 raft_consensus.cc:2410] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:07:10.851274 14022 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:10.853648 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.828783 20872 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.875068 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.880671 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.805783 20892 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.123s
I20260709 13:07:10.879568 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.878821 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.886670 13869 raft_consensus.cc:2368] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18.
I20260709 13:07:10.849679 20698 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:10.890466 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.893149 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.891168 13871 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
W20260709 13:07:10.895628 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 96945.3us
I20260709 13:07:10.896235 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:10.899106 20787 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:10.900300 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.901914 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:10.902608 20607 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:10.903271 20607 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:10.903644 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:10.906502 20602 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:10.905617 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:10.908340 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.909211 13875 raft_consensus.cc:2410] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:07:10.892458 13943 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:07:10.906195 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:10.883622 20872 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:10.919011 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.920190 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.914440 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.914933 20602 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:10.915244 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:10.923765 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.924418 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.925002 13875 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:10.931191 13867 raft_consensus.cc:2410] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:07:10.931977 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.906246 20823 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.915225 20494 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:10.933266 14018 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:10.933249 14559 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 22: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:10.927716 13869 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:10.911856 20607 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:10.941435 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.952365 13871 raft_consensus.cc:2410] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:07:10.953459 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.955265 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:10.963274 20917 log.cc:927] Time spent T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 13:07:10.964782 20916 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:07:10.955960 20823 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:10.925580 20600 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:07:10.936579 20494 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:10.974928 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.975800 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.976466 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.968251 20823 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:10.980104 14016 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:10.982431 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.983758 13874 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:10.983865 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:10.986135 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 89397.1us
I20260709 13:07:10.986945 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:10.987769 13873 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:10.979523 20494 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:10.985639 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.995940 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:10.998314 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.005944 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:11.007596 14016 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
W20260709 13:07:11.008809 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 103328us
I20260709 13:07:11.017794 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.020113 20600 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:11.021929 20600 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:11.022816 13868 raft_consensus.cc:1275] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:07:11.025832 20922 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:07:11.018548 13869 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:11.063095 13874 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:07:11.000923 20864 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 13:07:11.070730 13873 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:11.076184 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.077267 13869 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:07:11.078583 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.079450 13873 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:07:11.088112 13875 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:11.091238 13868 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:07:11.093888 20864 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:11.094414 20864 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:11.106981 13875 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:07:11.107446 13875 pending_rounds.cc:85] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 9
I20260709 13:07:11.107653 13875 pending_rounds.cc:107] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:07:11.107892 13875 raft_consensus.cc:2889] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:11.095428 20864 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:11.092080 13874 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
W20260709 13:07:11.109436 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.112319 13873 raft_consensus.cc:1240] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:07:11.115705 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.116925 13870 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:07:11.128257 20929 log.cc:927] Time spent T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:07:11.131016 13875 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:11.134984 20864 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:11.146231 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.150679 13870 raft_consensus.cc:2410] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:07:11.164561 13989 leader_election.cc:336] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Timed out: RequestConsensusVote RPC to 127.12.109.65:43607 timed out after 2.913s (SENT)
I20260709 13:07:11.169248 20830 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.173768 20928 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:11.175631 14019 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:11.173848 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:11.170830 13941 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:07:11.178150 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:11.184508 20657 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:11.184971 20657 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.185354 20657 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:11.186152 14017 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 13.
I20260709 13:07:11.190933 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:11.191733 14024 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:07:11.182447 14016 raft_consensus.cc:1240] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:11.203002 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.249629 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:11.253695 14556 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:11.253847 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:11.196287 20830 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:11.286376 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:11.287310 14551 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:07:11.290122 14554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59646 (request call id 1261) took 1052 ms. Trace:
I20260709 13:07:11.293785 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:07:11.317176 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.306051 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.331255 13873 raft_consensus.cc:2393] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:11.331341 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:11.325975 14556 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:11.332280 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.327474 14554 rpcz_store.cc:276] 0709 13:07:10.237384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.237762 (+   378us) service_pool.cc:224] Handling call
0709 13:07:10.238106 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.310221 (+ 72115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.289689 (+979468us) spinlock_profiling.cc:243] Waited 68.9 ms on lock 0x7b54003d72d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:11.289972 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158538624}
W20260709 13:07:11.343283 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 54654.1us
I20260709 13:07:11.265247 20867 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:11.348287 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 119843us
I20260709 13:07:11.352169 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:11.353633 20932 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:07:11.321120 20937 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 13:07:11.302881 20941 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.104s	user 0.001s	sys 0.002s
I20260709 13:07:11.372095 14561 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:11.361902 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:11.314922 20938 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:07:11.366956 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:11.372560 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42526 (request call id 986) took 1494 ms. Trace:
I20260709 13:07:11.371557 13988 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:07:11.381836 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.317169 20830 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:11.380452 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:11.278555 20584 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:11.345402 20442 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.416222 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:11.419754 14021 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:11.379212 13872 rpcz_store.cc:276] 0709 13:07:09.864496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.868316 (+  3820us) service_pool.cc:224] Handling call
0709 13:07:11.358411 (+1490095us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b54000ea454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:11.358551 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1572807552}
I20260709 13:07:11.412106 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:11.438606 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.276556 20940 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.144s
I20260709 13:07:11.460322 13869 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:07:11.449041 20600 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:07:11.461972 13869 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:07:11.375203 20867 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:11.413720 20787 consensus_queue.cc:1059] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:07:11.408066 20932 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
W20260709 13:07:11.502650 20951 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:07:11.509711 20662 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:07:11.511850 20662 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:07:11.513069 20872 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.000s
W20260709 13:07:11.513150 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:11.514380 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.519310 20588 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:11.411767 20945 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:07:11.541394 13870 raft_consensus.cc:2410] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
W20260709 13:07:11.524606 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 113588us
I20260709 13:07:11.442459 20442 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:11.513727 20872 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:11.567876 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:11.573171 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 201708us
W20260709 13:07:11.575726 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.582330 20793 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:11.585337 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:11.626559 20662 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.570989 20787 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:07:11.647410 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62309.1us
I20260709 13:07:11.647854 13950 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:11.648083 14559 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:11.642252 20902 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:11.650897 20946 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:07:11.650612 20902 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:11.651182 20442 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:11.653223 20902 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:11.653764 20935 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.377s	user 0.002s	sys 0.000s
I20260709 13:07:11.655652 20773 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:07:11.656030 20942 log.cc:927] Time spent T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.205s	user 0.002s	sys 0.000s
I20260709 13:07:11.633986 20823 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:11.649785 14021 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:11.635119 20787 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:11.648299 13950 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:11.666692 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:11.674849 13950 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:11.676131 13950 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:11.675369 20918 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:07:11.665693 20494 consensus_meta.cc:287] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.661s	user 0.010s	sys 0.005s
I20260709 13:07:11.692111 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:11.698158 20864 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:11.699173 20864 consensus_queue.cc:1243] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Peer 4dd265d61fc141a48ca9d3d5c8f52921 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:07:11.699700 20864 raft_consensus.cc:3055] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:11.700030 20864 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:11.700820 20864 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:11.702361 20600 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:11.702981 20600 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:11.703194 20872 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:11.703472 20600 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:11.704367 20787 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:11.705646 20787 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:11.666198 20647 raft_consensus.cc:2764] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:07:11.708813 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:11.713531 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:11.715096 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 306553us
W20260709 13:07:11.717687 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.715786 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:11.661756 20657 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:11.707082 20864 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:11.719313 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:11.720822 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:11.649287 20662 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:11.722247 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.724036 13944 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:07:11.712340 20823 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:11.720477 20902 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.748826 13950 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:07:11.752650 14554 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:07:11.748660 14522 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:11.724826 20894 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.722199 20436 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:11.706794 20494 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:11.767738 20965 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:11.783196 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 65290.3us
W20260709 13:07:11.784528 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.790721 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:11.764426 20436 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.769819 20686 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:07:11.824381 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:11.830672 20947 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:07:11.835186 20657 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:11.835517 20823 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:11.836055 20823 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:11.819152 20686 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.838543 20686 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:11.775249 20856 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.067s
I20260709 13:07:11.842561 13946 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 26 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:07:11.845064 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:11.846148 20602 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.846518 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:11.846668 20602 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:11.847436 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:11.848119 14021 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:11.848361 14020 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:11.849985 14015 raft_consensus.cc:1217] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 14.11->[16.12-16.12]   Dedup: 16.12->[]
I20260709 13:07:11.850657 14015 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:07:11.845872 14017 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:11.852821 20824 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:11.847368 14022 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:11.855479 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3491) took 1028 ms. Trace:
I20260709 13:07:11.855983 13871 rpcz_store.cc:276] 0709 13:07:10.827368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.999573 (+172205us) service_pool.cc:224] Handling call
0709 13:07:11.010185 (+ 10612us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:11.168303 (+158118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:11.168339 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:11.205763 (+ 37424us) log.cc:844] Serialized 52 byte log entry
0709 13:07:11.574862 (+369099us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:11.574907 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.574992 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.855193 (+280201us) raft_consensus.cc:1685] finished
0709 13:07:11.855227 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.855404 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":50957,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":3415,"thread_start_us":491889,"threads_started":2,"wal-append.queue_time_us":535468}
I20260709 13:07:11.770248 20894 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:11.861100 20894 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:11.863117 20662 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:11.863512 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:11.864977 13989 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:11.852629 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:11.868366 14018 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:11.875969 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:11.876755 13950 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:11.879006 14022 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:11.764079 20902 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:11.887697 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:11.888520 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:11.892783 14023 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:11.889459 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:11.893561 14016 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:07:11.818984 20436 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:11.899544 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.900826 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:11.897194 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:11.904992 14019 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
W20260709 13:07:11.901422 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 142525us
I20260709 13:07:11.907115 13838 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:11.908449 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:11.909789 13946 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:11.910000 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.911944 13874 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:11.880044 20647 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:11.916906 13869 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:11.890313 20965 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:11.887095 20824 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:11.900096 20894 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.934083 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:11.943516 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 78611.4us
I20260709 13:07:11.947840 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:11.949378 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.896718 20662 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 13:07:11.953399 13874 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:07:11.922827 20876 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:11.957055 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 186213us
I20260709 13:07:11.971309 13873 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:11.958310 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:11.919412 20647 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:11.960187 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:11.960343 14021 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:11.958177 13947 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 21.
I20260709 13:07:11.961930 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:11.962894 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.964089 13875 raft_consensus.cc:2410] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:07:11.943854 20442 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:11.972582 14019 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:11.980098 13869 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:11.985175 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.986742 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:11.983738 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:11.998521 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:11.999275 14019 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:11.977418 20647 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:12.006172 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:12.006628 14024 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:12.006173 14018 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:07:12.009575 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:12.009900 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:12.010512 13869 raft_consensus.cc:2410] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:07:12.012343 20976 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:07:12.012434 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:12.013399 13869 raft_consensus.cc:2410] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:07:12.013841 20965 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:11.989053 20442 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:12.014703 13869 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:07:12.015305 20442 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:12.011397 14019 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:12.017695 13870 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:07:11.987629 20962 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:11.975795 20876 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.019304 13868 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 18. (term mismatch)
I20260709 13:07:12.019764 13868 pending_rounds.cc:85] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 17
I20260709 13:07:12.019982 13868 pending_rounds.cc:107] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 24.18
I20260709 13:07:12.020220 13868 raft_consensus.cc:2889] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:11.952117 20894 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:12.013170 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:12.018550 20824 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:12.022478 20876 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:12.031596 14019 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:12.006639 20602 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:12.028398 20931 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:07:12.043067 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.044557 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 92863us
I20260709 13:07:11.986383 20961 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:07:12.030349 20494 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:11.991097 20979 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:07:12.056809 13989 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:12.055011 13869 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:12.011113 20600 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:12.060439 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.070381 13915 leader_election.cc:400] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 22. Current term is 23.
I20260709 13:07:12.043632 20962 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:12.072762 14554 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:07:12.074133 20600 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:12.074354 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:12.026978 20607 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:07:12.084375 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:12.085141 14018 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:07:12.085769 14022 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:12.086351 20902 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:12.086511 14016 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:07:12.087651 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:12.088397 14018 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:12.092245 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:12.092245 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:12.097327 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:12.100948 14018 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:07:12.101414 14559 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:12.054286 20945 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:12.098729 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:07:12.103400 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.102018 14015 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:12.068233 20979 consensus_queue.cc:1243] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:07:12.101148 13840 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:12.108595 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:12.110862 13943 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:12.113688 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:12.113904 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:12.117776 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:12.118435 14016 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:12.097353 20619 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:12.121232 13987 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:12.126434 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:12.101420 20892 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.130949 14015 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:07:12.131831 14022 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:07:12.139106 14019 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:07:12.139442 14021 raft_consensus.cc:1275] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:12.138675 13945 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 14.
I20260709 13:07:12.151607 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:12.152347 14016 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:12.162335 14024 raft_consensus.cc:2410] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 13:07:12.166865 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.122877 20864 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:12.119313 20977 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:12.105192 20494 consensus_queue.cc:1059] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:07:12.107396 20945 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:12.180384 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 49421.2us
I20260709 13:07:12.193771 14022 raft_consensus.cc:1240] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:12.167749 20965 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:12.197057 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 96401.9us
W20260709 13:07:12.200286 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 104604us
I20260709 13:07:12.172013 20864 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:12.177661 20977 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:12.223322 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 171902us
I20260709 13:07:12.224562 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:12.218282 20864 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:12.211483 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:07:12.252923 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.256161 20984 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:07:12.219328 20866 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:12.258240 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 46871.1us
I20260709 13:07:12.258713 20866 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:12.259711 20943 consensus_queue.cc:1059] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:12.233784 20965 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:12.260550 20943 consensus_queue.cc:1243] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:07:12.261063 20943 raft_consensus.cc:3055] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:12.261381 20943 raft_consensus.cc:740] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:12.155140 20892 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:12.262356 20943 consensus_queue.cc:260] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:12.263723 20943 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:12.257946 20830 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:12.264633 20830 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.265058 20830 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:12.266892 20945 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:12.221256 20977 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:12.259944 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:12.207551 13915 leader_election.cc:400] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 25 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 25. Current term is 26.
I20260709 13:07:12.274978 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:12.283072 20974 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
W20260709 13:07:12.285321 20972 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.225s	user 0.000s	sys 0.000s
I20260709 13:07:12.290623 20823 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:12.292243 20944 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.796s	user 0.006s	sys 0.000s
I20260709 13:07:12.293123 20823 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:12.311739 20867 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:12.316233 14015 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:07:12.320016 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 120435us
I20260709 13:07:12.321502 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:12.323356 20602 raft_consensus.cc:3055] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:12.323870 20602 raft_consensus.cc:740] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:12.323561 20980 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:12.324910 20602 consensus_queue.cc:260] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:12.326350 20979 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:12.328490 20588 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:12.329504 20961 raft_consensus.cc:3055] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:12.329890 20961 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:12.330771 20961 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:12.331526 20973 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.276s	user 0.001s	sys 0.002s
I20260709 13:07:12.332206 20961 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:12.332824 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:12.337600 20793 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:07:12.338174 20991 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.095s	user 0.002s	sys 0.000s
W20260709 13:07:12.340942 20978 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:07:12.272509 20894 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:12.342458 20892 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:12.342754 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.343739 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.344147 20966 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.386s	user 0.000s	sys 0.001s
I20260709 13:07:12.344709 20619 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.345366 20584 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:12.345265 20619 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:12.342558 13943 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:07:12.346575 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 1002) took 1145 ms. Trace:
I20260709 13:07:12.345950 20584 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:12.347867 20830 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:12.348289 20619 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:12.348783 20584 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:12.350302 13944 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:12.347105 13872 rpcz_store.cc:276] 0709 13:07:11.201399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.455728 (+254329us) service_pool.cc:224] Handling call
0709 13:07:11.496485 (+ 40757us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:11.503653 (+  7168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:11.503697 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:11.504280 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.739945 (+235665us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:11.740008 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.740107 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:12.343408 (+603301us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:12.346214 (+  2806us) raft_consensus.cc:1685] finished
0709 13:07:12.346248 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.346502 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20648,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"spinlock_wait_cycles":23769344,"thread_start_us":234924,"threads_started":1,"wal-append.queue_time_us":375405}
I20260709 13:07:12.356561 14019 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:07:12.363373 20982 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.274s	user 0.002s	sys 0.000s
W20260709 13:07:12.364187 20975 log.cc:927] Time spent T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.413s	user 0.003s	sys 0.000s
I20260709 13:07:12.366590 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 1079) took 1080 ms. Trace:
I20260709 13:07:12.366990 14552 rpcz_store.cc:276] 0709 13:07:11.286101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.286479 (+   378us) service_pool.cc:224] Handling call
0709 13:07:11.286832 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:11.566811 (+279979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:11.566864 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:11.569399 (+  2535us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.780803 (+211404us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:11.780848 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.023732 (+242884us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54003372d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:12.023809 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:12.363201 (+339392us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:12.366330 (+  3129us) raft_consensus.cc:1685] finished
0709 13:07:12.366354 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.366523 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":322,"raft.queue_time_us":519,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":10894,"spinlock_wait_cycles":621119744,"thread_start_us":147541,"threads_started":1,"wal-append.queue_time_us":323964}
W20260709 13:07:12.270826 20971 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.230s	user 0.000s	sys 0.002s
I20260709 13:07:12.403184 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:12.405218 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:12.415449 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 47889.3us
I20260709 13:07:12.427150 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:12.427976 13872 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
W20260709 13:07:12.433228 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 309190us
I20260709 13:07:12.380486 20962 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:12.439600 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:12.346367 20773 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.088s
W20260709 13:07:12.447909 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 188065us
I20260709 13:07:12.385154 20793 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:12.451297 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:12.437091 14522 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:12.450728 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:12.420074 20940 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:12.433418 20602 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:12.450126 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:12.440326 20965 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:12.486723 20986 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:07:12.415000 20894 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 13:07:12.486609 14524 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:12.489480 20992 log.cc:927] Time spent T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.160s	user 0.002s	sys 0.000s
I20260709 13:07:12.489754 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:12.490975 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:12.491631 13941 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
I20260709 13:07:12.491863 20647 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:12.492749 20902 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.493292 20902 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:12.494516 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:12.495599 20902 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:12.491639 20693 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:12.496491 20693 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.496843 20693 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:12.502714 20988 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:12.503294 20988 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.503670 20988 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:12.507267 20693 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:12.470703 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:12.476060 20983 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.277s	user 0.002s	sys 0.000s
I20260709 13:07:12.478521 20955 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:07:12.481572 20987 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:07:12.484169 20793 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:12.511813 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.526571 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:12.528028 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:12.446738 20588 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
W20260709 13:07:12.532027 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 159214us
W20260709 13:07:12.558691 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 139247us
W20260709 13:07:12.565860 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.567551 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.573484 14023 consensus_meta.cc:287] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.654s	user 0.014s	sys 0.000s
I20260709 13:07:12.574715 14023 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
I20260709 13:07:12.576112 14017 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18. Ops: []
W20260709 13:07:12.579239 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 154523us
W20260709 13:07:12.573321 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 186203us
W20260709 13:07:12.582492 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.563825 20787 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.858s	user 0.017s	sys 0.001s
I20260709 13:07:12.555357 20494 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:12.599151 14561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 912) took 1042 ms. Trace:
W20260709 13:07:12.597721 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 140298us
W20260709 13:07:12.602005 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 68176.8us
I20260709 13:07:12.603122 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:12.593017 20985 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.142s	user 0.005s	sys 0.000s
W20260709 13:07:12.603034 20686 consensus_meta.cc:287] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.761s	user 0.013s	sys 0.001s
I20260709 13:07:12.599603 14561 rpcz_store.cc:276] 0709 13:07:11.556326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.556465 (+   139us) service_pool.cc:224] Handling call
0709 13:07:11.569644 (+ 13179us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:11.653132 (+ 83488us) spinlock_profiling.cc:243] Waited 9.77 ms on lock 0x7b18003c2b00. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9
0709 13:07:11.739091 (+ 85959us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b18003c2b00. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c342bbfb 00007fb2c343664b 00007fb2c3432d8f
0709 13:07:11.890475 (+151384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:11.890519 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:11.919940 (+ 29421us) log.cc:844] Serialized 26 byte log entry
0709 13:07:12.027109 (+107169us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:12.027153 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.027243 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:12.365641 (+338398us) spinlock_profiling.cc:243] Waited 4.7 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:12.598894 (+233253us) raft_consensus.cc:1685] finished
0709 13:07:12.598920 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.599083 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3136,"raft.queue_time_us":57216,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":3909,"spinlock_wait_cycles":43144576,"thread_start_us":217196,"threads_started":2,"wal-append.queue_time_us":139755}
I20260709 13:07:12.609472 13840 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:12.574271 20823 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:07:12.467727 20962 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:12.589753 20866 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:12.647439 20979 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:07:12.647852 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 44860.1us
I20260709 13:07:12.681782 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:12.609061 20647 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:12.707846 20979 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:12.744412 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 166414us
I20260709 13:07:12.621429 20686 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:12.817802 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:12.818725 14023 raft_consensus.cc:3060] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:12.790162 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 81023.6us
I20260709 13:07:12.829931 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:12.821049 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:12.872365 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:12.873332 14018 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:07:12.879666 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:12.880787 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.929351 14586 heartbeater.cc:646] Failed to heartbeat to 127.12.109.126:43315 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.12.109.126:43315 timed out after 15.000s (SENT)
I20260709 13:07:12.931282 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:12.880430 13944 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:07:12.935932 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:12.941650 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:12.745093 20979 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:12.737983 20584 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:12.950271 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:12.942618 14020 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 22.
I20260709 13:07:12.780033 20866 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:12.787158 20894 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:12.823942 20436 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.913s	user 0.006s	sys 0.008s
I20260709 13:07:12.966235 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:12.958897 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 137951us
I20260709 13:07:12.801095 21001 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:12.970165 20977 consensus_meta.cc:287] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.676s	user 0.008s	sys 0.004s
I20260709 13:07:12.815769 20932 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.156s
I20260709 13:07:12.742116 20647 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.216s
I20260709 13:07:12.971870 20823 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:12.814263 20999 consensus_queue.cc:1048] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.204s
I20260709 13:07:13.007269 20436 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:12.966702 21014 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:07:13.058360 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 59511.6us
I20260709 13:07:13.059865 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:13.080814 14023 raft_consensus.cc:2468] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
W20260709 13:07:13.064893 20961 consensus_meta.cc:287] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.725s	user 0.013s	sys 0.002s
I20260709 13:07:13.002756 20894 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:13.083252 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.018638 20977 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:13.097507 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:13.104701 13944 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:13.106328 13988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.009s	sys 0.007s
W20260709 13:07:13.107177 13988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.010s	sys 0.007s
W20260709 13:07:13.107474 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 524231us
W20260709 13:07:13.112959 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 512038us
I20260709 13:07:13.108176 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:13.063567 20823 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:13.116562 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.100243 20962 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:13.120432 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:13.076082 20980 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:07:13.074234 20494 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:07:13.187040 13866 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 20: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:13.123914 14559 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 13:07:13.166905 13869 pending_rounds.cc:85] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 10
I20260709 13:07:13.187749 13869 pending_rounds.cc:107] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:07:13.188062 13869 raft_consensus.cc:2889] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:13.182775 20686 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:13.192025 20686 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:13.169088 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.120232 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:13.164702 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:07:13.170956 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 62868.2us
W20260709 13:07:13.129508 20988 consensus_meta.cc:287] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.625s	user 0.015s	sys 0.000s
I20260709 13:07:13.198118 13867 pending_rounds.cc:85] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 11
I20260709 13:07:13.199563 13867 pending_rounds.cc:107] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 18.12
W20260709 13:07:13.118851 20824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.007s	sys 0.011s
I20260709 13:07:13.199910 13867 raft_consensus.cc:2889] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:13.200186 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 587024us
I20260709 13:07:13.249739 13944 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
W20260709 13:07:13.203418 21006 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.370s	user 0.000s	sys 0.010s
I20260709 13:07:13.192503 20686 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:13.195659 20647 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:13.202775 21023 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.243323 21013 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:13.260154 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:07:13.264559 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 328700us
W20260709 13:07:13.211107 20824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.007s	sys 0.011s
W20260709 13:07:13.260510 20984 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:07:13.271229 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 151164us
I20260709 13:07:13.210799 21018 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:13.287914 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:13.277556 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:13.259547 21003 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:07:13.256518 14522 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:07:13.326478 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 70116.6us
W20260709 13:07:13.328411 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 68398.4us
I20260709 13:07:13.328555 20657 raft_consensus.cc:2804] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:13.329313 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:13.330315 21022 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:13.330858 21022 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:13.331238 21022 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:13.290284 20876 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.244s	user 0.014s	sys 0.001s
W20260709 13:07:13.317231 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.259044 21008 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:13.257558 21024 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.329358 20657 raft_consensus.cc:697] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:13.258576 20999 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:13.259313 21010 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:13.269923 20647 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.250488 21001 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:07:13.337494 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 60029.3us
I20260709 13:07:13.341228 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:13.372738 14019 raft_consensus.cc:3060] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:13.341806 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:13.212268 20988 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:13.357146 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:13.378808 14024 raft_consensus.cc:2393] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:07:13.342418 21006 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:07:13.367439 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:13.381560 14018 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:13.325248 21018 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:13.371006 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:13.343863 21008 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.314308 21013 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:13.384693 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:13.407830 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:13.409618 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:13.404109 14024 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:13.374964 20647 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:13.290706 21023 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:13.416404 14552 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:13.379015 20436 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:13.344770 20876 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:13.411441 21008 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:13.402199 21018 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:13.435446 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2789) took 1350 ms. Trace:
W20260709 13:07:13.437309 20997 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.388s	user 0.000s	sys 0.002s
I20260709 13:07:13.419140 20932 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:07:13.415338 21001 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:13.430361 20588 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:07:13.442401 21035 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:13.415422 20962 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:13.417294 21028 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:13.445433 20980 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:13.446166 20686 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:13.425541 20494 consensus_queue.cc:1048] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:07:13.446394 20988 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:13.451625 21013 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:13.517587 21028 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:13.453361 14024 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:13.456679 20657 consensus_queue.cc:237] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:13.464475 21020 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 13:07:13.437633 21026 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:13.431466 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 94633.1us
W20260709 13:07:13.517987 21036 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:07:13.539379 13840 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:13.540302 20600 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:13.540755 20600 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:13.363580 21024 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:13.495375 20962 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:13.469930 20943 consensus_meta.cc:287] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.206s	user 0.011s	sys 0.006s
I20260709 13:07:13.472178 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:13.428781 20662 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:13.422243 20977 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:13.475652 14018 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:13.378599 21010 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:13.432126 20894 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:13.429176 20823 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:13.468874 20864 consensus_meta.cc:287] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.214s	user 0.015s	sys 0.001s
I20260709 13:07:13.447046 20436 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:13.503832 14019 raft_consensus.cc:2468] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:07:13.471974 13766 catalog_manager.cc:5697] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 reported cstate change: term changed from 8 to 14, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 14 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:13.515388 20830 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:13.515790 20647 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:13.516639 21022 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:13.531741 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59650 (request call id 1083) took 1042 ms. Trace:
I20260709 13:07:13.518798 13871 raft_consensus.cc:1240] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:07:13.527215 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3357) took 1406 ms. Trace:
I20260709 13:07:13.569775 14552 rpcz_store.cc:276] 0709 13:07:12.489282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.489440 (+   158us) service_pool.cc:224] Handling call
0709 13:07:13.531448 (+1042008us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b54005609d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:13.531633 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2122317184}
I20260709 13:07:13.467569 14017 rpcz_store.cc:276] 0709 13:07:12.085287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.097240 (+ 11953us) service_pool.cc:224] Handling call
0709 13:07:12.575872 (+478632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.576573 (+   701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.435209 (+858636us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b5400412650. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:13.435366 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1099970816}
I20260709 13:07:13.495052 21035 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:13.520042 21032 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:13.549690 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 80586.2us
I20260709 13:07:13.553874 20662 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:07:13.528316 13768 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57626 (request call id 248) took 68241 ms (client timeout 15000 ms). Trace:
I20260709 13:07:13.551267 20962 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:13.570098 14016 rpcz_store.cc:276] 0709 13:07:12.120450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.353099 (+232649us) service_pool.cc:224] Handling call
0709 13:07:12.353518 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:12.353981 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:12.354010 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:12.354722 (+   712us) log.cc:844] Serialized 52 byte log entry
0709 13:07:12.530233 (+175511us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:12.530299 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.530407 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:13.526977 (+996570us) raft_consensus.cc:1685] finished
0709 13:07:13.527000 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:13.527152 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19308,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":37980,"thread_start_us":175087,"threads_started":1,"wal-append.queue_time_us":596098}
I20260709 13:07:13.532249 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:13.574159 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.564458 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:13.555787 21027 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:13.526856 21023 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:13.566903 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 159145us
I20260709 13:07:13.604780 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:13.606529 14524 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:13.550388 21000 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:07:13.592808 20584 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.547940 21026 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:13.576490 21035 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:13.621498 13915 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:13.561820 20864 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:13.625006 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 83866.9us
I20260709 13:07:13.630254 21019 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:13.645768 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 1015) took 1257 ms. Trace:
I20260709 13:07:13.601070 21027 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:13.648115 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:13.627740 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:13.648875 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:13.646262 13873 rpcz_store.cc:276] 0709 13:07:12.388557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.388785 (+   228us) service_pool.cc:224] Handling call
0709 13:07:12.389211 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.394415 (+  5204us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:07:12.433812 (+ 39397us) log.cc:844] Serialized 14 byte log entry
0709 13:07:13.645238 (+1211426us) spinlock_profiling.cc:243] Waited 7.08 ms on lock 0x7b180042c880. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c342b060 00007fb2c343d716 00007fb2c344e287
0709 13:07:13.645375 (+   137us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:13.645433 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:13.645481 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.645552 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:13.645696 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16934,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":54603,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":16276224,"thread_start_us":25757,"threads_started":1}
I20260709 13:07:13.654489 14019 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:13.613512 20977 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:13.656231 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:13.617728 20969 raft_consensus.cc:2804] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:07:13.661078 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.520476 21001 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:13.663225 13867 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:13.671866 14022 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
W20260709 13:07:13.660952 20602 consensus_meta.cc:287] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 1.131s	user 0.016s	sys 0.000s
I20260709 13:07:13.678683 20686 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:13.690658 20686 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:13.693078 20686 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:13.678893 20969 raft_consensus.cc:697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:13.694813 20969 consensus_queue.cc:237] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:13.698500 21012 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.428s	user 0.004s	sys 0.000s
I20260709 13:07:13.700232 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 914) took 1186 ms. Trace:
I20260709 13:07:13.700742 14551 rpcz_store.cc:276] 0709 13:07:12.513401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.528201 (+ 14800us) service_pool.cc:224] Handling call
0709 13:07:12.552121 (+ 23920us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:12.552689 (+   568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:12.552737 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:12.555687 (+  2950us) log.cc:844] Serialized 52 byte log entry
0709 13:07:12.723760 (+168073us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:12.723824 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.723926 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:13.698118 (+974192us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b24002cf048. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c01cc9c4 00007fb2c01caacb 00007fb2c0385dc9 00007fb2c0385a55 00007fb2c038509d 00007fb2c2fb710e 00007fb2c2fce88c 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221
0709 13:07:13.699750 (+  1632us) raft_consensus.cc:1685] finished
0709 13:07:13.699787 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:13.700146 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":365,"raft.queue_time_us":1236,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":15075,"spinlock_wait_cycles":26315008,"thread_start_us":131233,"threads_started":1,"wal-append.queue_time_us":605350}
W20260709 13:07:13.596122 13768 rpcz_store.cc:269] 0709 13:06:05.287116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.287414 (+   298us) service_pool.cc:224] Handling call
0709 13:07:13.528044 (+68240630us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:02.789065 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:02.931338 (+142273us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:02.931405 (+    67us) write_op.cc:435] Acquired schema lock
0709 13:07:02.931429 (+    24us) tablet.cc:662] Decoding operations
0709 13:07:02.960421 (+ 28992us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:03.005546 (+ 45125us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:03.005586 (+    40us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:07:03.006756 (+  1170us) tablet.cc:701] Row locks acquired
0709 13:07:03.006826 (+    70us) write_op.cc:260] PREPARE: finished
0709 13:07:03.007277 (+   451us) write_op.cc:270] Start()
0709 13:07:03.007457 (+   180us) write_op.cc:276] Timestamp: P: 1783602423007179 usec, L: 0
0709 13:07:03.007509 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 13:07:03.008396 (+   887us) log.cc:844] Serialized 3637 byte log entry
0709 13:07:04.174280 (+1165884us) op_driver.cc:464] REPLICATION: finished
0709 13:07:04.575087 (+400807us) write_op.cc:301] APPLY: starting
0709 13:07:04.739730 (+164643us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:05.118670 (+378940us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:05.118982 (+   312us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:05.595704 (+476722us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:05.596206 (+   502us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 13:07:05.596278 (+    72us) write_op.cc:312] APPLY: finished
0709 13:07:05.596818 (+   540us) log.cc:844] Serialized 89 byte log entry
0709 13:07:05.906321 (+309503us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:05.906494 (+   173us) write_op.cc:454] Released schema lock
0709 13:07:05.906568 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:05.912662 (+  6094us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:05.912710 (+    48us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":892463,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":352726,"apply.run_cpu_time_us":55523,"apply.run_wall_time_us":1375087,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41693,"num_ops":9,"prepare.queue_time_us":626544,"prepare.run_cpu_time_us":37097,"prepare.run_wall_time_us":1024161,"raft.queue_time_us":397411,"raft.run_cpu_time_us":31825,"raft.run_wall_time_us":307714,"replication_time_us":1147967,"thread_start_us":1900947,"threads_started":5,"wal-append.queue_time_us":1618841,"wal-append.run_cpu_time_us":96215,"wal-append.run_wall_time_us":2414203}]]}
W20260709 13:07:13.706418 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 78764us
I20260709 13:07:13.707373 13840 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:13.658332 21027 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:13.704104 13766 catalog_manager.cc:5697] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 reported cstate change: term changed from 10 to 12, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 12 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.602916 20600 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:13.725009 20600 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:13.725257 21008 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:13.727622 20979 consensus_meta.cc:287] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.741s	user 0.013s	sys 0.000s
I20260709 13:07:13.728154 20979 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:13.728482 21054 consensus_queue.cc:1175] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" responder_term: 17 status { last_received { term: 17 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 17 index: 10 } }
I20260709 13:07:13.670804 20943 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:13.595811 21032 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:13.673050 21013 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:07:13.673811 20436 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:13.627113 20584 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:13.773402 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 66133.3us
W20260709 13:07:13.799053 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.717705 20999 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.809968 13840 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:13.723556 13766 catalog_manager.cc:5697] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:13.778625 20793 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.795420 21000 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:13.772473 20943 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:13.876267 14017 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
W20260709 13:07:13.878355 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 68509us
I20260709 13:07:13.808291 21033 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.888934 20943 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:13.893718 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:07:13.886739 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 262219us
I20260709 13:07:13.896905 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:13.711797 20662 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:13.899612 13948 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:07:13.872071 20931 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.258s	user 0.012s	sys 0.019s
I20260709 13:07:13.899837 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:13.875564 21009 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:13.907042 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:13.915836 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:13.915726 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:13.916718 13945 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
W20260709 13:07:13.905711 20931 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.297s	user 0.012s	sys 0.020s
I20260709 13:07:13.890213 20787 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:13.954536 21049 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:13.956780 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:13.906940 21055 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.871603 20999 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:13.910656 21009 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:13.914479 21024 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:13.971556 13766 catalog_manager.cc:5697] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:13.987561 13989 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.002s	sys 0.011s
W20260709 13:07:13.987964 13989 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.002s	sys 0.011s
W20260709 13:07:13.988538 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 659264us
I20260709 13:07:13.988279 13766 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.989878 21018 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:13.980085 21022 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:13.970173 20787 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:13.920672 21033 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:13.981163 21026 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:13.979379 21010 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:13.993535 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:13.889387 20793 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:13.999495 21032 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:13.999403 13766 catalog_manager.cc:5697] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:14.030256 21033 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:13.976310 21055 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.029850 13766 catalog_manager.cc:5697] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 9 to 12, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 12 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:14.032415 13948 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:07:14.032148 13766 catalog_manager.cc:5697] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:14.032995 21055 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:14.033296 21048 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.223s	user 0.002s	sys 0.000s
W20260709 13:07:14.000614 21060 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:07:13.979429 21028 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:13.994520 21024 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:07:14.035264 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:13.977867 20647 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:14.036337 20647 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:14.004616 20584 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:14.036670 13989 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:14.011839 20955 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:14.037581 21024 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:14.027631 21010 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:14.013453 20824 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:13.976672 21009 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:13.973006 21013 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:07:14.020623 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 63899.3us
I20260709 13:07:14.006711 20600 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:14.038043 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:14.024438 20864 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.059069 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:14.103608 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:14.104817 13948 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:14.048043 21066 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:14.039232 20940 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:14.038292 20955 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:14.117739 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.043794 20824 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:14.047680 20793 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.018817 21027 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.123999 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:14.125345 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 66581.2us
I20260709 13:07:14.098682 20965 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:14.068765 20999 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:14.130242 14021 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:14.138237 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:14.139106 13872 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:14.117650 20620 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:14.120195 20940 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:14.098457 20600 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.151674 21068 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:14.038511 13766 catalog_manager.cc:5697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:14.120146 20864 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.123569 21066 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:14.137194 21001 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.156440 14021 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
W20260709 13:07:14.159148 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 121171us
I20260709 13:07:14.150645 21073 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:14.135344 20955 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.131716 20793 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.170872 20940 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.172564 21018 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:14.173722 20979 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.180044 20787 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:14.180444 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.180873 20600 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.182240 20793 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:14.184671 20793 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:14.174808 21073 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.186480 21073 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:14.187253 21027 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.187386 20864 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:14.188201 21027 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:14.191565 21066 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:14.190825 13766 catalog_manager.cc:5697] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 10 to 11, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 11 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:14.192560 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1237) took 1135 ms. Trace:
W20260709 13:07:14.195269 21047 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
I20260709 13:07:14.179117 21001 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:14.166499 20962 consensus_meta.cc:287] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.554s	user 0.012s	sys 0.000s
I20260709 13:07:14.197510 21001 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.197395 20962 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.192972 13943 rpcz_store.cc:276] 0709 13:07:13.057154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.057340 (+   186us) service_pool.cc:224] Handling call
0709 13:07:13.076643 (+ 19303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:13.427141 (+350498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.427190 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:13.427250 (+    60us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:13.427297 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.192298 (+765001us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b5400091d54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:14.192355 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.192495 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":805411584}
I20260709 13:07:14.202124 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.205365 20962 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:14.210269 20940 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:14.214673 20943 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.206580 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:14.206874 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:14.216164 14554 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:14.194168 20955 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.216630 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:14.168776 20620 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.218008 20619 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:14.208356 21008 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:14.187297 21017 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.913s	user 0.003s	sys 0.000s
I20260709 13:07:14.193181 21072 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:07:14.220232 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:14.164558 21013 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:14.232527 13872 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
W20260709 13:07:14.219163 21035 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.005s
W20260709 13:07:14.233901 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 100542us
I20260709 13:07:14.236609 20962 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:14.230182 21018 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:14.192972 13766 catalog_manager.cc:5697] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 reported cstate change: term changed from 9 to 14, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 14 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:14.243371 20943 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.244238 13766 catalog_manager.cc:5697] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:14.246639 13989 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:14.189961 20864 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:14.249461 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:14.249665 20620 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.250238 21018 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:14.250977 21075 consensus_queue.cc:1059] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:14.253383 21075 raft_consensus.cc:3055] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:14.253793 21075 raft_consensus.cc:740] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:14.247839 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:14.256925 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:14.258564 13989 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:14.259191 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:14.260679 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:14.262059 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:14.268347 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.259766 20864 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:14.263067 21072 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:14.270750 21025 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:14.273505 14524 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:14.269270 13870 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 25.
I20260709 13:07:14.261649 20943 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:14.264390 20962 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:14.286623 20693 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:14.304303 21061 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.349s	user 0.001s	sys 0.001s
I20260709 13:07:14.307737 13987 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:14.307606 21009 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.284795 20619 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:14.263343 20620 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:14.308777 20620 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:14.255291 21035 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:14.309717 20620 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.254671 21075 consensus_queue.cc:260] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.290426 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.297180 20902 raft_consensus.cc:2804] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:14.300446 21073 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.285565 20943 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.263126 20955 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.322625 20902 raft_consensus.cc:697] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:07:14.325570 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 67072.7us
I20260709 13:07:14.327417 20943 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:14.314724 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.316221 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:14.314579 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:14.317356 14016 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:14.311873 13766 catalog_manager.cc:5697] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:14.325119 20902 consensus_queue.cc:237] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:14.332697 14018 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:14.333211 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.333900 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:14.313589 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:14.335944 14016 raft_consensus.cc:2393] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 17: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:14.334352 14023 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:14.330238 20965 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.004s
I20260709 13:07:14.337736 20894 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.306178 21025 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.337244 20955 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.337457 21072 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.338344 20830 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.338472 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:14.338828 20999 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.339229 20748 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:14.339418 21076 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.357786 20748 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:14.357764 20999 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.340118 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:14.358451 20748 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:14.359491 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:07:14.340580 21051 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.541s	user 0.001s	sys 0.000s
I20260709 13:07:14.360316 20999 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:14.311865 21035 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.362115 21072 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.358385 21076 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:14.354574 20955 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.364374 21072 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.365005 21076 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:14.365617 20955 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.343387 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.309955 21009 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.344518 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.318081 20962 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:14.369185 13871 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:07:14.317827 20864 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.349700 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.376200 13875 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:14.351037 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:14.353216 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:14.377281 14018 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:14.354980 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.355698 20830 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.356559 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:14.345659 20619 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.343024 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.368633 13989 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:14.345197 21024 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:14.370213 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.374691 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.377911 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.356863 20989 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:14.378640 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:14.377368 14023 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:14.382951 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.383852 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.384243 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.383575 13868 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:14.346076 20894 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.385664 13871 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:14.380483 13869 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:07:14.375512 20962 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.384586 13870 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:14.379989 20619 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.395017 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3367) took 1051 ms. Trace:
I20260709 13:07:14.395140 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.395644 14024 rpcz_store.cc:276] 0709 13:07:13.343277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.416590 (+ 73313us) service_pool.cc:224] Handling call
0709 13:07:13.416919 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:13.541587 (+124668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.394683 (+853096us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b540023b754. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:14.394935 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48715648}
I20260709 13:07:14.392602 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.322036 21013 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.404865 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.406345 14020 raft_consensus.cc:2410] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:07:14.397325 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.410961 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:14.412341 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.415328 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.416188 13944 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:14.417671 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:14.422881 20748 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:14.423713 20619 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:14.423974 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.424701 14017 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:14.425302 20748 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 26 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.425699 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:14.351251 21025 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.426203 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:14.426877 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.430124 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:14.441385 21056 log.cc:927] Time spent T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.486s	user 0.003s	sys 0.000s
I20260709 13:07:14.381428 20864 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.430814 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:14.434468 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:07:14.437158 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.438541 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.439657 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.446852 14020 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:14.441313 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.446141 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:14.446854 13872 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:14.447710 13948 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:14.398470 20962 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:14.390770 20989 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.453066 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.453395 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.454113 14024 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:14.445333 13873 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:07:14.464553 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:14.475000 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:14.482612 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.483569 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:14.485060 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:14.485177 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.485669 13875 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:14.493176 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:14.493276 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:14.495667 14553 raft_consensus.cc:2393] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:14.502017 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:14.446611 21068 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:14.505280 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:14.507761 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.463282 20989 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:14.509483 14024 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:07:14.474277 21054 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:14.513408 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.517539 13943 raft_consensus.cc:2410] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:07:14.513332 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:14.520093 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.521857 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:14.528756 13872 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:07:14.529570 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.530129 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:14.519483 20940 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:14.537173 14018 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:14.540633 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:14.541919 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.528354 21073 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.554232 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:14.552369 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:14.555330 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.523576 21054 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:14.598546 13875 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:14.602639 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.513772 21068 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:14.593487 13838 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.66:43953 timed out after 1.566s (SENT) [suppressed 19 similar messages]
I20260709 13:07:14.619966 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.622390 14018 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:07:14.619966 21082 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:07:14.631316 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.646477 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.650151 14018 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:14.654505 13860 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b098cc2d51e471cb2c9c1b17eebcb57 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 10)) from {username='slave'} at 127.0.0.1:46040
I20260709 13:07:14.642524 20602 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:14.659488 20943 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.646692 20940 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.665104 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.668665 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:14.671941 14559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 1085) took 2083 ms. Trace:
I20260709 13:07:14.672547 14559 rpcz_store.cc:276] 0709 13:07:12.588637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.596965 (+  8328us) service_pool.cc:224] Handling call
0709 13:07:12.608624 (+ 11659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:13.189770 (+581146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.664439 (+1474669us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b540040a6d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:14.671836 (+  7397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179320064}
I20260709 13:07:14.674562 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:14.687587 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:14.697441 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 104007us
W20260709 13:07:14.676663 21083 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:07:14.689735 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:14.698678 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.697243 21066 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:14.700812 13839 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:14.678972 21072 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:14.680940 21076 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:14.712785 13944 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:14.703702 21066 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.714731 21066 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:14.676299 20940 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.667112 20602 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:14.722191 21054 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:14.722505 20602 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:14.727792 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.728597 13874 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:14.729516 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.731406 20962 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.732486 21075 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:14.738744 21091 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:07:14.740608 13870 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [14.10-15.11]
I20260709 13:07:14.740702 21068 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:14.741587 21068 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:14.742625 21073 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:14.742661 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.743911 14019 raft_consensus.cc:2410] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:07:14.746524 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:14.746868 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.753412 13874 raft_consensus.cc:2393] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:14.711767 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:14.767712 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:14.768384 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.741994 21068 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.769129 13950 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 23.
I20260709 13:07:14.769379 21068 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:14.771998 21086 log.cc:927] Time spent T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.133s	user 0.003s	sys 0.000s
I20260709 13:07:14.772217 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.770632 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:14.775264 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:14.776316 13874 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:14.777148 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.779959 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:14.780503 13874 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:14.761878 20962 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:14.766753 21076 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.766753 21073 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.792263 14018 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:14.800158 20876 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.803644 20787 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:14.804265 20787 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:14.809864 21011 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 1.988s	user 0.000s	sys 0.004s
W20260709 13:07:14.813792 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 446290us
I20260709 13:07:14.814689 13874 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:14.815711 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.816331 13874 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:14.817577 13989 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:14.812211 21076 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:14.821326 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 131697us
I20260709 13:07:14.822144 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:14.823567 20876 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:14.824810 21076 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.826545 21023 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:14.827675 13759 catalog_manager.cc:5028] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): tablet 6b098cc2d51e471cb2c9c1b17eebcb57 (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:07:14.827766 13871 raft_consensus.cc:1240] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:07:14.828963 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3355) took 2709 ms. Trace:
I20260709 13:07:14.829761 13871 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:14.832036 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:14.834728 13871 raft_consensus.cc:1240] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:14.827900 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:14.836185 13941 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:14.810084 21073 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.839838 21076 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:14.840291 21076 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.840626 21076 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:14.841255 21073 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.820328 21024 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:14.843317 21024 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:14.844816 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:14.845454 13868 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:14.847564 14023 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:07:14.829358 14015 rpcz_store.cc:276] 0709 13:07:12.119966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.317216 (+197250us) service_pool.cc:224] Handling call
0709 13:07:12.317583 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:12.343994 (+ 26411us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:07:12.550812 (+206818us) log.cc:844] Serialized 14 byte log entry
0709 13:07:13.054540 (+503728us) log.cc:844] Serialized 14 byte log entry
0709 13:07:13.071225 (+ 16685us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:13.075791 (+  4566us) log.cc:844] Serialized 26 byte log entry
0709 13:07:13.076032 (+   241us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:13.076079 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.076175 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:14.809162 (+1732987us) spinlock_profiling.cc:243] Waited 8.7 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:14.828720 (+ 19558us) raft_consensus.cc:1685] finished
0709 13:07:14.828748 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.828879 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95446,"prepare.queue_time_us":261040,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":86,"raft.queue_time_us":544562,"raft.run_cpu_time_us":499,"raft.run_wall_time_us":23985,"spinlock_wait_cycles":20019456,"thread_start_us":672122,"threads_started":3,"wal-append.queue_time_us":239505}
I20260709 13:07:14.852703 21049 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.858841 13943 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:14.859654 21026 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.860270 21026 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.864856 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.871134 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.877326 14021 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: [18.10-19.11]
W20260709 13:07:14.880635 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 179871us
I20260709 13:07:14.881718 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:14.885404 14554 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:14.853466 21049 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:14.887357 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 69882.1us
W20260709 13:07:14.894002 14022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49644 (request call id 3364) took 1551 ms (client timeout 1626 ms). Trace:
I20260709 13:07:14.841856 21073 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:14.894673 14022 rpcz_store.cc:269] 0709 13:07:13.342343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.370687 (+ 28344us) service_pool.cc:224] Handling call
0709 13:07:14.893706 (+1523019us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54002986d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:14.893901 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400721664}
W20260709 13:07:14.888198 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.896252 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:14.887362 14023 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:14.915447 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:14.892622 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:14.916471 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:14.917636 14522 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:14.918507 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:14.919341 13949 raft_consensus.cc:3060] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:14.930361 13868 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:14.931849 13868 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:14.933104 13868 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:14.934033 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.935456 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:14.936049 13868 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:14.886727 21018 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:14.947852 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:14.916983 21010 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:14.953595 13868 pending_rounds.cc:85] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 7
I20260709 13:07:14.969481 13868 pending_rounds.cc:107] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:07:14.969909 13868 raft_consensus.cc:2889] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:14.909976 21023 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:14.934446 20962 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:14.986481 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:14.887215 21032 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.068s
I20260709 13:07:14.937700 20748 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:14.942447 20619 raft_consensus.cc:2764] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:07:14.987762 13914 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:14.942716 20872 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:14.952560 20787 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:14.942842 20647 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.925981 20876 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 13:07:14.968293 20943 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:14.923707 21073 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:14.969178 21010 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:14.979313 21022 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.984549 21026 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:14.989259 20931 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:14.990998 20962 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:14.891013 20824 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:15.015722 21100 log.cc:927] Time spent T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.217s	user 0.001s	sys 0.001s
I20260709 13:07:15.008158 20980 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:15.016897 20872 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:15.029846 21010 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:15.040257 20962 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:14.992394 21023 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:15.039211 21099 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:15.017855 20647 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.033077 21066 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.051643 20931 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:15.031878 21022 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.115223 21023 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:15.149080 20830 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.124372 20647 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:15.152312 21025 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.168748 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:15.169273 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.170141 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.170543 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:15.170677 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:15.171417 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:15.172633 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.173205 13948 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:15.181919 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:15.182935 13870 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:15.184177 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:15.187994 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.138499 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3526) took 1826 ms. Trace:
I20260709 13:07:15.191475 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.141567 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.191560 13867 rpcz_store.cc:276] 0709 13:07:13.311767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.528938 (+217171us) service_pool.cc:224] Handling call
0709 13:07:13.529248 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:13.679457 (+150209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.138255 (+1458798us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b540026c5d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:15.138436 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307443456}
I20260709 13:07:15.140040 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3375) took 1088 ms. Trace:
I20260709 13:07:15.208087 13949 raft_consensus.cc:2468] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:07:15.222206 14016 rpcz_store.cc:276] 0709 13:07:14.051838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.348206 (+296368us) service_pool.cc:224] Handling call
0709 13:07:14.348538 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:14.813251 (+464713us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:14.813284 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:14.835967 (+ 22683us) log.cc:844] Serialized 26 byte log entry
0709 13:07:14.905493 (+ 69526us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:14.905549 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.139607 (+234058us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b540029a4d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:15.139667 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.139723 (+    56us) raft_consensus.cc:1685] finished
0709 13:07:15.139736 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.139973 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":451686,"raft.queue_time_us":464816,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":28448640,"thread_start_us":62895,"threads_started":1,"wal-append.queue_time_us":59850}
I20260709 13:07:15.228749 13838 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:15.125840 20989 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.604s	user 0.011s	sys 0.001s
W20260709 13:07:15.143836 21016 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 2.322s	user 0.003s	sys 0.003s
I20260709 13:07:15.128237 20864 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.251340 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:15.140492 20894 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.266654 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:15.144405 21073 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.274154 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:15.266719 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 327688us
I20260709 13:07:15.221446 21014 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:15.219946 21025 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:15.138661 20940 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.148311 21049 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:15.278600 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:15.124418 21066 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:15.261869 21000 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.356s	user 0.009s	sys 0.015s
I20260709 13:07:15.288043 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:15.136195 21076 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:15.260720 20830 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:15.147334 21068 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.132496 20931 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:15.204846 20872 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:15.332633 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 54129.1us
I20260709 13:07:15.348593 13989 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:15.277472 20965 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.349522 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:15.349411 20965 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:15.351349 21023 raft_consensus.cc:2804] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:15.351907 21023 raft_consensus.cc:697] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:07:15.332856 21000 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.436s	user 0.010s	sys 0.015s
I20260709 13:07:15.352548 21000 raft_consensus.cc:697] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:07:15.353546 21101 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:07:15.353011 21023 consensus_queue.cc:237] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.353581 21000 consensus_queue.cc:237] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.332907 20856 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:07:15.358249 21049 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:15.358860 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:15.359061 13989 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:15.360415 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:15.351847 20965 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:15.361234 21026 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:15.361294 20965 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:15.361747 21026 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:15.361893 20965 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:15.362227 21026 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:15.362407 20965 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:15.344847 21025 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.367902 21025 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.337321 13838 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:15.343739 21018 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:15.368778 21018 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:15.369803 21018 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.273203 20962 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.373267 21010 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:15.373888 20962 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:15.346900 20773 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:15.374756 20773 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:15.375635 21010 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:15.375717 20773 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:15.377704 21068 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.377182 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:15.283288 20647 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:15.381896 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:15.382625 13874 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:15.387099 13869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3402) took 1035 ms. Trace:
I20260709 13:07:15.389693 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:15.133117 21099 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.395816 13871 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:15.396826 13867 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:15.387601 13869 rpcz_store.cc:276] 0709 13:07:14.351686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.354754 (+  3068us) service_pool.cc:224] Handling call
0709 13:07:15.386857 (+1032103us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b54002e4fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:15.387023 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4599040}
I20260709 13:07:15.398233 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:15.399044 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 61958.4us
I20260709 13:07:15.399590 13989 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:15.320596 21073 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:15.266631 20989 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:15.411955 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:15.407833 13838 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:15.380681 20647 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:15.418728 13873 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:15.401870 21068 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:15.423146 14552 raft_consensus.cc:2393] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:15.425166 13867 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:15.405393 21073 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:15.343924 21014 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.426138 13838 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:15.410868 21010 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:15.430495 13874 raft_consensus.cc:2468] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:15.454496 21026 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.458083 21026 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:15.460618 20965 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.464820 20965 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:15.475369 20989 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:15.476656 21084 log.cc:927] Time spent T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.348s	user 0.005s	sys 0.001s
I20260709 13:07:15.478998 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:15.479884 21021 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:15.486361 21021 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:15.487208 21076 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:15.487581 21021 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:15.488997 21099 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.489148 21021 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:15.422091 21072 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:15.482720 21014 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.483179 21022 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:15.490388 14553 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:15.491060 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:15.492690 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41684 (request call id 3397) took 1151 ms. Trace:
I20260709 13:07:15.497653 13871 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:07:15.504853 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.451453 21116 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:15.513131 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:15.514990 13871 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:15.430390 21073 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:15.515409 13873 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:15.444304 21010 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:15.493181 13866 rpcz_store.cc:276] 0709 13:07:14.340858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.340994 (+   136us) service_pool.cc:224] Handling call
0709 13:07:14.341372 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:14.369754 (+ 28382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:14.369787 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:14.378853 (+  9066us) log.cc:844] Serialized 26 byte log entry
0709 13:07:14.410663 (+ 31810us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:14.466905 (+ 56242us) log.cc:844] Serialized 14 byte log entry
0709 13:07:14.467752 (+   847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.126302 (+658550us) spinlock_profiling.cc:243] Waited 9.69 ms on lock 0x7b18000281e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:07:15.128653 (+  2351us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b18000281e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:07:15.150578 (+ 21925us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b18000281e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03142a6 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419
0709 13:07:15.151061 (+   483us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b54004619d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:15.151110 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.475997 (+324887us) spinlock_profiling.cc:243] Waited 91.9 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:15.492433 (+ 16436us) raft_consensus.cc:1685] finished
0709 13:07:15.492461 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.492620 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4594,"prepare.queue_time_us":55996,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":3103,"raft.queue_time_us":3990,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":3164,"spinlock_wait_cycles":319024384,"thread_start_us":27737,"threads_started":1,"wal-append.queue_time_us":139371}
I20260709 13:07:15.511996 21014 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.518460 21010 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:15.528030 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 193044us
I20260709 13:07:15.587291 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:15.594758 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 436851us
W20260709 13:07:15.595602 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:15.596246 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:15.605010 21120 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:07:15.606007 13866 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:15.607643 13866 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:07:15.533360 21014 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.602540 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:15.615075 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:15.584565 20584 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:15.591655 21074 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:15.622339 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:15.625393 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:15.626264 14017 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:15.626540 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:15.628594 14018 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:15.603168 20961 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.655279 14022 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:15.625173 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:15.667011 21119 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:07:15.669467 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2804) took 1193 ms. Trace:
I20260709 13:07:15.669893 14021 rpcz_store.cc:276] 0709 13:07:14.476525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.878192 (+401667us) service_pool.cc:224] Handling call
0709 13:07:14.878507 (+   315us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:14.899309 (+ 20802us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:14.899361 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:14.931438 (+ 32077us) log.cc:844] Serialized 52 byte log entry
0709 13:07:15.443056 (+511618us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:15.470960 (+ 27904us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b48002115d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:07:15.514726 (+ 43766us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.523951 (+  9225us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.524257 (+   306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.524354 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.669185 (+144831us) raft_consensus.cc:1685] finished
0709 13:07:15.669210 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.669395 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8547,"prepare.queue_time_us":76508,"raft.queue_time_us":1985,"raft.run_cpu_time_us":3013,"raft.run_wall_time_us":588347,"spinlock_wait_cycles":47758848,"thread_start_us":450483,"threads_started":1,"wal-append.queue_time_us":548518}
I20260709 13:07:15.674979 14024 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: []
W20260709 13:07:15.676406 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51375.9us
I20260709 13:07:15.677265 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:15.678184 20931 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:15.678887 21008 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:15.679327 21008 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:15.679679 21008 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:15.679680 21075 consensus_queue.cc:1059] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:15.656853 21074 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:15.685237 20647 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:15.685503 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:15.686255 13872 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:15.619378 21077 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:15.696617 14023 raft_consensus.cc:1275] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:15.628952 20607 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:15.708761 21054 raft_consensus.cc:3055] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:15.709244 21054 raft_consensus.cc:740] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:15.710193 21054 consensus_queue.cc:260] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.709807 21074 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:15.661449 20793 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:15.656090 20584 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:15.713078 14559 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:15.692144 20961 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.716849 20961 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.718513 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:15.719280 21103 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.573s	user 0.004s	sys 0.001s
I20260709 13:07:15.721793 14023 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:15.711584 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:15.721683 13867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3552) took 1022 ms. Trace:
I20260709 13:07:15.721406 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3379) took 1613 ms. Trace:
I20260709 13:07:15.726370 13871 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:07:15.729876 14554 raft_consensus.cc:2393] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 23: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:15.725396 13867 rpcz_store.cc:276] 0709 13:07:14.699672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.430803 (+731131us) service_pool.cc:224] Handling call
0709 13:07:15.497059 (+ 66256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.497274 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:15.519026 (+ 21752us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.721348 (+202322us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:15.721401 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:15.721458 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.721524 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.721627 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36808,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":4129,"thread_start_us":201912,"threads_started":1}
I20260709 13:07:15.725601 14020 rpcz_store.cc:276] 0709 13:07:14.107730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.493027 (+385297us) service_pool.cc:224] Handling call
0709 13:07:14.493449 (+   422us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:14.852399 (+358950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:14.852434 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:14.853127 (+   693us) log.cc:844] Serialized 52 byte log entry
0709 13:07:14.931826 (+ 78699us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:14.994801 (+ 62975us) log.cc:844] Serialized 14 byte log entry
0709 13:07:14.996880 (+  2079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.996946 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.145447 (+148501us) spinlock_profiling.cc:243] Waited 5.64 ms on lock 0x7b180019a680. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:07:15.252554 (+107107us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b180019a680. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c02c0c0a 00007fb2c9a00219 00007fb2c730758f 00007fb2c0346571 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd
0709 13:07:15.718505 (+465951us) spinlock_profiling.cc:243] Waited 76 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:15.721147 (+  2642us) raft_consensus.cc:1685] finished
0709 13:07:15.721173 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.721342 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":411415,"prepare.queue_time_us":75464,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":367489,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":12648,"spinlock_wait_cycles":196087296,"thread_start_us":78265,"threads_started":1,"wal-append.queue_time_us":181432}
I20260709 13:07:15.716490 21022 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:15.713891 20584 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:15.699793 21077 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:15.751751 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:15.759207 14017 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 23.
I20260709 13:07:15.763168 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:15.766300 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:15.767448 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:15.742654 21077 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:15.772545 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:15.778807 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:15.753082 20931 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:15.769421 21029 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.146s	user 0.001s	sys 0.001s
W20260709 13:07:15.780320 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 101812us
I20260709 13:07:15.742883 21022 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:15.776244 21121 log.cc:927] Time spent T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:07:15.782735 13987 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:15.784549 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.774935 20647 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:15.788111 13987 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:15.789507 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:15.781313 21023 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:15.798103 20607 consensus_queue.cc:1059] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:15.790594 20965 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:15.799263 20607 raft_consensus.cc:3055] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:15.783363 20998 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:15.803059 21128 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:07:15.801376 13987 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:15.792176 21026 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:15.800978 21033 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:15.799649 20607 raft_consensus.cc:740] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:15.803546 20620 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:15.810395 20961 consensus_queue.cc:1059] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:15.808857 21014 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.818274 20998 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:15.818569 21135 consensus_queue.cc:1059] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:15.820056 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:15.818300 13840 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:15.825630 13949 raft_consensus.cc:3060] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:15.826313 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.826948 13944 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:15.827772 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.822127 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:15.829080 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.829173 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:15.830349 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.831071 13948 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:15.800908 21134 consensus_queue.cc:1059] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:07:15.830420 20961 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:15.832638 13989 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:15.834560 13840 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:15.834932 13869 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:15.835232 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.835578 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:15.836274 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.831894 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:15.839059 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:15.842259 13915 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:15.844040 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:15.845299 14022 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 22. Current term is 23. Ops: [20.16-22.17]
I20260709 13:07:15.845767 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:15.842846 13989 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:15.846481 14017 raft_consensus.cc:2393] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:15.846505 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:15.847061 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:15.849623 14021 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:07:15.851092 13949 raft_consensus.cc:2468] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:15.851507 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:15.853878 13948 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:07:15.837919 21000 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:15.856225 13989 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:15.857295 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:15.877111 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:15.865408 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:15.877777 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:15.838724 20961 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:15.875216 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:15.880393 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:15.882355 14024 raft_consensus.cc:2393] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:15.905607 14022 raft_consensus.cc:2393] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 21: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:15.859546 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:15.881634 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:15.886545 14021 raft_consensus.cc:2393] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:15.858793 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:15.909072 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:15.910544 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62159.6us
I20260709 13:07:15.912798 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:15.913862 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:15.918290 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:15.919543 14017 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:15.920632 14016 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:15.855751 21068 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:15.857901 20686 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:07:15.928105 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:15.929311 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:15.934583 14020 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:15.857648 20584 raft_consensus.cc:2804] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:15.937469 21074 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:15.936873 21008 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.939996 20824 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:15.942284 21023 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:07:15.942701 21136 log.cc:927] Time spent T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:07:15.955884 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:15.859189 21014 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:15.943427 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:15.823354 20931 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.979758 21014 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:15.980315 21068 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:15.980304 21014 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:15.981724 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:15.881502 20902 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 13:07:15.982601 21068 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:15.948009 14017 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:15.856566 20998 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:15.984228 21014 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:15.983099 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:15.987078 21068 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:15.987803 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:15.988822 21068 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:15.989881 21068 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.990377 21068 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:15.992508 21068 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:15.929076 21135 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:15.925454 21124 consensus_queue.cc:1048] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:07:15.993163 21023 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:15.976032 21022 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:15.926223 20969 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:15.929322 21099 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:15.954644 21074 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:15.959645 21027 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:15.994237 20584 raft_consensus.cc:697] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:16.019656 20856 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:07:16.021051 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 164884us
W20260709 13:07:16.019538 21130 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:07:16.026966 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:15.944188 21076 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.082s
I20260709 13:07:15.888139 20607 consensus_queue.cc:260] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.029537 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:16.030834 13869 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 20: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:16.031757 20998 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:07:16.032323 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:16.032423 21137 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:07:16.027233 21018 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:16.028290 20969 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:07:16.064500 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.037659 20872 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:16.050076 21021 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.560s	user 0.006s	sys 0.011s
I20260709 13:07:16.071661 13988 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:16.072140 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:07:16.074581 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 232401us
I20260709 13:07:16.050778 20607 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:16.077417 13869 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:16.089290 14020 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:16.077399 20998 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:16.051168 21023 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.074914 21018 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.116251 14556 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 13:07:16.047079 20584 consensus_queue.cc:237] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:16.122040 21008 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:16.068243 21074 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.115070 21022 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:16.130754 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 233556us
I20260709 13:07:16.134428 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:16.135331 20619 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:16.135722 20793 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:16.136292 20793 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:16.136528 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:16.137470 21142 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:16.138849 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:16.129535 21134 consensus_queue.cc:1059] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:07:16.135747 20619 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:16.140987 20619 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:16.143455 21018 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:16.143750 21142 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:16.128515 13867 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
W20260709 13:07:16.144366 21073 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.622s	user 0.009s	sys 0.005s
I20260709 13:07:16.144727 13867 raft_consensus.cc:606] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:07:16.144817 21073 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:16.145197 21142 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.145785 21142 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:16.147163 21010 consensus_meta.cc:287] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.002s
I20260709 13:07:16.147578 13873 raft_consensus.cc:1240] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:16.147980 21142 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:16.147630 21010 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.148694 21073 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:16.150650 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:16.137409 20793 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.154155 21014 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:16.154922 20793 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:16.145093 13867 raft_consensus.cc:614] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:16.158509 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:16.158432 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:16.159400 13868 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:16.159874 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:16.163477 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:16.138512 21021 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:16.149945 21010 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:16.161202 21074 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:16.171041 21014 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.183542 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:16.184541 21018 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:16.184974 21140 raft_consensus.cc:3055] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:16.177654 21014 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.189697 13868 raft_consensus.cc:2468] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:16.119571 20969 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:16.207019 13989 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:16.193953 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:16.157514 21022 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.216383 13868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17624553356541c69c1644ca409df1e9"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:07:16.217284 13868 raft_consensus.cc:606] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:07:16.217823 13868 raft_consensus.cc:614] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:16.219674 14553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 922) took 1335 ms. Trace:
I20260709 13:07:16.220242 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:16.220129 14553 rpcz_store.cc:276] 0709 13:07:14.884343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.884517 (+   174us) service_pool.cc:224] Handling call
0709 13:07:15.490164 (+605647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.490956 (+   792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.219472 (+728516us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b54006b2e50. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:16.219612 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1307754752}
I20260709 13:07:16.202344 21010 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:16.179239 21021 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.228112 20619 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.228911 21021 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:16.230338 21026 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:16.194553 21140 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:16.230772 21026 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:16.225391 21146 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:16.232672 21026 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.233322 20824 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:16.191398 21134 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:07:16.213138 21143 raft_consensus.cc:2804] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:16.246538 13840 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:16.249217 13840 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:16.258386 13840 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:16.223835 20931 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:07:16.262158 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:16.251682 21076 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:16.249413 21143 raft_consensus.cc:697] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:16.271234 13840 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:16.287547 13869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "3779649036364744823fc035dac892c7"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:07:16.288365 13869 raft_consensus.cc:606] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Received request to transfer leadership to TS 3779649036364744823fc035dac892c7
I20260709 13:07:16.288750 13869 raft_consensus.cc:614] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:16.290443 13840 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:16.265491 21027 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:16.263278 21073 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:16.310671 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 92983us
I20260709 13:07:16.289165 21021 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:07:16.315675 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.318287 14020 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:16.318470 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:16.319836 14018 raft_consensus.cc:1275] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:16.320219 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:16.319836 14022 raft_consensus.cc:1240] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 18. Current term is 19. Ops: [16.13-18.14]
I20260709 13:07:16.321334 14016 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:16.298034 20584 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.320569 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:16.296026 21001 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:16.323230 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:16.312769 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:16.328195 14021 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:16.297286 21099 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:16.330794 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:16.298170 21142 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:16.298182 21072 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:16.352273 13869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:07:16.353125 13869 raft_consensus.cc:606] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:16.358701 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 46075.6us
I20260709 13:07:16.357928 21072 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:16.364130 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:16.316067 21143 consensus_queue.cc:237] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.342795 21000 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:16.361222 21152 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:16.342566 21073 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:16.294301 21140 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.357836 21099 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:16.341742 20584 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.376262 21000 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.418704 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:16.421461 21159 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:16.380826 21152 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:16.349888 21142 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:16.412786 21077 consensus_meta.cc:287] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.634s	user 0.014s	sys 0.000s
I20260709 13:07:16.428131 21161 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:16.422859 13869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76071b78766a43e0b5097f615de4c569"
dest_uuid: "e1649107c781428692458f10597c5742"
mode: GRACEFUL
new_leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:07:16.437942 13869 raft_consensus.cc:606] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Received request to transfer leadership to TS d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:16.438337 13869 raft_consensus.cc:614] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:16.400693 21014 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:16.417414 21008 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:07:16.442865 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.451148 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:16.429592 21000 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:16.454557 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:16.455065 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.434465 20931 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:16.413178 21001 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:16.382939 21073 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.456998 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.468582 13948 raft_consensus.cc:2393] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 21: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:16.471266 13873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 1037) took 1340 ms. Trace:
I20260709 13:07:16.418255 20824 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:07:16.473924 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:16.482206 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.484939 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:16.455320 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:16.475133 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.437089 21140 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:16.431789 21152 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:16.431950 21134 consensus_queue.cc:1059] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:07:16.457252 21008 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.483116 13873 rpcz_store.cc:276] 0709 13:07:15.131326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.517068 (+385742us) service_pool.cc:224] Handling call
0709 13:07:15.517369 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.148254 (+630885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.470929 (+322675us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b540026bbd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:16.471178 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1448856448}
I20260709 13:07:16.492944 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.497526 21167 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:16.501057 21166 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:16.509124 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:16.428221 21159 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.481119 21054 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:16.478953 21162 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:16.521011 13875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41676 (request call id 3538) took 2002 ms (client timeout 1569 ms). Trace:
I20260709 13:07:16.445845 21077 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.456041 20872 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:07:16.451567 21149 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:07:16.480492 21157 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:16.521523 13875 rpcz_store.cc:269] 0709 13:07:14.518629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.608704 (+ 90075us) service_pool.cc:224] Handling call
0709 13:07:16.520774 (+1912070us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b5400450954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:16.520934 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2185812736}
I20260709 13:07:16.528776 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.528553 21167 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.530344 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.531003 13941 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
W20260709 13:07:16.531507 21147 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 13:07:16.534596 21154 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:07:16.448879 21161 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.539700 21073 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:16.540434 21161 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:16.489652 21099 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:16.542162 21137 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:07:16.547737 21077 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:16.549204 13947 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:16.526293 21150 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:07:16.559197 21072 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:16.561192 21142 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:16.510900 21008 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:16.503811 21151 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:07:16.528904 21166 consensus_queue.cc:1243] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:07:16.560845 21167 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:16.489717 20793 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:07:16.579741 13947 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:16.575747 20872 consensus_queue.cc:1243] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:07:16.566947 21124 consensus_queue.cc:1048] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.014s
W20260709 13:07:16.644136 20998 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.002s
I20260709 13:07:16.644876 20998 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:16.660692 21026 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:16.570225 21157 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.618770 20773 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:16.627112 21159 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:16.628288 21158 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:07:16.596143 21169 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:16.637657 20584 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:16.564801 21162 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.639073 21161 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:16.645009 21151 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:07:16.591295 21001 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.673908 21018 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:16.590096 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 88964.7us
W20260709 13:07:16.681311 21125 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.943s	user 0.000s	sys 0.005s
I20260709 13:07:16.687180 20961 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:16.692158 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.697510 21141 raft_consensus.cc:3055] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:07:16.614768 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:16.722957 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.735386 14017 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:07:16.674461 21000 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.755076 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:07:16.765182 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.719090 21167 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.738458 21146 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:16.730980 21162 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:16.698547 21074 consensus_meta.cc:287] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.003s
I20260709 13:07:16.767328 20619 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:16.768944 20902 raft_consensus.cc:3055] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:16.801935 13945 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:16.731598 21161 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.712054 21027 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:16.698021 21076 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:16.733590 21026 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:16.718199 21157 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:16.771440 20662 consensus_queue.cc:1059] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:07:16.723978 21164 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:16.737142 21022 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:16.728047 21169 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.714393 20620 raft_consensus.cc:3055] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:16.761029 21141 raft_consensus.cc:740] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:16.774983 20961 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:16.775311 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.776052 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.776712 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.777679 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.709856 20856 raft_consensus.cc:3055] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:07:16.780522 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.782140 13838 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:16.751843 21033 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:16.869369 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 87329.4us
I20260709 13:07:16.812068 14024 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 13:07:16.822942 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.870555 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:16.720737 21072 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.091s
I20260709 13:07:16.839689 21164 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:16.797498 21068 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:16.841599 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:16.846479 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.814599 20902 raft_consensus.cc:740] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:07:16.857549 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 365641us
I20260709 13:07:16.863771 13949 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:16.914870 13945 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:07:16.778843 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:16.786037 21167 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.849957 21010 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.847870 20620 raft_consensus.cc:740] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:16.867048 20856 raft_consensus.cc:740] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:16.779708 21140 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:16.877560 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:16.872961 13987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.032s	sys 0.066s
I20260709 13:07:16.872596 13838 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:16.928985 13987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.032s	sys 0.066s
W20260709 13:07:16.929641 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 57546.5us
W20260709 13:07:16.932732 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 171401us
I20260709 13:07:16.934331 13915 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:16.933622 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.850410 21152 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.930609 13838 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:16.940639 13874 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:16.921526 21027 raft_consensus.cc:3055] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:16.915289 21076 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:16.945221 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.942651 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:16.923020 20662 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:16.955871 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:16.956874 13869 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:07:16.958436 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:16.959939 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:16.962404 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:16.944583 21149 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:16.963971 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:16.950120 13838 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:16.951206 13871 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [14.8-15.9]
I20260709 13:07:16.874930 21033 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:16.952138 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:16.972332 13874 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:16.952569 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:16.973709 13868 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:16.947543 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:16.975854 13873 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:16.944236 21024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.825s	user 0.000s	sys 0.033s
I20260709 13:07:16.980494 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:16.981403 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:16.982079 13875 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:16.923524 20607 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:07:16.983392 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:16.985455 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:16.934981 21068 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:16.986744 13870 raft_consensus.cc:2393] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 21: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:16.984668 13987 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:16.928790 21153 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:07:16.924986 21010 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.027738 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:16.938832 20931 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:16.977627 13915 connection.cc:256] RPC call timeout handler was delayed by -1.2798763869999448s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
I20260709 13:07:17.058465 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:17.082099 14020 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:07:17.054737 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:17.082756 13875 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: [19.12-20.13]
I20260709 13:07:17.054737 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:17.083482 14556 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:17.083724 14015 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:17.025558 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:17.003750 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:17.026902 21159 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:16.988296 14017 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:16.988144 21162 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.074151 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:17.074954 14018 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:16.989284 13950 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 20. Current term is 21. Ops: []
W20260709 13:07:17.075896 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 91331.7us
W20260709 13:07:17.077497 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 90205.8us
I20260709 13:07:17.231801 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:17.233093 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:17.233861 13949 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:16.944885 21027 raft_consensus.cc:740] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:17.080626 13873 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:16.989306 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:16.952045 21072 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:17.059574 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:17.243083 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 162406us
I20260709 13:07:17.332476 14023 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:16.934340 21140 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:17.085340 13866 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:16.935421 21065 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.095s
W20260709 13:07:16.980600 21024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.888s	user 0.000s	sys 0.033s
I20260709 13:07:17.031742 21008 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.074627 20965 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:16.990828 21022 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.198088 21157 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:16.941813 21141 consensus_queue.cc:260] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:17.201202 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 118081us
I20260709 13:07:17.182279 21162 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:17.196964 13874 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:17.249877 13867 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:07:17.343436 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 354226us
I20260709 13:07:17.345292 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:17.083053 21173 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.025190 21099 consensus_queue.cc:1059] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.243s
I20260709 13:07:16.987057 20773 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:17.394259 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 162539us
I20260709 13:07:17.230306 21001 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:17.404757 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:17.342721 21165 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:17.412626 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 79464.7us
I20260709 13:07:17.471971 13838 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:17.393615 21075 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.466482 13949 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:17.467089 13943 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:17.409550 20867 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.409124 21054 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.349946 20998 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:17.387593 21124 consensus_queue.cc:1059] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.161s
I20260709 13:07:17.487305 13942 raft_consensus.cc:1275] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:07:16.952903 21164 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.061266 20856 consensus_queue.cc:260] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:17.407125 21022 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.504035 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:07:17.517820 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 281860us
I20260709 13:07:17.357000 21171 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:07:17.529510 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 57642.6us
I20260709 13:07:17.538002 21021 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:17.409865 21146 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:17.237000 21149 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.167s
I20260709 13:07:17.465742 21161 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:17.598974 21182 log.cc:927] Time spent T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:07:17.599982 21162 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:17.542714 21075 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.601811 21008 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.602464 21075 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.545629 20867 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:17.603857 13868 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:17.605087 20867 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.605309 13868 raft_consensus.cc:1240] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:17.607537 21022 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:17.608083 13868 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:17.608655 21033 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:17.546065 21054 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.609473 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:17.610370 13948 raft_consensus.cc:2410] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:07:17.611074 21054 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.601701 13870 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:17.552313 20998 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:17.613646 21179 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.219s	user 0.000s	sys 0.000s
I20260709 13:07:17.615384 20998 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:17.617624 21149 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:17.618090 21149 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:17.619048 21149 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.620407 21149 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:17.486582 13874 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:17.357010 21176 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.622996 21176 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.625176 21176 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.568516 13838 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:17.630928 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 62515.6us
I20260709 13:07:17.631609 21000 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.632479 13838 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:17.407753 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:17.634433 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 226735us
I20260709 13:07:17.634683 21172 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:17.635121 21172 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.635300 21162 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:17.636083 21162 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.542524 21099 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:17.531618 20773 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.635447 21172 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:17.415266 21157 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.586443 13947 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:17.565459 21164 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:17.635741 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:17.638950 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 140870us
I20260709 13:07:17.538548 21001 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:17.591253 21072 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:17.636655 21162 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:17.704319 21010 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:17.648546 21099 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.649901 21144 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:17.663480 13949 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:07:17.496587 21173 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:17.640110 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:17.713582 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 166688us
W20260709 13:07:17.714233 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 74209.9us
I20260709 13:07:17.666488 20965 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:17.688583 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:17.636860 21178 log.cc:927] Time spent T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:07:17.582769 21021 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:17.538291 21165 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:17.697556 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 61875us
I20260709 13:07:17.593521 21167 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.695714 21143 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:17.711939 13942 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:17.667748 21024 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.697244 21169 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:17.667486 21076 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:17.542090 20931 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.687273 21152 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.632525 21000 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:17.772136 13870 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:17.735972 21021 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:17.708509 21112 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.751220 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:17.719177 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.713907 21193 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:17.716624 20965 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:17.721385 21010 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:17.724300 20998 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:17.714408 14524 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:17.730768 13915 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:17.828841 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 98148.2us
I20260709 13:07:17.836999 21010 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:17.837594 21010 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:17.839010 20961 consensus_meta.cc:287] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.970s	user 0.012s	sys 0.001s
I20260709 13:07:17.840886 21001 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:17.842806 20773 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:17.843395 21140 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:17.752707 21143 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.846277 21143 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:17.848126 21165 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.756567 21167 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.849021 21167 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.851064 21188 raft_consensus.cc:3055] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:17.851469 21167 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.851541 21188 raft_consensus.cc:740] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:17.761826 21068 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.852425 21068 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.852675 21188 consensus_queue.cc:260] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:17.854815 21068 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.711089 21072 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:17.879853 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:17.856582 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 142263us
I20260709 13:07:17.858583 13945 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:07:17.887290 13945 pending_rounds.cc:85] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 10
I20260709 13:07:17.887576 13945 pending_rounds.cc:107] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:07:17.887835 13945 raft_consensus.cc:2889] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:17.818390 21180 log.cc:927] Time spent T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.114s	user 0.003s	sys 0.000s
I20260709 13:07:17.762670 21191 raft_consensus.cc:3055] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:17.889866 21191 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:17.890848 21191 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.891932 21072 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.892293 21191 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:17.894152 21177 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.894757 21177 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.895987 21140 consensus_queue.cc:1059] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:17.897531 21153 consensus_queue.cc:1059] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:17.898454 21065 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:17.797777 21157 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:17.902381 21074 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:17.808704 21099 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:17.877688 21161 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:17.906945 21161 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:17.788540 21173 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:17.816139 21027 consensus_queue.cc:260] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:17.911535 21027 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:17.832636 21193 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.915745 21193 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:17.915874 21173 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.916447 21173 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:17.809216 21159 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.000s
I20260709 13:07:17.931492 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:17.934125 13873 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 26.
I20260709 13:07:17.940186 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:17.943118 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:17.944164 13874 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:17.930537 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:17.950544 13873 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:17.772360 21176 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:17.879403 21152 raft_consensus.cc:2804] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:17.772855 21169 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:17.882345 20965 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:17.968075 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:17.970727 13867 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:17.918239 20619 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.884934 21026 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:17.920974 20867 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:17.758334 21024 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.855945 21000 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:17.887048 21165 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:17.909914 21161 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:17.973953 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:17.770614 21076 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:17.924823 21022 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.967931 21146 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.928275 21188 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:17.978911 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 3132) took 1525 ms. Trace:
I20260709 13:07:17.977659 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 3133) took 1523 ms. Trace:
I20260709 13:07:17.979466 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:17.926955 20998 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:17.975412 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3567) took 1022 ms. Trace:
I20260709 13:07:17.982026 14556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:17.982324 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:17.943427 21159 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:17.987708 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:17.885439 21112 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:17.977689 21169 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:17.974895 21176 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:17.905725 21010 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:17.989372 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 48693.9us
I20260709 13:07:18.055171 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3583) took 1030 ms. Trace:
I20260709 13:07:18.047196 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:18.028648 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:18.050395 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.083159 13873 raft_consensus.cc:2410] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:07:18.083192 14559 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 23.
I20260709 13:07:18.066900 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:18.085800 13943 rpcz_store.cc:276] 0709 13:07:16.453808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.529929 (+ 76121us) service_pool.cc:224] Handling call
0709 13:07:16.530308 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.536776 (+1006468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.977309 (+440533us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54000f5854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:17.977573 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2026294528}
I20260709 13:07:18.081573 13950 rpcz_store.cc:276] 0709 13:07:16.453541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.527713 (+ 74172us) service_pool.cc:224] Handling call
0709 13:07:16.528167 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.236777 (+708610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.978662 (+741885us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b5400086e54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:17.978850 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1059952000}
I20260709 13:07:18.081916 13870 rpcz_store.cc:276] 0709 13:07:17.024952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.613537 (+588585us) service_pool.cc:224] Handling call
0709 13:07:17.614192 (+   655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.772612 (+158420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.054811 (+282199us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54002e68d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:18.055076 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362295040}
I20260709 13:07:18.013407 21033 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.013243 21022 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:18.093899 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.098155 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:18.105474 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 354318us
I20260709 13:07:18.106431 13987 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:18.107887 13866 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:17.989169 20867 raft_consensus.cc:697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:18.036201 21173 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:17.981029 20619 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.162091 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:18.116345 13872 rpcz_store.cc:276] 0709 13:07:16.953299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.953491 (+   192us) service_pool.cc:224] Handling call
0709 13:07:16.953867 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.954132 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:17.489778 (+535646us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.974951 (+485173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:17.975013 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:17.975070 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.975167 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:17.975341 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":646056,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":3057,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":7923,"thread_start_us":932896,"threads_started":3,"wal-append.queue_time_us":281688}
I20260709 13:07:18.152175 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:18.032222 20773 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:18.150722 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:18.163542 13942 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:17.977275 21152 raft_consensus.cc:697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:18.160574 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 66496.5us
I20260709 13:07:18.089061 21143 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:18.069126 21177 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.168834 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.174512 13867 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 15: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:18.174898 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:18.175523 13949 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:18.169581 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.172379 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:18.180146 13873 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:18.104545 21099 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.012537 21146 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:18.199807 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.200841 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.202975 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:17.992033 21165 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:18.211324 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 104970us
I20260709 13:07:18.211195 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:17.985276 21018 consensus_queue.cc:1059] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.186s
I20260709 13:07:18.212255 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:18.213637 13875 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 21: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:07:18.214357 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.094556 21033 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.215382 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.101963 21026 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.081s
I20260709 13:07:18.091468 20998 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.058914 20902 consensus_queue.cc:260] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.226449 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.117980 21161 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.094s
I20260709 13:07:18.173604 21143 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:18.276479 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.223558 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.297282 13873 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:07:18.192862 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.042519 20620 consensus_queue.cc:260] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.291131 13875 raft_consensus.cc:2393] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:18.210417 21202 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.197602 21159 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:18.292409 21201 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:07:18.314831 13838 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:18.349216 13838 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:18.324491 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.176474 21173 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 13:07:18.115131 21000 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.173559 20773 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:18.320544 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 152049us
I20260709 13:07:18.379769 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.315500 21018 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:07:18.343950 21205 log.cc:927] Time spent T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:07:18.382619 13988 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:07:18.394114 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.343271 21024 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.216202 21099 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.301169 21143 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:18.301649 21157 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:07:18.299331 21199 consensus_queue.cc:1059] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 13:07:18.301121 21211 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:18.379000 21072 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:18.437071 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:18.279381 21210 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.195s
I20260709 13:07:18.387189 21076 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:18.339316 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.441080 13875 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:18.442045 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.402320 21018 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:18.446645 13875 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:07:18.342357 21152 consensus_queue.cc:237] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:18.285315 21166 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.020s	sys 0.067s
I20260709 13:07:18.427523 21157 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:18.382407 21159 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:18.424508 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.313813 21001 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:07:18.471336 21099 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.473905 21169 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:18.474611 21112 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.475767 21157 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.477451 21157 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:07:18.479053 21204 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:07:18.451675 21176 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.441732 21167 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:18.482705 21167 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.487277 21026 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:18.487967 21026 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.489271 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.490793 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3573) took 1506 ms. Trace:
I20260709 13:07:18.491228 13866 rpcz_store.cc:276] 0709 13:07:16.984416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.333906 (+349490us) service_pool.cc:224] Handling call
0709 13:07:17.385492 (+ 51586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.108413 (+722921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.490546 (+382133us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54000a6c54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:18.490730 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1660815232}
I20260709 13:07:18.492921 21165 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.494112 21146 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.496861 20773 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.442176 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.497671 13870 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:18.328049 20867 consensus_queue.cc:237] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.489998 13868 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 25.
I20260709 13:07:18.506779 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:18.509294 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 126738us
I20260709 13:07:18.356178 21068 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:18.438346 21072 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:18.409308 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 112006us
W20260709 13:07:18.410384 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 198201us
I20260709 13:07:18.361328 21206 consensus_queue.cc:1059] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.131s
W20260709 13:07:18.454252 21166 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.020s	sys 0.068s
I20260709 13:07:18.534902 14554 raft_consensus.cc:1240] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 13:07:18.537640 21001 consensus_queue.cc:1243] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:07:18.446939 21018 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:18.355705 21202 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.460909 21022 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.452229 21161 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:18.542114 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:18.515995 21072 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:18.545696 21202 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:18.548581 20965 consensus_meta.cc:287] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.001s
W20260709 13:07:18.549782 21149 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.929s	user 0.011s	sys 0.000s
I20260709 13:07:18.538683 21169 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:18.557459 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:18.531689 21172 consensus_meta.cc:287] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.834s	user 0.008s	sys 0.004s
I20260709 13:07:18.567090 13943 raft_consensus.cc:2410] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 13:07:18.531281 21027 consensus_meta.cc:287] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.005s
I20260709 13:07:18.485898 21167 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:18.560634 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:07:18.561565 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 54888.1us
I20260709 13:07:18.518282 21068 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:18.532744 21162 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.829s	user 0.008s	sys 0.005s
I20260709 13:07:18.570498 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:18.568425 20969 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:18.577497 20969 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.577102 21162 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.577893 20969 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:18.562951 21161 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.558883 21000 raft_consensus.cc:3055] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:07:18.587055 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 89068.4us
W20260709 13:07:18.583222 21021 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.698s	user 0.016s	sys 0.001s
I20260709 13:07:18.589054 21021 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.591504 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3391) took 1812 ms. Trace:
I20260709 13:07:18.591946 13945 rpcz_store.cc:276] 0709 13:07:16.778886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.920175 (+141289us) service_pool.cc:224] Handling call
0709 13:07:16.920437 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.888294 (+967857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.591204 (+702910us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b54000f4954. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:18.591430 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2156181888}
I20260709 13:07:18.558899 21146 consensus_queue.cc:1048] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:07:18.564214 20773 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:18.595839 20773 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:18.575137 21068 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:18.565497 20867 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:18.574638 21169 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.564307 20965 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.585644 21022 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:18.601783 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:18.605196 21169 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:18.606196 20867 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.632934 20867 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.587462 21161 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:18.634374 20607 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:18.634778 20607 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:18.636258 21193 consensus_meta.cc:287] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.720s	user 0.012s	sys 0.000s
I20260709 13:07:18.636766 21193 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.612139 21216 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.638653 21216 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.574908 21172 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.641469 21172 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.642364 13948 raft_consensus.cc:2393] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:18.642658 21159 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:18.613371 20793 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:18.643241 20793 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:18.644171 20793 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.645141 20793 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:18.615157 21021 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.616217 21010 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.616834 21162 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.617308 21146 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:18.596884 20773 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.646962 21146 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.649222 21146 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:18.618378 20902 raft_consensus.cc:3060] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:18.653638 21141 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:18.653971 21010 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.588708 21000 raft_consensus.cc:740] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:18.623961 14017 raft_consensus.cc:1240] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:18.623870 21022 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:18.624519 21207 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.626122 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:18.600030 21197 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:18.666896 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:18.629878 14015 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:07:18.630465 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:18.630998 21027 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:18.596756 20856 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:18.661994 14022 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:07:18.662577 14016 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:07:18.615231 20619 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:18.664232 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:18.664201 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:18.650724 21191 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.736s	user 0.015s	sys 0.000s
I20260709 13:07:18.629433 21217 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:18.679956 21143 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.680709 21217 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.669579 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:18.682634 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:18.684314 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:18.682384 21143 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.686683 14016 raft_consensus.cc:2368] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18.
I20260709 13:07:18.687690 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:18.691018 21022 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.695604 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:18.696237 14024 raft_consensus.cc:3060] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:18.673902 21021 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:18.698424 21021 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:18.698343 21000 consensus_queue.cc:260] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.699074 21021 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:18.699833 21000 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:18.670369 21207 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.703845 21207 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.675802 21197 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.664451 21010 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.708125 21010 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.708596 14552 raft_consensus.cc:1275] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:18.709153 21210 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.709735 21210 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.712054 21217 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.688338 14016 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:07:18.721482 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:18.724180 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:18.671327 20965 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.725500 14016 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:18.713938 21216 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.738219 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:18.738699 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:18.668746 13840 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:18.674894 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:18.744212 14022 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:18.675135 14018 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:18.680140 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:18.746723 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 78050.7us
I20260709 13:07:18.752373 14016 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:07:18.755772 13840 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:18.714962 21197 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.763568 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:18.766213 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.764918 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:18.753146 21162 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:18.766876 13874 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:18.807827 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:18.784189 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.816030 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.819314 13872 raft_consensus.cc:2410] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:07:18.819348 13866 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:18.762610 21159 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.775599 14024 raft_consensus.cc:2468] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:18.805416 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:18.837234 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:18.840500 13872 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:18.837158 21146 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.841259 21146 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.843559 21146 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.849220 13872 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:07:18.850395 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.851810 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:18.855479 21161 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.856911 14019 raft_consensus.cc:2368] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 23. Current term is 24.
I20260709 13:07:18.852722 13872 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 20: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:18.859464 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.864364 13872 raft_consensus.cc:2410] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 20 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:07:18.869985 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:18.870720 14551 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:18.870618 21159 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.872978 21159 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.877671 21099 raft_consensus.cc:3055] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:18.878191 21099 raft_consensus.cc:740] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:18.881546 21026 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:18.887058 21021 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.888705 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:18.889456 14554 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
I20260709 13:07:18.889560 21021 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:18.809047 21188 consensus_meta.cc:287] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.718s	user 0.004s	sys 0.007s
I20260709 13:07:18.893744 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.899603 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.767944 21202 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:18.915390 14023 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:18.778847 21162 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:18.910157 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1264) took 2133 ms. Trace:
I20260709 13:07:18.805418 21193 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.889453 20620 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:18.892423 14020 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:18.813647 20969 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:18.808733 21072 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:18.918090 21183 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 1.310s	user 0.005s	sys 0.002s
I20260709 13:07:18.918515 21159 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:18.920745 13944 rpcz_store.cc:276] 0709 13:07:16.776717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.876668 (+ 99951us) service_pool.cc:224] Handling call
0709 13:07:16.915061 (+ 38393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.397131 (+482070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:17.397164 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:17.397209 (+    45us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:07:17.397269 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.909784 (+1512515us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b54001c6454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:18.909858 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.910071 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1108183808}
I20260709 13:07:18.923734 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.784301 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.962953 13868 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 20: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:18.765867 21027 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.130s
I20260709 13:07:18.892066 21161 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:18.943914 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.939845 21116 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.963515 13872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.972642 13872 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:18.930682 21162 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:18.967823 13866 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:18.969971 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:18.980247 13868 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:18.939237 21001 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:18.985199 14551 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 24.
I20260709 13:07:18.958384 21018 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.998723 14017 raft_consensus.cc:1240] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:07:18.999847 14558 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:07:19.005689 14558 pending_rounds.cc:85] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting all ops after (but not including) 13
I20260709 13:07:19.001178 20969 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 27 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.965731 21159 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:18.994843 14024 raft_consensus.cc:1240] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:19.005970 14558 pending_rounds.cc:107] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:07:19.006559 14558 raft_consensus.cc:2889] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:19.010802 20998 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:19.007402 21197 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.018270 21072 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.018620 21197 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.021240 21018 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:18.989876 21001 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.023166 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:19.024055 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 1105) took 2051 ms. Trace:
I20260709 13:07:19.024470 14552 rpcz_store.cc:276] 0709 13:07:16.972910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.973111 (+   201us) service_pool.cc:224] Handling call
0709 13:07:17.073406 (+100295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.708999 (+1635593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.023756 (+314757us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54004f1854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:19.023990 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3759575296}
I20260709 13:07:19.025202 21001 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:19.025518 21033 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:19.027172 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:19.028512 21159 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:18.967774 21068 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.033280 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1263) took 2254 ms. Trace:
W20260709 13:07:19.033892 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:19.034633 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.036909 13944 raft_consensus.cc:3055] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:19.037196 13944 raft_consensus.cc:740] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:19.038748 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:18.982471 21116 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.041326 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.033700 13941 rpcz_store.cc:276] 0709 13:07:16.776455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.777483 (+  1028us) service_pool.cc:224] Handling call
0709 13:07:16.777776 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:16.777999 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:16.778023 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:16.787271 (+  9248us) log.cc:844] Serialized 26 byte log entry
0709 13:07:17.076507 (+289236us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:17.076555 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.076634 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:18.917399 (+1840765us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:19.030898 (+113499us) raft_consensus.cc:1685] finished
0709 13:07:19.030921 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.031117 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391540608,"thread_start_us":239429,"threads_started":1,"wal-append.queue_time_us":728871}
I20260709 13:07:19.045433 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.045713 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:19.053464 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.056529 13945 raft_consensus.cc:2368] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16.
I20260709 13:07:19.037693 13944 consensus_queue.cc:260] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.057932 13944 raft_consensus.cc:3060] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:19.053308 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.060408 13949 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:07:19.060818 13949 pending_rounds.cc:85] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 8
I20260709 13:07:19.060740 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.061147 13949 pending_rounds.cc:107] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:07:19.061410 13949 raft_consensus.cc:2889] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:19.062238 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:19.063462 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.059191 13942 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 22.
I20260709 13:07:19.064754 13874 raft_consensus.cc:2368] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 21. Current term is 22.
I20260709 13:07:19.068920 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.072233 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.084441 13944 raft_consensus.cc:1275] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:07:19.086738 13943 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 21: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:19.086764 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.101696 13941 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:19.055148 21118 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:19.090929 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.096095 14019 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:19.066236 21164 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:19.110291 13944 pending_rounds.cc:85] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 12
I20260709 13:07:19.110718 13944 pending_rounds.cc:107] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:07:19.111013 13944 raft_consensus.cc:2889] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:19.113230 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.109011 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.114980 21072 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:19.115460 21116 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.067619 21153 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:19.099217 21191 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.116616 21153 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.117283 21191 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.118796 21153 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:19.119591 21191 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.100018 21172 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.112454 21118 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.120276 21172 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.121816 21118 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:19.122433 21172 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:19.109676 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.127555 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:19.127876 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.128252 14017 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:19.128986 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.129622 14019 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:19.130592 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.131667 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:19.132205 14019 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:19.128543 14023 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:19.134268 14018 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:19.136018 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:19.136615 14015 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:19.141950 21235 log.cc:927] Time spent T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:19.030323 21099 consensus_queue.cc:260] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.143374 21164 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.148469 14016 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:19.151919 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.152892 13941 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 12.
I20260709 13:07:19.155661 21068 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.130795 14024 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:19.115648 21072 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.112183 21148 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:19.170367 21148 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:19.175272 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.178754 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.190260 13941 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:19.197203 13944 raft_consensus.cc:2410] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 13:07:19.199671 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.198997 21195 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.194103 20793 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.219756 21195 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.220351 20793 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.222625 20793 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.223703 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.224692 13943 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:19.204560 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:19.241058 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.246747 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.247788 13941 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:19.253518 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.254762 13943 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:19.278614 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:07:19.282341 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:19.283088 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.286640 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:19.294847 14019 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:19.296893 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:19.297760 13944 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:19.265514 21118 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:19.294412 21124 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:19.299798 21124 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.299716 21118 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.295696 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.304726 14017 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:19.295974 13988 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:19.319187 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.342612 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:19.312341 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.346426 14017 raft_consensus.cc:2410] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:07:19.342178 14552 raft_consensus.cc:3055] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:19.347004 13944 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:19.347232 14552 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:19.349080 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.320614 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.322666 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:19.351052 13875 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:07:19.322263 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.347793 21001 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.358393 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.302074 21152 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.309715 21165 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:19.360579 21001 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.364283 21148 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.364970 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:19.361140 14561 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:19.365754 13947 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:07:19.371841 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:19.388468 13945 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:19.371949 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.389424 13948 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:19.390412 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.309911 21018 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:19.368737 21159 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.342028 21195 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.311231 21027 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:19.374179 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:19.319093 21033 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:07:19.406317 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.376981 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:19.407397 13867 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 24.
I20260709 13:07:19.408159 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.409251 13945 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:19.318725 21072 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:19.372785 14552 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.382349 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.419060 13943 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:19.420282 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.420995 13943 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:07:19.355841 21161 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:19.391959 13838 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.362700 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:19.394587 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.426396 14020 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:19.399076 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.375578 14019 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:19.383790 21026 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:19.364675 21162 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.302506 21068 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.409603 14017 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 23.
I20260709 13:07:19.410513 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.397681 21022 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:19.412204 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.411269 13875 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 27.
I20260709 13:07:19.434544 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.435248 13948 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 23.
I20260709 13:07:19.436065 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.412914 14524 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.408950 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.414724 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.416433 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:19.438825 13947 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:19.423895 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.359462 21164 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:19.425814 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:19.429194 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.398288 21000 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:07:19.376346 21152 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.386598 21165 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.432009 13945 raft_consensus.cc:2410] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:07:19.440236 14024 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:19.441629 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.404044 21159 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.425043 21161 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.444250 13875 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:19.410251 21027 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.448675 13838 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:19.449245 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.450651 21021 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:19.425974 21195 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:19.431089 21202 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:19.428906 21026 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.454792 14015 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:19.455047 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:19.434553 21022 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:19.432397 21072 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:19.433190 21148 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.470638 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.471472 14015 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:19.458451 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:19.459271 13867 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:19.472344 13875 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:19.461293 13950 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:07:19.473238 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:19.473979 13875 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:19.474972 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:19.475584 13875 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:19.447901 20989 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:19.470432 14023 raft_consensus.cc:1240] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:19.481334 13871 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:19.460011 21161 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:19.477240 13947 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
I20260709 13:07:19.479022 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.457242 20872 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:07:19.471778 13868 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:19.485318 13875 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
W20260709 13:07:19.457194 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.487058 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:19.437698 21068 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.498706 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:19.497182 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:19.499467 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:19.499992 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:19.500752 13915 connection.cc:256] RPC call timeout handler was delayed by -1.0405827069999987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
I20260709 13:07:19.499722 13868 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:19.470678 21026 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:19.480362 21022 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:19.499868 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 203986us
I20260709 13:07:19.507365 13988 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:19.503723 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.494706 21202 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:19.470436 21021 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:19.481851 20989 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:19.501358 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 179158us
I20260709 13:07:19.517452 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.478613 21072 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:19.478441 21116 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:19.525051 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:19.525564 13870 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 16. (term mismatch)
I20260709 13:07:19.526001 13870 pending_rounds.cc:85] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Aborting all ops after (but not including) 15
I20260709 13:07:19.512074 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.513209 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.519815 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.526255 13870 pending_rounds.cc:107] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Aborting uncommitted NO_OP operation due to leader change: 18.16
I20260709 13:07:19.527818 13870 raft_consensus.cc:2889] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:19.528379 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:19.532168 14019 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:19.533864 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:19.535287 21243 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:19.543807 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:19.545221 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:19.545861 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.470268 21195 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.547382 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.506036 20872 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:19.561578 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.502451 20620 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:19.557718 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:19.482137 21148 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.552757 13873 raft_consensus.cc:1240] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:19.510565 21202 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:19.563930 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:19.522975 20989 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:19.524266 21072 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:19.519027 21242 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:19.531503 20998 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:07:19.574144 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45879.4us
I20260709 13:07:19.575641 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:07:19.578493 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 129862us
I20260709 13:07:19.551637 20961 raft_consensus.cc:2764] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:07:19.580823 13838 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.558990 21141 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:19.584259 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.522576 21033 consensus_queue.cc:1048] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:07:19.641458 14019 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:19.650939 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:19.578413 20998 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:19.638954 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 53151.1us
I20260709 13:07:19.592878 13867 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:19.654505 20998 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:19.656282 13867 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:07:19.681480 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 100733us
I20260709 13:07:19.682760 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:19.684219 13838 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:19.685734 13941 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:19.571384 21172 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:19.573364 21202 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:19.692307 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 161155us
I20260709 13:07:19.569767 21243 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:19.544608 21116 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.579176 21118 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:19.531023 21021 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:19.698796 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 123232us
I20260709 13:07:19.699841 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:19.706943 14019 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:19.580624 21001 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.585166 21162 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.704149 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.650240 21141 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:19.574438 20620 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.710426 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.716909 14015 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:19.695204 21172 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.714011 13989 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.626217 21191 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
W20260709 13:07:19.709537 21239 log.cc:927] Time spent T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 13:07:19.729423 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.714030 21141 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:19.704085 20872 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.736454 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.737398 14015 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:19.711036 21018 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:19.717084 21000 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:19.720880 21001 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:19.724995 21172 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:19.724517 21162 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:19.705542 20793 consensus_queue.cc:1059] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:07:19.709249 20961 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
W20260709 13:07:19.758543 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.754160 14015 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:07:19.759060 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.761799 14019 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:19.741480 21198 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:19.741901 21027 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.742702 21000 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.765066 21253 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:07:19.770730 13840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.697s (SENT) [suppressed 11 similar messages]
I20260709 13:07:19.771083 21253 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.771466 21253 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:19.745481 21001 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:19.753028 21252 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:19.756296 21162 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:19.790010 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.782049 14021 raft_consensus.cc:1240] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:07:19.754628 21165 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.785452 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:19.777868 21240 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 13:07:19.786329 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:19.801365 14018 raft_consensus.cc:2393] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 21: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:19.798286 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 84340.6us
I20260709 13:07:19.767062 21198 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.778308 21000 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:19.811949 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.814708 14015 raft_consensus.cc:2410] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:07:19.789260 21001 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:19.800657 21244 log.cc:927] Time spent T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 13:07:19.796785 21162 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:19.826705 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.818928 20998 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.819293 21026 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.820508 21021 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:19.789484 21068 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:19.823527 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.823608 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:19.817580 21256 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.792218 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:19.825349 21195 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:19.815413 21148 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.827565 14021 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:19.832892 14524 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:19.832700 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.813442 21198 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:19.846151 14019 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:19.838492 21242 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:19.769443 21243 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:19.870007 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 165963us
I20260709 13:07:19.871922 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.874774 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:19.876458 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.843371 21166 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:07:19.879666 13840 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:19.884315 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.848997 21256 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.858096 21252 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:19.864573 21245 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:19.885053 13948 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:19.865970 21159 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.882200 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.890668 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:19.880419 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:19.890322 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.892323 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:19.880316 21148 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:19.893846 21148 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.894238 21148 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:19.895285 21256 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.878242 20620 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:19.889541 21252 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.892746 21165 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:19.925014 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 240838us
I20260709 13:07:19.908974 20989 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.929100 21165 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:19.929857 21165 consensus_queue.cc:1243] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [LEADER]: Peer d1b4fd04200247d99ba9ad1fbeb42b42 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:07:19.911810 21197 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.885601 21021 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:19.886214 21195 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:19.931288 21195 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.931718 21195 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:19.931838 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:19.932777 21165 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:19.933683 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:19.934387 13944 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:07:19.918608 21202 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.936702 21202 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:19.923349 20998 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:19.938499 20989 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:19.902496 21252 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:19.913185 21161 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.940743 21164 raft_consensus.cc:3055] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:19.941190 21164 raft_consensus.cc:740] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:19.907804 13950 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 3147) took 1709 ms (client timeout 1697 ms). Trace:
W20260709 13:07:19.942584 13950 rpcz_store.cc:269] 0709 13:07:18.198913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.202575 (+  3662us) service_pool.cc:224] Handling call
0709 13:07:19.907584 (+1705009us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54003b01d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:19.907699 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2891651968}
I20260709 13:07:17.516897 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:19.912976 21026 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:19.930935 21021 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 21 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:19.945945 21164 consensus_queue.cc:260] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.946352 21021 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.947072 21164 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:19.950623 14016 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:19.951575 13944 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 25.
I20260709 13:07:19.952880 14522 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:19.957139 13840 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:19.964162 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 110410us
I20260709 13:07:19.965021 21202 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:19.883450 13915 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.961248 13838 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.969892 21027 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:19.970666 21027 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:19.971987 21027 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:19.977924 13838 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.979302 14019 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
I20260709 13:07:19.962785 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:19.967537 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:19.982466 13950 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:07:19.982877 13838 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:19.983238 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:19.962409 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:19.968330 13867 raft_consensus.cc:1240] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:19.983976 13875 raft_consensus.cc:3060] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:19.967939 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:19.985057 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:19.987584 13944 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 17: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:19.979523 21165 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:19.988611 21165 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.989094 21165 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:19.991298 13950 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 24.
I20260709 13:07:19.995255 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:19.996428 21249 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:19.997228 21249 consensus_queue.cc:1059] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:07:19.998967 21245 raft_consensus.cc:3055] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:19.999480 21245 raft_consensus.cc:740] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:20.000595 21245 consensus_queue.cc:260] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:20.001847 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:20.002292 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:20.005080 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:20.006625 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:20.006765 13949 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:20.002442 21245 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:20.010638 13944 raft_consensus.cc:2368] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 21. Current term is 22.
I20260709 13:07:19.885772 21068 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:07:20.014693 13838 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:20.063522 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 61739.5us
I20260709 13:07:19.904299 21210 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:19.995210 13875 raft_consensus.cc:2468] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 25.
I20260709 13:07:20.061035 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:20.064795 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 96031.1us
W20260709 13:07:20.064945 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 181570us
I20260709 13:07:20.125850 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:20.066717 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:20.032481 13944 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:19.962757 20867 raft_consensus.cc:2804] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:19.959743 21161 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:20.123912 13946 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
W20260709 13:07:20.184302 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 59688.7us
I20260709 13:07:20.197438 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:20.198731 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:20.207027 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.901852 21141 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:19.912789 21172 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.308425 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:20.311030 13869 raft_consensus.cc:1240] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:20.312780 13869 raft_consensus.cc:1240] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
W20260709 13:07:20.313109 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 119132us
W20260709 13:07:20.314117 13915 leader_election.cc:336] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.626s (SENT)
I20260709 13:07:20.314566 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:20.315404 20931 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:20.321240 14561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59630 (request call id 940) took 1008 ms. Trace:
I20260709 13:07:19.917886 21148 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.321880 14561 rpcz_store.cc:276] 0709 13:07:19.313272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.313445 (+   173us) service_pool.cc:224] Handling call
0709 13:07:20.320987 (+1007542us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b54001b3854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:20.321150 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34208640}
I20260709 13:07:20.065614 21246 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:20.008769 21159 consensus_queue.cc:1059] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.182s
I20260709 13:07:19.908310 21022 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:20.165283 21256 raft_consensus.cc:2804] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:07:20.350589 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.353461 13874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3480) took 1330 ms. Trace:
W20260709 13:07:20.299571 21232 log.cc:927] Time spent T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.254s	user 0.000s	sys 0.000s
I20260709 13:07:20.359431 13874 rpcz_store.cc:276] 0709 13:07:19.022726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.022852 (+   126us) service_pool.cc:224] Handling call
0709 13:07:20.353224 (+1330372us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b540026d9d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:20.353393 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93684992}
I20260709 13:07:20.013294 21001 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:20.374179 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.182260 21247 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:20.190003 21257 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:19.919010 21242 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:20.363327 20961 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:07:20.395084 14020 raft_consensus.cc:1240] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:20.182946 21198 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.418377 13867 raft_consensus.cc:1240] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:20.299510 20620 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 13:07:20.319679 21262 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.102s
W20260709 13:07:20.476161 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 290845us
I20260709 13:07:20.525630 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3415) took 1780 ms. Trace:
I20260709 13:07:20.394495 21159 consensus_queue.cc:1243] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [NON_LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:07:20.416499 21000 consensus_meta.cc:287] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.591s	user 0.014s	sys 0.003s
W20260709 13:07:20.424647 21072 consensus_meta.cc:287] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.793s	user 0.011s	sys 0.011s
I20260709 13:07:20.382390 21265 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:20.400885 21027 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:20.541507 13941 raft_consensus.cc:2368] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21.
I20260709 13:07:20.526275 14022 rpcz_store.cc:276] 0709 13:07:18.745272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.819484 (+ 74212us) service_pool.cc:224] Handling call
0709 13:07:18.819869 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.837213 (+ 17344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.837252 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.862786 (+ 25534us) log.cc:844] Serialized 26 byte log entry
0709 13:07:18.995602 (+132816us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:19.035713 (+ 40111us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.036082 (+   369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.036212 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.949045 (+912833us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b1800487c00. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:07:20.427380 (+478335us) raft_consensus.cc:1685] finished
0709 13:07:20.524618 (+ 97238us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.525557 (+   939us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16852,"prepare.queue_time_us":45004,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":17355,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":18220,"spinlock_wait_cycles":29714176,"thread_start_us":116840,"threads_started":1,"wal-append.queue_time_us":174842}
W20260709 13:07:20.554389 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 539722us
I20260709 13:07:20.428161 21247 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:20.396701 21246 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 22 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:20.560956 13838 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:20.576210 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 221892us
I20260709 13:07:20.578121 21256 raft_consensus.cc:697] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:20.544167 20931 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.586311 21259 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.595464 21259 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:20.470454 21257 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.125s
W20260709 13:07:20.584789 21253 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.813s	user 0.009s	sys 0.026s
I20260709 13:07:20.579402 21265 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:20.606163 20931 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.614650 21246 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.578925 21247 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.640950 20931 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:20.616981 21172 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:20.621711 21141 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 27 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:20.646754 21247 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:20.621169 21256 consensus_queue.cc:237] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.617420 21148 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:20.625574 21275 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
W20260709 13:07:20.626515 21271 log.cc:927] Time spent T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 13:07:20.606541 21068 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:20.628531 21253 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.635061 21265 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.641482 21001 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:20.623485 21259 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:20.563023 21000 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.570889 21072 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:20.656137 21165 consensus_meta.cc:287] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.003s
I20260709 13:07:20.662099 21022 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:20.657210 21068 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.664330 21068 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:20.664397 21269 log.cc:927] Time spent T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:07:20.664183 21165 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.651908 20867 raft_consensus.cc:697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:20.671523 20867 consensus_queue.cc:237] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:20.671729 20607 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.680123 20607 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:20.673825 21162 consensus_meta.cc:287] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.830s	user 0.014s	sys 0.009s
I20260709 13:07:20.673524 21068 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.675757 21072 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:20.676404 21195 consensus_meta.cc:287] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.744s	user 0.006s	sys 0.006s
W20260709 13:07:20.692967 21164 consensus_meta.cc:287] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.745s	user 0.008s	sys 0.003s
I20260709 13:07:20.693636 21099 raft_consensus.cc:3060] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:20.670528 21165 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:20.680491 21245 consensus_meta.cc:287] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.003s
I20260709 13:07:20.678279 21000 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:20.691241 21068 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:20.691752 21162 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:20.692565 21116 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:20.693259 21195 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.690986 21253 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:20.707165 21198 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:20.712320 21195 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:20.754547 13840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.006s	sys 0.005s
W20260709 13:07:20.755729 13840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.007s	sys 0.006s
W20260709 13:07:20.758944 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 197872us
W20260709 13:07:20.761021 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 153766us
I20260709 13:07:20.766944 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:20.771282 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:20.774557 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 707868us
I20260709 13:07:20.777415 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:20.779701 13948 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:20.780360 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:20.780845 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:20.781162 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:20.782393 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:20.783561 13947 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 21: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:20.782629 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:20.784706 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:20.785622 13947 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:20.786830 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:20.787695 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:20.792665 13946 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:07:20.797518 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.788187 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:20.805482 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:20.824079 21248 log.cc:927] Time spent T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 1.293s	user 0.004s	sys 0.005s
I20260709 13:07:20.824839 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:20.827080 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:20.830904 21162 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:20.830989 21149 consensus_queue.cc:1059] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:07:20.834400 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.838652 21195 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:20.717386 21252 consensus_meta.cc:287] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.778s	user 0.009s	sys 0.004s
I20260709 13:07:20.840775 21252 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.841782 21246 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.842869 21238 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:20.842742 21246 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.843417 21238 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:20.844905 21238 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.846346 21246 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:20.847414 21162 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:07:20.847672 21021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.009s	sys 0.005s
I20260709 13:07:20.851691 21288 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.858125 21288 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:20.859239 21021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.010s	sys 0.005s
I20260709 13:07:20.862238 21256 raft_consensus.cc:3055] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:20.863024 21256 raft_consensus.cc:740] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:20.863247 21162 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.864508 21256 consensus_queue.cc:260] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.865118 21252 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:20.866339 21162 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:20.860948 13947 raft_consensus.cc:2410] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:07:20.867098 21256 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:20.867534 21288 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:20.873515 13866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3594) took 1429 ms. Trace:
I20260709 13:07:20.874918 13949 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:20.877317 13870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42526 (request call id 1067) took 1514 ms. Trace:
I20260709 13:07:20.877803 13942 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:20.877761 13870 rpcz_store.cc:276] 0709 13:07:19.362459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.363443 (+   984us) service_pool.cc:224] Handling call
0709 13:07:19.385312 (+ 21869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.528311 (+142999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.876775 (+1348464us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54000d9b54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:20.877125 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321701632}
I20260709 13:07:20.880117 13942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3423) took 1535 ms. Trace:
I20260709 13:07:20.880502 13942 rpcz_store.cc:276] 0709 13:07:19.344780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.345067 (+   287us) service_pool.cc:224] Handling call
0709 13:07:20.879863 (+1534796us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54001482d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:20.880034 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3491865984}
I20260709 13:07:20.880909 13871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41684 (request call id 3473) took 1982 ms. Trace:
I20260709 13:07:20.881985 21191 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.882766 21191 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:20.888383 21149 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:20.888392 13941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3431) took 1008 ms. Trace:
I20260709 13:07:20.890440 13941 rpcz_store.cc:276] 0709 13:07:19.880029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.880162 (+   133us) service_pool.cc:224] Handling call
0709 13:07:20.887875 (+1007713us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b5400147dd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:20.888157 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1494466432}
I20260709 13:07:20.881363 13871 rpcz_store.cc:276] 0709 13:07:18.898889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.899274 (+   385us) service_pool.cc:224] Handling call
0709 13:07:20.880363 (+1981089us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b54000da554. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:20.880637 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1335763328}
I20260709 13:07:20.860073 21238 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:20.874006 13866 rpcz_store.cc:276] 0709 13:07:19.444264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.445826 (+  1562us) service_pool.cc:224] Handling call
0709 13:07:19.459026 (+ 13200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.465508 (+  6482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.465544 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:19.465588 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:19.465622 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.872942 (+1407320us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b54003abbd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:20.873017 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.873276 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14436224}
W20260709 13:07:20.935576 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 131728us
I20260709 13:07:20.937175 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:20.940850 13866 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 27 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:07:20.967461 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:20.968797 21288 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:20.970495 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:20.971550 13941 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:20.976742 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 64101.2us
I20260709 13:07:20.978322 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:20.979528 13942 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 21: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:20.980065 20867 consensus_queue.cc:1059] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:07:20.982214 21291 raft_consensus.cc:3055] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:20.984409 21291 raft_consensus.cc:740] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:20.988153 21188 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:20.988054 21291 consensus_queue.cc:260] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:20.988729 21264 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:07:20.989494 21291 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:20.990469 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:20.991894 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:20.992276 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.992707 13947 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:20.993736 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:21.000640 14017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2846) took 1652 ms (client timeout 1725 ms). Trace:
I20260709 13:07:21.007167 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2842) took 1877 ms. Trace:
I20260709 13:07:21.007671 14023 rpcz_store.cc:276] 0709 13:07:19.129688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.134872 (+  5184us) service_pool.cc:224] Handling call
0709 13:07:19.394319 (+259447us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:07:19.468663 (+ 74344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.480863 (+ 12200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.006665 (+1525802us) spinlock_profiling.cc:243] Waited 74.1 ms on lock 0x7b54002990d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34328ad 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:21.006889 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183864448}
W20260709 13:07:21.012733 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 75848.6us
I20260709 13:07:21.026937 14558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59630 (request call id 932) took 2289 ms. Trace:
I20260709 13:07:21.032222 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:21.014099 13838 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:21.029896 20989 consensus_queue.cc:1175] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" responder_term: 15 status { last_received { term: 15 index: 13 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:20.974745 13915 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:21.039016 21152 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:21.040218 21191 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:21.041260 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:21.040079 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:21.042279 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:21.042037 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3441) took 1037 ms. Trace:
I20260709 13:07:21.032544 14558 rpcz_store.cc:276] 0709 13:07:18.737339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.752855 (+ 15516us) service_pool.cc:224] Handling call
0709 13:07:18.780922 (+ 28067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.007057 (+226135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.016779 (+2009722us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54003359d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:21.026691 (+  9912us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481201536}
W20260709 13:07:21.001369 14017 rpcz_store.cc:269] 0709 13:07:19.348268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.348824 (+   556us) service_pool.cc:224] Handling call
0709 13:07:21.000023 (+1651199us) spinlock_profiling.cc:243] Waited 59.8 ms on lock 0x7b54000ac154. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:21.000299 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137507584}
I20260709 13:07:21.046502 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:21.044268 13948 rpcz_store.cc:276] 0709 13:07:20.004432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.004571 (+   139us) service_pool.cc:224] Handling call
0709 13:07:21.041806 (+1037235us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b5400085f54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:21.041973 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1777533824}
W20260709 13:07:21.052623 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.056046 13867 raft_consensus.cc:1240] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:21.058542 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:21.057879 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:21.059749 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3439) took 1056 ms. Trace:
I20260709 13:07:21.060153 13950 rpcz_store.cc:276] 0709 13:07:20.003756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.004588 (+   832us) service_pool.cc:224] Handling call
0709 13:07:21.057490 (+1052902us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b5800202e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030ee9c 00007fb2c030ea5b 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c83f90a0 00007fb2c83f8984 00007fb2c19736d4 00007fb2c1973662
0709 13:07:21.059676 (+  2186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65292928}
I20260709 13:07:21.063527 21264 consensus_queue.cc:1059] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:21.064872 21264 raft_consensus.cc:3055] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:21.065271 21264 raft_consensus.cc:740] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:07:21.066509 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52666.4us
I20260709 13:07:21.067142 14551 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:21.068437 13838 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:21.072002 13949 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:21.072360 21288 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:21.073487 21297 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:21.075579 21162 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:21.076114 21162 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:21.076648 21162 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:21.079342 21288 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 21 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:21.080910 21288 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:21.085718 21273 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.541s	user 0.004s	sys 0.000s
I20260709 13:07:21.087049 13872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41676 (request call id 3596) took 1642 ms. Trace:
I20260709 13:07:21.087620 13872 rpcz_store.cc:276] 0709 13:07:19.444807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.444928 (+   121us) service_pool.cc:224] Handling call
0709 13:07:19.445200 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.491344 (+ 46144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.491393 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:19.491454 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:19.491510 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.086404 (+1594894us) spinlock_profiling.cc:243] Waited 45.9 ms on lock 0x7b540026cfd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:21.086518 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.086811 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105531264}
I20260709 13:07:21.093410 21264 consensus_queue.cc:260] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:21.094709 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49628 (request call id 2851) took 1686 ms. Trace:
I20260709 13:07:21.095235 14016 rpcz_store.cc:276] 0709 13:07:19.408550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.408716 (+   166us) service_pool.cc:224] Handling call
0709 13:07:21.094229 (+1685513us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b54002995d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:21.094505 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177539840}
I20260709 13:07:21.119431 21264 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:21.107517 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:21.130124 21297 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:21.130977 21297 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.131422 21297 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:21.138675 21300 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:21.139789 21300 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.051963 13840 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:21.152199 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.161860 13987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.014s	sys 0.007s
W20260709 13:07:21.175285 13987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.015s	sys 0.008s
W20260709 13:07:21.170117 13839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.007s	sys 0.014s
W20260709 13:07:21.180132 13839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.008s	sys 0.015s
W20260709 13:07:21.182512 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 48281.5us
I20260709 13:07:21.186573 13944 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 26. Current term is 27. Ops: []
I20260709 13:07:21.186895 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2863) took 1611 ms. Trace:
I20260709 13:07:21.187336 14020 rpcz_store.cc:276] 0709 13:07:19.575841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.576082 (+   241us) service_pool.cc:224] Handling call
0709 13:07:19.647620 (+ 71538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.424351 (+776731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.186335 (+761984us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b5400325fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:21.186694 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1709482752}
I20260709 13:07:21.185864 21124 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:21.206359 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:21.212234 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 285475us
I20260709 13:07:21.217795 13948 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:07:21.242915 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.243536 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.244431 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.244894 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.246486 21288 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:21.248274 21152 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.248994 21152 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:21.252574 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.254274 21026 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:21.255072 21026 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:21.259068 21297 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.261626 21297 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:21.264478 13947 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:21.266253 21165 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:07:21.275142 21099 consensus_meta.cc:287] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.581s	user 0.007s	sys 0.010s
I20260709 13:07:21.278066 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:07:21.278607 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 245400us
I20260709 13:07:21.279160 21246 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:21.279541 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:21.279697 21246 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:21.280279 21246 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:21.281332 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:21.282840 21152 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:21.227746 21191 raft_consensus.cc:2804] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:21.287273 13950 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:21.296442 13941 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:07:21.228416 21124 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.039s
I20260709 13:07:21.305761 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1286) took 1577 ms. Trace:
I20260709 13:07:21.306213 21265 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.299335 21288 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:21.306845 21265 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.307713 21288 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.308259 21288 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:21.306484 13945 rpcz_store.cc:276] 0709 13:07:19.728467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.728629 (+   162us) service_pool.cc:224] Handling call
0709 13:07:19.728961 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.732675 (+  3714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.732709 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:19.732756 (+    47us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:19.732797 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.299205 (+1566408us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b5400086454. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:21.305078 (+  5873us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.305377 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8033536}
I20260709 13:07:21.313949 14551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59630 (request call id 944) took 1309 ms. Trace:
I20260709 13:07:21.312922 14559 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:21.314384 14551 rpcz_store.cc:276] 0709 13:07:20.001808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.001976 (+   168us) service_pool.cc:224] Handling call
0709 13:07:21.293924 (+1291948us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b54004b0fd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:21.310776 (+ 16852us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1856857856}
I20260709 13:07:21.310496 21165 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:21.302484 21191 raft_consensus.cc:697] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:21.308360 21018 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:07:21.330269 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:21.332231 13987 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:21.334616 13987 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:21.342506 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 61284.4us
I20260709 13:07:21.344228 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:21.342725 21252 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:21.346195 21252 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:21.349961 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:21.324990 21165 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:21.355705 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 77870.9us
I20260709 13:07:21.342532 21242 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:21.328043 21026 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:21.367623 14522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.367812 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:21.354611 21165 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:21.387796 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:21.350467 21099 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:07:21.394712 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:21.401281 14552 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:21.406205 13945 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:21.406787 13871 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:07:21.406582 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:21.378419 21265 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:21.401377 21300 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:21.415460 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:21.416841 13941 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:21.409549 13839 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:21.425951 21068 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:07:21.427107 21068 consensus_queue.cc:1243] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [LEADER]: Peer 4dd265d61fc141a48ca9d3d5c8f52921 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:07:21.429767 13942 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:07:21.430370 13942 pending_rounds.cc:85] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Aborting all ops after (but not including) 11
I20260709 13:07:21.430874 13942 pending_rounds.cc:107] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:07:21.425951 21288 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.431603 13942 raft_consensus.cc:2889] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:21.433288 21288 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:21.436141 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:07:21.438834 21309 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:07:21.430402 21166 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:07:21.439442 21166 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:21.436959 21191 consensus_queue.cc:237] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.445063 21166 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:21.447005 21166 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:21.455840 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1277) took 1981 ms. Trace:
I20260709 13:07:21.457899 21124 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:07:21.456346 13943 rpcz_store.cc:276] 0709 13:07:19.430908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.437622 (+  6714us) service_pool.cc:224] Handling call
0709 13:07:19.437882 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.581102 (+143220us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:19.763659 (+182557us) log.cc:844] Serialized 14 byte log entry
0709 13:07:20.891433 (+1127774us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b180023ff20. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd323a 00007fb2c343da2a
0709 13:07:20.978807 (+ 87374us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:20.978927 (+   120us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:20.979030 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.397888 (+418858us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54000b9d54. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:21.397980 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.411816 (+ 13836us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11065,"prepare.queue_time_us":195648,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":2755,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":192,"spinlock_wait_cycles":384056832,"thread_start_us":132706,"threads_started":2,"wal-append.queue_time_us":1216329}
W20260709 13:07:21.476862 21306 log.cc:927] Time spent T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:07:21.477131 21246 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:21.478564 14552 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:21.483345 13838 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:21.488730 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 89943.5us
I20260709 13:07:21.489744 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:21.490451 14552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59650 (request call id 1123) took 2176 ms. Trace:
I20260709 13:07:21.490943 14552 rpcz_store.cc:276] 0709 13:07:19.314299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.314423 (+   124us) service_pool.cc:224] Handling call
0709 13:07:19.342005 (+ 27582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:21.028179 (+1686174us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b180030d140. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c346e646 00007fb2c33a858e 00007fb2c3429f9e 00007fb2c342b7de 00007fb2c3435b49 00007fb2c3436416 00007fb2c3432d8f 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4
0709 13:07:21.392672 (+364493us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b58008366d0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c338f0aa 00007fb2c3411447 00007fb2c3429fe7 00007fb2c342b7de 00007fb2c3435b49 00007fb2c3436416 00007fb2c3432d8f 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4
0709 13:07:21.479012 (+ 86340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.490364 (+ 11352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4282588672}
I20260709 13:07:21.491634 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:21.492944 21068 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:21.496511 14551 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:21.465507 21288 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:21.503338 13947 raft_consensus.cc:1275] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:07:21.508363 21099 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:21.525107 13838 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:21.480065 21162 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:21.538255 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.540364 13941 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
I20260709 13:07:21.529639 21099 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.526382 21297 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:21.556043 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 64502.1us
I20260709 13:07:21.563642 21162 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:21.542135 21288 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 21 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:21.551540 21099 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:21.582613 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 238507us
I20260709 13:07:21.588832 21162 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:21.588989 13987 leader_election.cc:400] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16.
I20260709 13:07:21.589581 13987 leader_election.cc:403] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:21.533186 21018 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
W20260709 13:07:21.592873 21213 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:07:21.587311 13914 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:21.594403 13989 leader_election.cc:400] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 21. Current term is 22.
I20260709 13:07:21.594813 13989 leader_election.cc:403] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:07:21.594480 20969 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.596588 20969 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:21.594508 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 158256us
I20260709 13:07:21.589488 14553 pending_rounds.cc:85] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting all ops after (but not including) 12
I20260709 13:07:21.605089 14553 pending_rounds.cc:107] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:07:21.605405 14553 raft_consensus.cc:2889] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:21.596240 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:21.552629 21291 consensus_queue.cc:1048] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 13:07:21.589624 21162 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:21.609674 21149 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:21.620810 21300 raft_consensus.cc:3060] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:07:21.633941 21149 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:21.639497 21000 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.639851 21288 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:21.625164 21018 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer e1649107c781428692458f10597c5742 (127.12.109.65:43607) with higher term. Message: Invalid argument: T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 21. Current term is 22.
W20260709 13:07:21.650266 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 61542.4us
I20260709 13:07:21.643080 21149 raft_consensus.cc:3060] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:21.652372 13867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:21.659617 20969 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:21.660727 13868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:21.646131 21246 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:21.638306 21021 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.664819 13873 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:21.666261 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:21.667070 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:07:21.668937 21314 log.cc:927] Time spent T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:07:21.670226 21162 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.672529 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:21.673276 13875 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:21.657088 21165 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.678318 21238 consensus_queue.cc:1059] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:21.679677 21256 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:21.680572 21165 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:21.684808 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.666431 21316 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:21.687362 13987 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:21.666265 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:07:21.723624 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 213938us
I20260709 13:07:21.717805 13871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:21.732455 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:21.743801 13875 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:21.687120 21316 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.741283 13866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:21.650216 21000 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.740983 13947 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:07:21.731339 21246 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:07:21.755297 13866 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:21.770998 13838 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:21.732836 21242 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:21.783864 13871 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 21: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:21.785104 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:21.687347 21021 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.785573 13867 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:21.752466 21316 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:21.796504 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:21.797479 13875 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:21.798676 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
W20260709 13:07:21.798803 21318 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:07:21.800479 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:21.803023 13868 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:21.805335 13867 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 22.
I20260709 13:07:21.807205 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:21.811261 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 146647us
W20260709 13:07:21.816345 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 94885.2us
I20260709 13:07:21.786367 21068 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:21.786948 21256 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:21.825984 13871 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:21.831305 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:21.824402 14522 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:21.808020 21252 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:21.831331 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:21.838701 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:21.839329 13941 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:21.851223 21238 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:21.835764 21068 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 22 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:21.864282 21321 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.103s
I20260709 13:07:21.866597 21256 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:21.868808 21252 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.869490 21256 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:21.870709 21324 log.cc:927] Time spent T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:07:21.871575 21252 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:21.877238 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.880312 13941 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:21.841454 21242 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:07:21.889472 13871 raft_consensus.cc:1240] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:21.894783 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:21.896960 13840 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.697s (SENT)
I20260709 13:07:21.899919 21170 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:21.900506 21170 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:21.900897 21170 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:21.902932 13868 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 22.
I20260709 13:07:21.909019 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:21.909974 13870 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:21.910356 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 56090.9us
I20260709 13:07:21.910981 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:21.912196 13987 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:21.912380 13870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:21.913004 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:21.913872 13874 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 21: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:21.914033 13870 raft_consensus.cc:1240] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:21.915135 13874 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:21.920220 20998 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:21.920462 20793 raft_consensus.cc:3055] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:21.920850 20793 raft_consensus.cc:740] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:21.925462 21021 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:21.933173 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:21.883344 21238 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:21.936805 21238 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:21.937511 21000 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:21.936800 21316 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.913863 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:21.938642 13869 raft_consensus.cc:2410] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
W20260709 13:07:21.939209 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.941800 13866 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:21.948813 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:21.949551 13945 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
W20260709 13:07:21.949793 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 63214.2us
I20260709 13:07:21.929777 21068 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.950976 13989 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:21.951251 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:21.951406 21149 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:21.952039 14559 raft_consensus.cc:2393] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 21: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:21.945654 13839 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:07:21.946189 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 94958.2us
I20260709 13:07:21.929540 21099 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:21.955941 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:21.962749 13871 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:07:21.964857 13948 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:21.974705 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:21.976390 21342 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:21.976260 13989 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:21.979471 21342 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:21.980014 21342 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:21.980486 21342 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:21.968892 21026 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:21.969656 13838 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:21.998673 13872 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 19. Current term is 21. Ops: []
I20260709 13:07:21.958945 21322 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.004583 13838 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:22.005043 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:21.955885 21337 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:22.008257 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 111357us
W20260709 13:07:22.008512 21166 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.561s	user 0.011s	sys 0.003s
I20260709 13:07:22.010464 13941 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 22: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:22.010890 21000 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.012633 21242 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:21.973222 21265 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:22.004964 21288 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:22.022006 21000 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:22.027369 21337 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:22.027429 21000 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:22.031111 21242 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:22.038872 21000 raft_consensus.cc:2764] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:07:22.039793 21000 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:22.039013 21149 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:22.045189 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 50636.7us
I20260709 13:07:21.921960 20793 consensus_queue.cc:260] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:22.055104 20793 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:22.030041 21337 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:22.064181 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:22.076274 13941 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:22.022297 21288 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:07:22.069080 13840 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.725s (SENT)
I20260709 13:07:22.081141 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:22.080029 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.052502 21265 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:22.010908 21170 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:22.032260 21242 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:22.087509 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 111344us
I20260709 13:07:22.055881 21322 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:22.086196 21341 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:22.104378 14553 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:22.113317 21316 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:22.115046 21300 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16.
I20260709 13:07:22.117275 21341 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:22.092188 21321 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:07:22.120283 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 206523us
I20260709 13:07:22.137750 21170 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:22.138345 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:22.140306 13987 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:22.141325 21018 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:22.141750 21018 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:22.142182 21018 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:22.143078 21344 consensus_queue.cc:1048] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:22.144615 21344 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:22.146456 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:22.124600 21265 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:22.126577 21099 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:22.127370 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 65631.8us
I20260709 13:07:22.129086 21316 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:22.129841 21288 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:22.135583 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.119639 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:22.123965 21322 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:22.160029 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:22.162063 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:22.163662 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:22.196430 13947 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:22.170737 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.174993 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:22.175336 13869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742"
I20260709 13:07:22.175020 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
I20260709 13:07:22.175913 13868 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:22.197913 13874 raft_consensus.cc:2393] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:22.176753 13866 raft_consensus.cc:1240] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:22.186879 13838 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:22.195794 13949 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:22.195103 21342 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:22.211120 13840 leader_election.cc:336] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.809s (SENT)
I20260709 13:07:22.211696 13840 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:22.212935 21342 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:22.216817 21341 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:22.219256 13875 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:22.222509 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:22.226850 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:22.230227 21316 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:22.230489 13949 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:22.232587 21316 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:22.233143 21300 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:22.233596 21300 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:22.234007 21300 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:22.238442 13869 raft_consensus.cc:2393] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 20: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
W20260709 13:07:22.245710 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 99369.4us
I20260709 13:07:22.246819 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:22.254715 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 159034us
I20260709 13:07:22.256496 13989 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:22.256767 21345 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.257637 21344 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:22.257428 21345 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:22.258368 21344 consensus_queue.cc:1059] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:22.259246 21344 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:22.259869 21344 raft_consensus.cc:3055] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:22.260202 21344 raft_consensus.cc:740] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:22.260439 21238 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:22.261161 21344 consensus_queue.cc:260] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:22.262042 21345 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:22.267714 14552 raft_consensus.cc:2468] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
W20260709 13:07:22.268486 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 115343us
I20260709 13:07:22.275441 13838 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:22.280217 21297 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:22.280941 21297 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 21 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:22.284663 13914 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:22.285934 21341 raft_consensus.cc:2804] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:22.286623 21341 raft_consensus.cc:697] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:22.287781 21341 consensus_queue.cc:237] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:22.199671 20998 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:22.269254 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 46858us
I20260709 13:07:22.270254 21322 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:22.292954 21322 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.270495 21346 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:07:22.293502 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46822.2us
I20260709 13:07:22.269759 21321 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:22.294817 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:22.296032 21026 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:22.296684 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:22.296983 21026 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:22.263574 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:07:22.292202 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 81140.4us
I20260709 13:07:22.270778 21238 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:22.298418 21342 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:22.293579 21322 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:22.364542 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 108099us
I20260709 13:07:22.360150 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e1649107c781428692458f10597c5742" is_pre_election: true
W20260709 13:07:22.367938 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:22.364452 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.326588 21334 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:07:22.357661 21342 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:22.386008 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 167525us
I20260709 13:07:22.366153 21345 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:07:22.358011 21238 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:22.454392 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 254058us
W20260709 13:07:22.456084 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 63250.4us
W20260709 13:07:22.458295 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 80607.9us
I20260709 13:07:22.458875 21331 consensus_queue.cc:1059] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:22.459157 21334 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:22.462213 13915 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:22.467469 21238 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:22.469461 21344 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:22.410151 21342 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:22.479233 13947 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
W20260709 13:07:22.499229 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 173757us
W20260709 13:07:22.524770 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:22.527164 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 68307.3us
I20260709 13:07:22.505064 14522 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:22.319689 21297 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:22.532688 13987 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:22.548425 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 43502.6us
W20260709 13:07:22.559995 20793 consensus_meta.cc:287] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.504s	user 0.014s	sys 0.000s
I20260709 13:07:22.561533 21170 raft_consensus.cc:2804] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:22.562569 21170 raft_consensus.cc:697] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:22.562775 21337 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:22.563575 21170 consensus_queue.cc:237] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:22.564324 21337 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:22.564719 21337 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:22.566836 21300 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:22.567968 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 97392.8us
I20260709 13:07:22.568919 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:22.569378 21300 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:22.575588 21316 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:22.576361 21316 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:22.537684 21238 raft_consensus.cc:2804] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:22.579970 21238 raft_consensus.cc:697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 22 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:22.585479 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:07:22.592053 12725 tablet_server.cc:179] TabletServer@127.12.109.65:0 shutting down...
I20260709 13:07:22.542160 21334 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:22.595373 21334 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:22.559824 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:22.605553 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:22.615973 21341 raft_consensus.cc:3055] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:22.616456 21341 raft_consensus.cc:740] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:22.606413 21018 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:22.626714 21360 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:07:22.627884 13766 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57626 (request call id 249) took 53824 ms (client timeout 15000 ms). Trace:
I20260709 13:07:22.628830 21018 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:22.581287 21238 consensus_queue.cc:237] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:22.617385 21341 consensus_queue.cc:260] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:22.634773 21341 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:22.641552 21316 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:22.646479 13765 catalog_manager.cc:5697] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 reported cstate change: term changed from 14 to 15, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 15 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:22.628439 13766 rpcz_store.cc:269] 0709 13:06:28.782989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.783382 (+   393us) service_pool.cc:224] Handling call
0709 13:07:22.593998 (+53810616us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:15.007561 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:15.020139 (+ 12578us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:15.020219 (+    80us) write_op.cc:435] Acquired schema lock
0709 13:07:15.020251 (+    32us) tablet.cc:662] Decoding operations
0709 13:07:15.022083 (+  1832us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:15.022241 (+   158us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:15.022287 (+    46us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:07:15.024128 (+  1841us) tablet.cc:701] Row locks acquired
0709 13:07:15.024201 (+    73us) write_op.cc:260] PREPARE: finished
0709 13:07:15.024490 (+   289us) write_op.cc:270] Start()
0709 13:07:15.024724 (+   234us) write_op.cc:276] Timestamp: P: 1783602435024396 usec, L: 0
0709 13:07:15.024778 (+    54us) op_driver.cc:348] REPLICATION: starting
0709 13:07:15.025906 (+  1128us) log.cc:844] Serialized 5189 byte log entry
0709 13:07:15.808031 (+782125us) op_driver.cc:464] REPLICATION: finished
0709 13:07:16.085394 (+277363us) write_op.cc:301] APPLY: starting
0709 13:07:16.095327 (+  9933us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:16.227228 (+131901us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:16.227371 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:16.638978 (+411607us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:16.639444 (+   466us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 13:07:16.639511 (+    67us) write_op.cc:312] APPLY: finished
0709 13:07:16.639973 (+   462us) log.cc:844] Serialized 121 byte log entry
0709 13:07:16.640205 (+   232us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:16.640399 (+   194us) write_op.cc:454] Released schema lock
0709 13:07:16.640491 (+    92us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:16.640997 (+   506us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:16.641043 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1368230,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":245277,"apply.run_cpu_time_us":27518,"apply.run_wall_time_us":566493,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58389,"num_ops":13,"prepare.queue_time_us":216533,"prepare.run_cpu_time_us":44194,"prepare.run_wall_time_us":528063,"raft.queue_time_us":187522,"raft.run_cpu_time_us":11319,"raft.run_wall_time_us":57566,"replication_time_us":782980,"thread_start_us":642160,"threads_started":4,"wal-append.queue_time_us":475914,"wal-append.run_cpu_time_us":43919,"wal-append.run_wall_time_us":2113023}]]}
W20260709 13:07:22.653968 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 206936us
W20260709 13:07:22.641876 13770 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54072 (request call id 139) took 84708 ms (client timeout 15000 ms). Trace:
I20260709 13:07:22.565649 21337 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:22.658852 21213 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.638082 21334 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:22.657712 13770 rpcz_store.cc:269] 0709 13:05:57.933810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.938256 (+  4446us) service_pool.cc:224] Handling call
0709 13:07:22.641618 (+84703362us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5223416,"reported_tablets":24}
I20260709 13:07:22.688097 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:22.696027 14553 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:07:22.697455 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:07:22.701326 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.711234 21213 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:22.715222 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:22.724680 13944 raft_consensus.cc:2410] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 25 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 13:07:22.725090 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 351068us
W20260709 13:07:22.725274 21360 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:07:22.752912 13947 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:22.758646 13948 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:22.775095 13942 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:07:22.748914 13765 catalog_manager.cc:5697] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:22.790531 13944 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:22.748076 21099 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 13:07:22.797240 21213 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:22.799146 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 135297us
I20260709 13:07:22.801558 13987 leader_election.cc:400] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 21. Current term is 22.
I20260709 13:07:22.801353 21018 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:07:22.802989 21355 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:07:22.750808 21337 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:22.822863 14524 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.611s (SENT)
I20260709 13:07:22.827015 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:22.827863 14552 raft_consensus.cc:2393] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 18: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:22.830363 13947 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:22.827986 13765 catalog_manager.cc:5697] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 13 to 15, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 15 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:22.776546 21300 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:22.824855 20793 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:22.855760 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 249547us
I20260709 13:07:22.853811 13765 catalog_manager.cc:5697] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:22.865226 20793 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:22.854236 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 479128us
W20260709 13:07:22.879110 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71411.8us
I20260709 13:07:22.882771 13948 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:07:22.837145 21099 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:07:22.878011 13765 catalog_manager.cc:5697] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 14 to 17, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 17 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:22.836742 21322 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:22.885648 14524 consensus_peers.cc:597] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:22.889439 14558 raft_consensus.cc:1275] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:07:22.890205 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:22.888072 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:22.893532 14552 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:22.892993 13765 catalog_manager.cc:5697] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 12 to 15, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 15 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:22.863776 21165 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:07:22.898557 21165 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:22.899138 21165 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:22.901783 21165 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:22.873497 21252 raft_consensus.cc:3055] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:22.888664 21365 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
W20260709 13:07:22.895569 14524 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:22.908622 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:22.869035 21345 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:07:22.913004 13942 raft_consensus.cc:2393] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 27: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:07:22.916778 14524 leader_election.cc:336] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.753s (SENT)
I20260709 13:07:22.917446 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:22.914338 13765 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:07:22.935432 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.902187 21362 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 13:07:22.904450 21252 raft_consensus.cc:740] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:22.976888 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:22.996423 14552 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 27.
I20260709 13:07:22.904093 21342 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:22.902774 21300 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:22.895143 21018 consensus_queue.cc:1059] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:07:22.911774 21363 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.084s
I20260709 13:07:22.980121 21315 consensus_queue.cc:1059] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:22.910600 21374 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.095s
W20260709 13:07:22.978322 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 110959us
I20260709 13:07:22.977322 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:22.905889 21238 consensus_queue.cc:1059] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
W20260709 13:07:23.001186 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 123777us
W20260709 13:07:22.978972 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52261.4us
W20260709 13:07:23.019255 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 102524us
I20260709 13:07:23.025812 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:07:23.026564 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.981164 13765 catalog_manager.cc:5697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 reported cstate change: term changed from 16 to 20, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 20 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:23.031301 20872 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:23.031126 21373 consensus_queue.cc:1048] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 13:07:23.038398 20793 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:23.044889 13765 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 15 to 23, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 23 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:23.047660 13765 catalog_manager.cc:5697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:23.051505 21322 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:23.050524 13765 catalog_manager.cc:5697] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:23.053031 13765 catalog_manager.cc:5697] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:23.021644 21374 consensus_queue.cc:1243] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [LEADER]: Peer e1649107c781428692458f10597c5742 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:07:23.058888 21370 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:07:23.059908 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:23.057184 13765 catalog_manager.cc:5697] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 10 to 12, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 12 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:23.062237 21369 log.cc:927] Time spent T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.164s	user 0.001s	sys 0.001s
I20260709 13:07:23.062091 13765 catalog_manager.cc:5697] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 11 to 15, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 15 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:23.063807 13765 catalog_manager.cc:5697] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:23.065508 13765 catalog_manager.cc:5697] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 reported cstate change: term changed from 14 to 17, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 17 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:23.068041 13765 catalog_manager.cc:5697] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 12 to 14, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 14 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:23.069913 13765 catalog_manager.cc:5697] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 11 to 13, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 13 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:23.072353 13765 catalog_manager.cc:5697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 15 to 17, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 17 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:23.074406 13765 catalog_manager.cc:5697] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 13 to 15, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 15 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:23.076921 13765 catalog_manager.cc:5697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:23.092906 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:23.083346 21246 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:23.089140 21315 consensus_queue.cc:1243] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer 4dd265d61fc141a48ca9d3d5c8f52921 log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:07:23.100350 21068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.002s	sys 0.026s
W20260709 13:07:23.112025 13759 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 289877us
I20260709 13:07:23.102545 21246 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:23.103777 21331 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:23.112035 21068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.002s	sys 0.026s
I20260709 13:07:23.110451 21315 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:23.147277 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 43520.6us
I20260709 13:07:23.120170 21018 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:23.146797 21246 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:07:23.172962 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.178900 21345 raft_consensus.cc:3055] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:23.179378 21345 raft_consensus.cc:740] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:23.180423 21345 consensus_queue.cc:260] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.185624 21373 raft_consensus.cc:3055] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:23.186105 21373 raft_consensus.cc:740] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:23.187024 21373 consensus_queue.cc:260] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.188432 21373 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:23.199311 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.202836 13942 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:07:23.204097 14553 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:07:23.205410 13949 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:23.199187 21345 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:23.162896 21344 consensus_meta.cc:287] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.661s	user 0.009s	sys 0.005s
I20260709 13:07:23.178761 21372 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:23.224880 21252 consensus_queue.cc:260] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:23.221767 21346 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:23.226804 21346 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:23.228991 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.229206 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:23.230680 13947 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:23.229923 13987 leader_election.cc:400] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21.
I20260709 13:07:23.231904 13987 leader_election.cc:403] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:07:23.235949 21099 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.236527 21099 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:23.237051 21300 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:23.238391 21300 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.205322 21018 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:23.223183 21372 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:23.165153 21315 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:23.245369 21281 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:23.277722 13949 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:07:23.284999 13950 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:23.252022 21342 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:23.264874 21344 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:23.274629 21341 consensus_meta.cc:287] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.639s	user 0.010s	sys 0.005s
I20260709 13:07:23.269393 21252 raft_consensus.cc:3060] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:23.301947 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 127461us
I20260709 13:07:23.325409 13947 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:23.328789 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:23.288794 21281 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:23.336289 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 415431us
I20260709 13:07:23.299427 21165 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:07:23.322317 21000 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:23.350262 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:23.350940 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:23.356281 13949 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:23.348914 21342 consensus_queue.cc:1059] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
W20260709 13:07:23.346766 21337 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.001s
W20260709 13:07:23.360707 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.341861 21099 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:23.361830 21000 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:23.366278 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.366830 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 208082us
I20260709 13:07:23.351723 13838 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:23.372787 13838 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:23.358837 21342 raft_consensus.cc:3055] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:23.324180 21018 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.369596 21099 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:23.386643 13759 catalog_manager.cc:4755] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6b098cc2d51e471cb2c9c1b17eebcb57: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:07:23.427168 21099 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.344564 21372 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.435019 21372 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:23.435163 21315 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:23.351580 21363 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:07:23.360810 21297 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:07:23.429796 21099 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:23.447021 13915 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.593s (SENT)
I20260709 13:07:23.378330 21346 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:23.454850 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:23.456890 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:23.423830 21321 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:07:23.463141 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 152657us
I20260709 13:07:23.464949 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:23.382562 21342 raft_consensus.cc:740] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:23.448496 21238 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.447862 21337 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:23.454363 21385 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:23.496510 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 49505.7us
I20260709 13:07:23.465988 21366 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:23.470089 21321 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:07:23.501068 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.454210 21189 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.174s
W20260709 13:07:23.503193 13915 leader_election.cc:336] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.762s (SENT)
I20260709 13:07:23.514599 13915 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:23.512372 21281 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:23.490937 21300 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.504115 21321 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:23.536691 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:23.492246 21238 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.525547 21374 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:23.495239 21337 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:23.568703 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 125953us
I20260709 13:07:23.527714 21316 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:23.533449 21149 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:23.570937 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:23.573487 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 76095.8us
I20260709 13:07:23.517966 21366 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:23.578418 13915 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.766s (SENT)
I20260709 13:07:23.570724 21323 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:23.567006 21297 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:23.593101 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.579816 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 57438.3us
I20260709 13:07:23.547320 21385 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.583160 21000 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:23.568058 21213 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.565122 21342 consensus_queue.cc:260] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.611512 21342 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:23.599555 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:23.609858 21337 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:23.610255 21213 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:23.550908 21300 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.625334 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:23.632483 21366 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:23.601069 21323 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 20 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:23.640131 13950 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:23.641336 21384 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.650110 21068 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.658134 21068 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:23.659649 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 84766.7us
I20260709 13:07:23.641767 21315 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:23.661697 13915 leader_election.cc:336] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.798s (SENT)
I20260709 13:07:23.665375 13915 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:23.663683 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:23.663516 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:23.685072 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.670756 14020 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 26.
I20260709 13:07:23.661218 21344 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21.
W20260709 13:07:23.685866 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.704970 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:23.710222 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:23.710740 21390 log.cc:927] Time spent T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:07:23.711241 14017 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:23.719417 21389 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:07:23.723892 21300 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:23.664444 21384 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.730018 13838 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:23.732106 21366 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:23.732523 21366 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.732869 21366 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:23.737278 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.739113 21213 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:23.740444 21099 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.741025 21099 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.669571 21315 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:23.758939 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.707175 21341 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:23.783280 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:23.785702 21388 log.cc:927] Time spent T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.164s	user 0.000s	sys 0.004s
I20260709 13:07:23.786420 14017 raft_consensus.cc:1275] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:07:23.762845 21322 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:23.792917 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:23.793903 14020 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:07:23.797251 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 136889us
I20260709 13:07:23.773062 21238 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:23.773775 21165 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:23.812529 21384 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:23.813225 21366 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.815302 21366 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:23.815587 21246 consensus_meta.cc:287] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.000s
W20260709 13:07:23.809145 13915 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.807s (SENT)
W20260709 13:07:23.817065 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.817334 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:23.816265 13989 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:23.818595 13989 leader_election.cc:336] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:23.819384 13989 leader_election.cc:336] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:23.819800 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:07:23.821116 13989 leader_election.cc:336] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:23.821507 13989 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:07:23.846021 13989 leader_election.cc:336] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:23.846879 13989 consensus_peers.cc:597] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:23.847481 13989 leader_election.cc:336] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:23.849363 13989 consensus_peers.cc:597] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:23.848599 21252 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:23.851611 13840 leader_election.cc:336] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.529s (SENT)
I20260709 13:07:23.852751 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:23.816027 21246 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:23.753633 21323 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:23.858817 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:23.849972 13989 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:23.863204 21362 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:23.866564 21068 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:23.867121 21265 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:23.867779 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 69262.5us
W20260709 13:07:23.878226 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:23.845474 21238 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:07:23.887465 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 269379us
W20260709 13:07:23.897569 14524 consensus_peers.cc:597] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:23.806618 21322 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:23.849222 21384 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:23.882352 13989 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:23.902277 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:23.905678 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:23.818588 21165 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:23.877293 21345 consensus_meta.cc:287] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.660s	user 0.007s	sys 0.007s
W20260709 13:07:23.905735 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:23.909130 14524 leader_election.cc:336] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:23.909721 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:23.906001 13989 consensus_peers.cc:597] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:23.911727 13989 leader_election.cc:336] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:23.894651 21213 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:23.916321 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.923491 14016 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
W20260709 13:07:23.935940 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 71591.6us
I20260709 13:07:23.901788 21265 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:24.000671 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.000672 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 65130.2us
I20260709 13:07:23.927973 20969 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:24.000900 21342 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.925426 21213 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:24.048089 21373 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: Time spent flushing consensus metadata: real 0.859s	user 0.009s	sys 0.009s
W20260709 13:07:24.075503 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.070994 21406 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:07:24.052801 21315 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:24.071796 21409 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:24.078118 21385 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:24.079545 21342 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:24.105381 21363 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:24.104667 21018 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:24.125937 14524 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.248s (SENT)
I20260709 13:07:24.141791 14023 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 12. Current term is 13. Ops: []
W20260709 13:07:24.142437 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 90419.6us
I20260709 13:07:24.140591 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:24.146756 21165 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:24.118723 21409 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:24.138482 21363 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:24.138389 21322 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:24.156719 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:24.157752 13950 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
W20260709 13:07:24.163265 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.173038 14020 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 27.
I20260709 13:07:24.174710 14017 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 27 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:07:24.175279 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:24.175987 13947 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:24.178694 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:24.179309 13941 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:24.182591 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:24.184991 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:24.155459 21409 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:24.188314 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.197335 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:24.197608 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:24.199414 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:24.141911 21018 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:24.204975 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:24.213498 13948 raft_consensus.cc:2468] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:24.215848 13943 raft_consensus.cc:2393] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 27: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:24.187835 21000 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:24.220935 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.195328 21362 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:24.228132 13838 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:24.205644 21413 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:24.249817 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.250690 14022 raft_consensus.cc:2376] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 27 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 27. Re-sending same reply.
I20260709 13:07:24.250946 21099 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:24.262276 13840 leader_election.cc:336] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.657s (SENT)
I20260709 13:07:24.264740 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:24.258486 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 69421.8us
W20260709 13:07:24.267560 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.267881 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:24.265666 14016 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:07:24.253851 21265 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:24.255519 21322 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:07:24.273126 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 74411.7us
I20260709 13:07:24.227258 21000 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:24.260169 21213 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:24.283414 13941 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
W20260709 13:07:24.295243 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.295379 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.235016 21362 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:24.315279 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 53037.1us
I20260709 13:07:24.308005 21018 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:24.310344 21363 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:24.310609 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.316684 14019 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:24.312100 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:24.271016 21099 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:24.317173 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:24.275954 21322 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.286019 21342 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:24.329362 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 101350us
I20260709 13:07:24.332278 13838 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:24.337400 14524 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.501s (SENT)
I20260709 13:07:24.338109 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:24.350941 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:24.351413 13759 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 123516us
I20260709 13:07:24.352236 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.353097 14016 raft_consensus.cc:2410] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:07:24.331749 21322 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:24.399873 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:24.427871 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:24.451160 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 113807us
W20260709 13:07:24.453158 14524 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.033s (SENT)
I20260709 13:07:24.454440 21420 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:24.428959 14016 raft_consensus.cc:2376] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 27. Re-sending same reply.
I20260709 13:07:24.459088 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:24.462030 13840 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.025s (SENT)
I20260709 13:07:24.385324 21265 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:24.453716 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:24.426155 21389 log.cc:927] Time spent T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 13:07:24.477394 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 77672us
I20260709 13:07:24.480823 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:24.484797 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.485708 14016 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:07:24.472369 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.493822 21252 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.494658 21252 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:24.472860 21265 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.475471 21421 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.475682 21416 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:24.505234 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.506186 13838 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:24.509128 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:24.497779 21252 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:24.510490 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:24.510878 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 57791.4us
I20260709 13:07:24.511240 14021 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:07:24.485077 21068 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:24.517671 14524 leader_election.cc:336] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:24.516633 14019 raft_consensus.cc:2410] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:07:24.510283 21425 raft_consensus.cc:2804] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:24.514384 13838 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:24.518620 14524 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:24.505836 21265 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:24.529901 14524 leader_election.cc:336] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:24.517776 21365 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:24.532673 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:24.536005 14016 raft_consensus.cc:1275] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:07:24.547061 14019 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 LEADER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 20. Current term is 22. Ops: []
I20260709 13:07:24.522826 21323 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:24.550832 21323 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:24.518463 21421 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:24.551980 21323 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:24.557698 14524 leader_election.cc:336] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.220s (SENT)
I20260709 13:07:24.558328 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:24.559573 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.564429 13915 leader_election.cc:336] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.113s (SENT)
W20260709 13:07:24.565622 13915 leader_election.cc:336] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.127s (SENT)
I20260709 13:07:24.533156 21415 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:24.537919 13759 catalog_manager.cc:4755] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6b098cc2d51e471cb2c9c1b17eebcb57: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:07:24.567487 21421 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:24.568539 13915 leader_election.cc:336] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.144s (SENT)
I20260709 13:07:24.569110 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:24.568660 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.570010 21374 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:24.571408 13915 leader_election.cc:336] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.144s (SENT)
I20260709 13:07:24.571914 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:24.572913 21374 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:24.575150 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:24.575747 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.576052 14019 raft_consensus.cc:2410] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:07:24.576371 14018 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:24.577661 14017 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:24.578323 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.579264 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.580049 21374 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.580572 21374 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:24.582859 21374 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:24.583602 14019 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:24.562598 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:24.584502 21000 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:24.584815 21068 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:24.537370 21344 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.587105 21344 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:24.588399 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:24.589083 13942 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 21.
I20260709 13:07:24.590845 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.519616 21425 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:24.592080 21425 raft_consensus.cc:3060] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:24.580293 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.593828 14018 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:24.601580 21392 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:24.605018 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 47353.2us
W20260709 13:07:24.607998 13914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.608486 14017 raft_consensus.cc:1275] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:24.610116 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:24.617787 21246 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 27 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:24.625919 13838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.627506 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.629020 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.630121 14017 raft_consensus.cc:2410] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:07:24.630874 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:24.631678 13948 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:24.634406 14023 raft_consensus.cc:2468] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:24.638232 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:24.639487 13942 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:24.643262 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.620303 21331 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:24.622663 21321 consensus_meta.cc:287] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 1.057s	user 0.016s	sys 0.000s
W20260709 13:07:24.659684 13915 leader_election.cc:336] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.504s (SENT)
I20260709 13:07:24.660596 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.615670 21344 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:24.662732 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:24.667115 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:07:24.673175 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.671378 13840 leader_election.cc:336] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.606s (SENT)
W20260709 13:07:24.683933 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.683698 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:24.672798 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.667325 14023 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:24.682369 14559 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:24.690359 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:24.695508 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.698467 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 74249us
W20260709 13:07:24.699580 13915 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.851s (SENT)
I20260709 13:07:24.702211 21337 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.702723 21337 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:24.705098 21337 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:24.718489 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:24.718897 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:24.666941 21432 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:24.723191 13915 leader_election.cc:336] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:24.689670 21342 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.734196 21300 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.734781 21300 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:24.735766 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:24.736387 14023 raft_consensus.cc:3060] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:24.715070 21344 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.742483 13839 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:24.666121 21331 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:24.678606 21362 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:24.666251 21321 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:24.742699 13989 leader_election.cc:336] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:24.750028 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 78676.4us
I20260709 13:07:24.737593 21357 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.767454 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:24.769132 14017 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:24.744031 21372 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.741899 21284 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:07:24.729286 13915 consensus_peers.cc:597] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:24.752739 21068 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:24.747596 21252 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:24.778973 14023 raft_consensus.cc:2468] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 22.
I20260709 13:07:24.763015 21414 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:07:24.779837 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51896.1us
W20260709 13:07:24.786104 13915 consensus_peers.cc:597] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:24.791091 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.792809 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:24.793524 14016 raft_consensus.cc:2376] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 21. Re-sending same reply.
W20260709 13:07:24.798295 13915 leader_election.cc:336] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:24.740574 21432 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:24.791689 21346 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:07:24.781540 21284 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:24.822341 21284 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:24.824424 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.792261 21252 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:24.828969 21404 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.787s	user 0.014s	sys 0.001s
I20260709 13:07:24.830432 13945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58096 (request call id 1331) took 1146 ms. Trace:
I20260709 13:07:24.830803 13945 rpcz_store.cc:276] 0709 13:07:23.683944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.684082 (+   138us) service_pool.cc:224] Handling call
0709 13:07:23.691524 (+  7442us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:23.691950 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:23.691981 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:23.693170 (+  1189us) log.cc:844] Serialized 52 byte log entry
0709 13:07:23.914938 (+221768us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:23.914992 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.915073 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:24.828413 (+913340us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:24.830120 (+  1707us) raft_consensus.cc:1685] finished
0709 13:07:24.830146 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:24.830368 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5848,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5627,"spinlock_wait_cycles":10845056,"thread_start_us":204999,"threads_started":1,"wal-append.queue_time_us":219827}
I20260709 13:07:24.833761 21409 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.834303 21409 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:24.836516 21321 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:24.808512 21372 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:24.765516 21344 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:24.839129 13916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.824395 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:24.809387 13840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 4.898s (SENT) [suppressed 123 similar messages]
I20260709 13:07:24.768901 21342 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:24.774842 21357 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:24.787709 21414 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:24.848742 13840 leader_election.cc:336] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.034s (SENT)
I20260709 13:07:24.849189 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:24.844214 13915 leader_election.cc:336] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:24.846287 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.853837 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:24.854658 14016 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:24.860281 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.861078 14022 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:24.861876 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.862514 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.863101 14024 raft_consensus.cc:2376] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 16. Re-sending same reply.
I20260709 13:07:24.863966 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.865048 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.863024 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.866683 14022 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:24.867015 21432 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:24.854660 13942 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:24.867534 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.868129 21344 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:24.870780 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:24.872426 21265 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:24.874130 14020 raft_consensus.cc:2368] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18.
I20260709 13:07:24.875295 14020 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 19. Current term is 21. Ops: []
I20260709 13:07:24.875568 21342 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:24.877465 14020 raft_consensus.cc:1240] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:24.852581 21414 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:24.880033 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.881426 14019 raft_consensus.cc:1240] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:07:24.882097 14024 raft_consensus.cc:2410] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 13:07:24.883069 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.883349 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.884198 14023 raft_consensus.cc:2410] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:07:24.884964 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 132545us
W20260709 13:07:24.886590 13989 leader_election.cc:336] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:24.890542 13942 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:24.891923 14553 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 27.
I20260709 13:07:24.891923 14016 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:24.893396 14022 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:07:24.894637 13838 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:24.894903 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:24.895534 21421 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:24.895790 21252 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:24.896039 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:24.896220 21322 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:24.896720 13947 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 27.
I20260709 13:07:24.897279 21421 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:24.899183 14522 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:24.888144 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:24.901765 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:24.902416 14021 raft_consensus.cc:2376] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 18. Re-sending same reply.
W20260709 13:07:24.915738 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 105610us
W20260709 13:07:24.921283 13915 leader_election.cc:336] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:24.911583 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:07:24.922605 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 73895.4us
W20260709 13:07:24.923442 13915 consensus_peers.cc:597] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:24.925995 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.897815 21421 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:24.944641 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:24.943249 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.945616 13947 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:24.955423 13915 consensus_peers.cc:597] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:24.957990 21365 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:07:24.906472 21291 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:07:24.961828 21372 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:24.922292 20793 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:24.964711 21322 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:24.971588 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.007098 13941 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:24.972323 21265 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:24.973439 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:24.975106 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.010488 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:24.975363 13840 leader_election.cc:336] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.098s (SENT)
I20260709 13:07:25.011122 14020 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:25.011240 13840 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:24.976601 14022 raft_consensus.cc:1275] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:24.982630 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:24.990837 14524 leader_election.cc:336] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:24.991580 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.013119 14016 raft_consensus.cc:2376] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 16. Re-sending same reply.
I20260709 13:07:25.013509 14015 raft_consensus.cc:2410] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:07:24.970034 14017 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
W20260709 13:07:24.993415 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 70817.3us
I20260709 13:07:25.001745 21357 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:25.002950 21365 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 27 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:25.018750 21300 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:25.018399 21365 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:24.998775 21291 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.019583 21291 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:25.021545 21297 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:25.025262 21300 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.038141 21300 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.043265 21300 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:25.045110 14022 raft_consensus.cc:1275] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:25.049220 21421 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.050431 21409 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:25.041353 21284 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:25.009563 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:25.006006 13915 consensus_peers.cc:597] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.059710 13942 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
W20260709 13:07:25.059899 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 53961.3us
W20260709 13:07:25.060989 13915 consensus_peers.cc:597] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.058775 13947 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:25.061666 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.062376 14022 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:25.064469 13941 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
W20260709 13:07:25.067135 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 91811us
I20260709 13:07:25.069293 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:25.063262 21331 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:25.090031 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.091003 13915 consensus_peers.cc:597] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.094882 13838 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:25.098417 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.099793 13840 leader_election.cc:336] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.745s (SENT)
I20260709 13:07:25.100297 13840 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:25.102948 13840 leader_election.cc:336] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.690s (SENT)
I20260709 13:07:25.103358 13840 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:25.103664 21323 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.104166 21323 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.102308 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.109454 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.108028 13838 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:25.112073 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.112965 14015 raft_consensus.cc:2410] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:07:25.114035 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.114292 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.114428 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.115087 14019 raft_consensus.cc:2410] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:07:25.115281 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.115772 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.115877 14023 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:25.117067 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.117185 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.117784 14020 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
I20260709 13:07:25.117843 14015 raft_consensus.cc:2376] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 21. Re-sending same reply.
I20260709 13:07:25.122087 14022 raft_consensus.cc:1275] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:07:25.123652 13915 consensus_peers.cc:597] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.125514 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.126029 14023 raft_consensus.cc:2376] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 22. Re-sending same reply.
W20260709 13:07:25.124498 13915 leader_election.cc:336] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.109388 21165 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:25.107741 21265 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:25.118546 21297 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:25.133206 21297 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.133608 21297 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:25.138499 21323 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:25.139698 13989 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:25.141191 13989 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:25.141801 13989 consensus_peers.cc:597] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.145584 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.145869 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.159121 14019 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:25.146289 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.160609 14020 raft_consensus.cc:2376] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3779649036364744823fc035dac892c7 in term 21. Re-sending same reply.
I20260709 13:07:25.146426 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.147051 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.147157 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.162930 14023 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:07:25.147851 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.147943 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.164551 14015 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:25.147893 21284 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:25.167289 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.168052 13941 raft_consensus.cc:2368] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 19. Current term is 20.
I20260709 13:07:25.169023 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.073917 21414 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:25.162945 21363 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:25.171276 21414 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:25.171691 21363 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 20 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:25.172717 21363 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.173434 21291 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:25.150184 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.159970 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 52020.8us
I20260709 13:07:25.174945 13838 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:25.175664 21291 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:25.175787 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.176589 14552 raft_consensus.cc:2393] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 28: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:25.178678 21265 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:07:25.169051 21425 consensus_meta.cc:287] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.576s	user 0.006s	sys 0.004s
I20260709 13:07:25.179368 21265 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 20 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:25.179452 21425 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.180496 21265 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.182274 21425 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:25.184993 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.185719 13945 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:25.187290 13943 raft_consensus.cc:1240] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:25.188387 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.190289 21323 consensus_queue.cc:1059] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:25.190972 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.192229 21323 raft_consensus.cc:3055] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:25.192653 21323 raft_consensus.cc:740] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:25.192230 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.194558 13943 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:07:25.194262 21323 consensus_queue.cc:260] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:25.195920 21323 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:25.199641 13944 raft_consensus.cc:1217] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 14.8->[16.9-16.9]   Dedup: 16.9->[]
I20260709 13:07:25.200169 13944 raft_consensus.cc:1240] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:07:25.201889 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.196623 13948 raft_consensus.cc:1240] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:07:25.197007 13941 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:07:25.204259 13838 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:25.197598 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.205229 13945 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:25.206441 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.207024 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.189725 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.210446 13942 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:25.212141 21300 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.212661 21300 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.213636 13950 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 21.
I20260709 13:07:25.215049 21300 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:25.215255 21346 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:25.215984 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:07:25.218186 13989 leader_election.cc:336] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:25.158964 13915 leader_election.cc:336] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.235546 13942 raft_consensus.cc:2468] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 28.
W20260709 13:07:25.235901 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 76980.1us
W20260709 13:07:25.238039 13915 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.245910 13915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.218781 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.223572 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.223687 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.228706 13945 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:25.207057 13947 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:25.235368 13989 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:25.253422 13943 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:25.257776 13840 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.892s (SENT)
W20260709 13:07:25.258790 13840 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.894s (SENT)
I20260709 13:07:25.257120 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:25.260264 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:07:25.261409 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 100572us
I20260709 13:07:25.261919 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
W20260709 13:07:25.262077 13989 leader_election.cc:336] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.260907 14552 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:25.260766 14522 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:07:25.261281 14524 leader_election.cc:336] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.275763 13946 raft_consensus.cc:1217] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Deduplicated request from leader. Original: 14.12->[15.13-15.13]   Dedup: 15.13->[]
I20260709 13:07:25.276301 13946 raft_consensus.cc:1240] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
W20260709 13:07:25.279605 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.283558 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.235124 21346 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:25.283591 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.286769 13989 leader_election.cc:336] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.289155 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.285490 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.292407 14017 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:07:25.293421 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:25.288484 13989 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.294710 14019 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:25.296610 14020 pending_rounds.cc:85] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 8
I20260709 13:07:25.297851 14020 pending_rounds.cc:107] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:07:25.298135 14020 raft_consensus.cc:2889] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:25.298455 14020 pending_rounds.cc:107] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:07:25.298693 14020 raft_consensus.cc:2889] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:25.299551 13989 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.299934 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.301116 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.311151 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.311896 14021 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:25.313867 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.273963 21357 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:25.326539 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.327492 13944 raft_consensus.cc:2368] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21.
I20260709 13:07:25.328333 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.329461 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.330873 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.331120 13945 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 18. Current term is 20. Ops: [18.11-18.11]
I20260709 13:07:25.331508 13946 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:25.332170 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.332357 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.333321 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.333461 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.334434 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:25.313510 13989 leader_election.cc:336] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.335189 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.335464 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.335871 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.257293 21409 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:07:25.336545 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:25.346517 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 49656.5us
I20260709 13:07:25.339011 13947 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 22.
I20260709 13:07:25.346644 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.336545 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.353185 13942 raft_consensus.cc:2410] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:07:25.356460 14024 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 21: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:25.357292 14019 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 21: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:25.286475 21166 raft_consensus.cc:2804] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:07:25.289098 21346 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:25.372687 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 112886us
W20260709 13:07:25.358925 13989 leader_election.cc:336] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.362422 13947 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:07:25.366901 13942 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:07:25.372761 14017 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
W20260709 13:07:25.388021 13989 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.392835 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:25.392941 13989 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.400837 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.401585 14022 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:25.289386 21297 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.412694 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:25.396456 21442 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:07:25.428758 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.429314 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.430308 14022 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:25.400533 21166 raft_consensus.cc:697] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 28 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:25.440618 14017 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:25.441515 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.442476 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.447964 14024 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 13:07:25.449112 21445 log.cc:927] Time spent T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:07:25.451673 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.452512 14023 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:25.452834 21432 consensus_queue.cc:1048] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
W20260709 13:07:25.453334 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.454242 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.454815 14019 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:25.457141 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.458300 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.458891 14015 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
W20260709 13:07:25.463091 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 50001.4us
I20260709 13:07:25.464269 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.468835 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.471258 13942 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 20: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:25.432572 21345 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:25.485015 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.485967 14024 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:25.483719 13944 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 28 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 27. Current term is 28. Ops: []
I20260709 13:07:25.509487 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.576794 13943 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
I20260709 13:07:25.541822 21166 consensus_queue.cc:237] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:25.500970 14022 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:07:25.573570 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 102933us
I20260709 13:07:25.448658 14552 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
I20260709 13:07:25.576790 14024 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
W20260709 13:07:25.572527 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 100238us
I20260709 13:07:25.582813 21451 consensus_queue.cc:1048] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:07:25.584365 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.586318 21297 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:25.587605 21281 raft_consensus.cc:2764] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader election decision vote started in defunct term 21: won
I20260709 13:07:25.589303 21344 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:25.589687 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.592702 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.594162 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.594771 14016 raft_consensus.cc:2468] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:25.599334 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.601193 14018 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:07:25.595649 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.602491 14016 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:25.603864 14021 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 15.
I20260709 13:07:25.605139 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.605851 14021 raft_consensus.cc:2376] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 22. Re-sending same reply.
I20260709 13:07:25.606948 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.607398 14019 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:25.611151 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:25.611716 13942 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:25.607637 14021 raft_consensus.cc:2376] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 22. Re-sending same reply.
I20260709 13:07:25.614984 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:25.615841 13989 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.616007 14024 raft_consensus.cc:2393] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 21: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:25.620381 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.620344 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.621588 14021 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 27.
I20260709 13:07:25.622601 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.609494 14017 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:25.632997 14021 raft_consensus.cc:2376] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 17. Re-sending same reply.
I20260709 13:07:25.625640 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.635484 14015 raft_consensus.cc:2410] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:07:25.633742 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.636493 14019 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 20: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:25.623303 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.637868 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.639384 14021 raft_consensus.cc:2410] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:07:25.642431 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.643018 14017 raft_consensus.cc:2376] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 17. Re-sending same reply.
I20260709 13:07:25.643872 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.644728 14017 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:07:25.645738 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.646559 14019 raft_consensus.cc:2376] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 18. Re-sending same reply.
I20260709 13:07:25.653939 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.660567 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.661629 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.662573 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.663448 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.664053 14019 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 22: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:25.589852 21344 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:25.668555 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.686676 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.687258 14024 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:25.687518 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:25.671320 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.677682 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.678869 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.690716 14022 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 13:07:25.679945 21449 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:07:25.694412 13945 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:25.681993 21297 consensus_queue.cc:1059] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:07:25.631953 21357 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:25.693334 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.580260 21345 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:25.686190 14524 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:25.697149 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.710608 14019 raft_consensus.cc:2393] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 22: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:25.654227 21300 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:25.700482 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.704835 14021 raft_consensus.cc:2410] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:07:25.685652 21392 consensus_queue.cc:1059] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:25.716646 14015 raft_consensus.cc:2393] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 21: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:07:25.718564 13840 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.994s (SENT)
I20260709 13:07:25.694831 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.715173 13947 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:25.713222 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.724690 13942 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 23.
I20260709 13:07:25.724547 13838 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:25.727072 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:25.727833 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 61304.2us
I20260709 13:07:25.731333 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.733156 14016 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 23.
I20260709 13:07:25.733914 14022 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:25.732920 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.728433 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.734908 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.735294 14019 raft_consensus.cc:2376] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 19. Re-sending same reply.
I20260709 13:07:25.737929 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.738806 14021 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 20: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:25.739216 14019 raft_consensus.cc:1240] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:25.742586 14020 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:25.746747 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.747320 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.747617 14023 raft_consensus.cc:2410] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:07:25.747359 14015 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:25.749184 14024 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:25.762077 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:25.730288 21284 raft_consensus.cc:2749] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:25.764403 14020 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:07:25.733031 21365 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:25.767750 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.767779 14020 pending_rounds.cc:85] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 11
I20260709 13:07:25.774302 14020 pending_rounds.cc:107] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:07:25.774708 14020 raft_consensus.cc:2889] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:25.776605 14021 raft_consensus.cc:2376] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 15. Re-sending same reply.
I20260709 13:07:25.779356 14022 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:07:25.782660 21344 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:25.783943 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.784722 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.710482 21453 consensus_queue.cc:1059] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
W20260709 13:07:25.771975 13840 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.032s (SENT)
I20260709 13:07:25.772825 14019 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:25.712051 21357 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:25.785858 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.786240 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.793200 14020 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:25.766649 14018 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:25.785068 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.794525 14015 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 20: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:25.795660 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.796590 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.798161 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.767529 20793 consensus_queue.cc:1059] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:07:25.731204 21300 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:25.811537 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.811709 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:25.816403 14023 raft_consensus.cc:2393] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 20: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:25.812490 14015 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:07:25.813120 14022 raft_consensus.cc:2410] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:07:25.814497 14024 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:25.744071 21166 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.047s
I20260709 13:07:25.827512 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.819520 21357 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:25.821058 20793 raft_consensus.cc:3055] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:07:25.821318 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.792240 21454 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:25.772123 21392 consensus_queue.cc:1048] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
W20260709 13:07:25.832877 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 70968.6us
I20260709 13:07:25.846046 13915 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:25.847512 21300 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:25.811419 21365 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 22 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:25.849109 21451 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:25.848982 21365 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:25.849911 21451 consensus_queue.cc:1048] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:25.852427 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:25.790640 21068 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:07:25.853519 21453 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:07:25.853538 21323 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.006s
I20260709 13:07:25.821547 14017 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
W20260709 13:07:25.840664 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.855922 21453 consensus_queue.cc:1048] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:25.856242 21346 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.857841 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:25.858673 21453 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:25.859175 21453 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:25.859622 21453 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:25.865024 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:25.871116 13988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.872180 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:25.873071 21452 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:25.843376 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.873667 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:25.832708 20793 raft_consensus.cc:740] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:25.875252 20793 consensus_queue.cc:260] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.876820 20793 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:25.879773 21452 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:25.881348 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:25.883107 21281 raft_consensus.cc:3055] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:25.883258 21453 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.872262 21337 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:25.883670 21281 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:25.885457 21345 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:25.885346 21281 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:25.886399 14019 raft_consensus.cc:3055] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:25.887121 14019 raft_consensus.cc:740] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:25.887768 21281 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:25.887925 14019 consensus_queue.cc:260] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.888834 14019 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:25.862629 21454 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:25.890297 21454 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:25.894415 21452 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:25.895198 21444 raft_consensus.cc:3055] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:25.895663 21444 raft_consensus.cc:740] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:25.896528 21444 consensus_queue.cc:260] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.897787 21444 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:25.900709 13840 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.926s (SENT)
W20260709 13:07:25.909698 13840 leader_election.cc:336] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.070s (SENT)
I20260709 13:07:25.901703 21453 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:25.910135 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:25.911453 21414 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:25.903574 21346 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:25.906626 21421 raft_consensus.cc:3055] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:07:25.906528 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:25.914160 21421 raft_consensus.cc:740] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:25.915131 21421 consensus_queue.cc:260] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:25.915668 13840 leader_election.cc:336] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 2.131s (SENT)
I20260709 13:07:25.916112 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:25.916210 21421 raft_consensus.cc:3060] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:25.906628 13915 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:25.917840 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:25.918393 21453 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:25.918668 21452 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:25.919946 21414 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:25.920032 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:25.920818 21414 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:25.921964 21452 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:25.923249 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:25.924136 21452 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:25.924523 21452 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:25.924930 21452 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:25.841463 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:25.931756 21346 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:25.933190 21452 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.936005 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:25.936134 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:25.938580 21068 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:07:25.939054 21068 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:25.939201 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:25.939469 21068 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:25.937197 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:25.937500 21323 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:25.941345 21453 raft_consensus.cc:2764] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:07:25.942251 21453 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:25.942780 21453 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 21 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:25.943725 21453 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.947193 21452 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:25.949612 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 208570us
I20260709 13:07:25.842646 14020 raft_consensus.cc:2410] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:07:25.957578 13987 leader_election.cc:400] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 19. Current term is 20.
I20260709 13:07:25.961484 13840 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:25.964334 14019 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:25.959303 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:25.956828 21409 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:25.979405 13987 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:25.983035 14018 raft_consensus.cc:2468] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:07:25.983445 13915 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:25.984728 13915 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:25.984934 14552 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 28 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 27. Current term is 28. Ops: []
I20260709 13:07:25.981739 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:25.978364 21454 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:25.989061 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:25.989874 21452 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:25.995230 21452 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.995630 21452 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:26.000967 13987 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.001909 21345 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:26.002692 21345 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:25.989878 21337 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:26.002715 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.987819 21323 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:26.009115 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:26.004525 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 210608us
I20260709 13:07:26.007354 13987 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.010449 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:26.012995 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:26.013311 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.014925 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:25.982813 21444 consensus_queue.cc:1048] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:07:26.034013 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.035416 21300 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:26.004779 21346 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:26.009389 21357 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 13:07:26.008019 21421 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:26.038501 21454 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:26.058373 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:26.059180 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:26.059742 14559 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:07:26.046617 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:26.063030 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:26.061996 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.009486 21451 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:26.045083 13838 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:26.060238 20793 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:26.065026 21357 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:26.067344 13987 leader_election.cc:400] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21.
W20260709 13:07:26.068742 14524 leader_election.cc:336] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.075189 13987 leader_election.cc:403] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:07:26.076727 13987 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.015578 21337 raft_consensus.cc:2764] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:07:26.079617 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:26.014712 21252 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:26.068147 21018 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:26.012225 21345 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:26.040437 21323 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:07:26.078202 21344 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 20. Current term is 21.
I20260709 13:07:26.087522 21451 raft_consensus.cc:697] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 21 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:26.093111 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 80193.4us
I20260709 13:07:26.106429 13942 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:26.107796 13947 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:07:26.092033 21414 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:26.099588 13943 raft_consensus.cc:1240] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 28 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
I20260709 13:07:26.103816 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:26.088536 21365 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:26.100283 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:26.112097 13945 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:26.088621 21372 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:26.087919 21425 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:26.113252 21068 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:26.118284 21068 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 28 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:26.113430 21421 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:07:26.113440 21468 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:26.127504 21468 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:26.127094 21451 consensus_queue.cc:237] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:26.127949 21468 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:26.130173 21165 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:26.131228 13945 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:26.113938 21363 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:07:26.113758 21346 consensus_queue.cc:1059] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:07:26.142593 21345 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:07:26.143193 21425 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:26.144459 21425 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:26.154845 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.174238 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:26.177950 13840 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:26.182304 13943 raft_consensus.cc:1275] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 28 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 13:07:26.185093 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 104915us
I20260709 13:07:26.187634 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:26.188977 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:26.105386 21409 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
W20260709 13:07:26.192451 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 76641.6us
I20260709 13:07:26.163655 21365 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 23 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:26.199003 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.183887 21165 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:07:26.183873 21444 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:26.193185 21372 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:26.206406 21372 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.206950 21372 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:26.210273 21463 consensus_queue.cc:1048] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:07:26.215817 21345 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:26.218822 13915 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.219640 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:26.221276 13915 leader_election.cc:336] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.228515 21467 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:26.182015 21297 raft_consensus.cc:3055] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:26.237711 21297 raft_consensus.cc:740] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:26.190969 21265 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:07:26.238723 21297 consensus_queue.cc:260] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:26.240037 21297 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:26.237780 13915 leader_election.cc:336] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.240796 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:26.239754 21372 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:26.260478 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 150372us
W20260709 13:07:26.260479 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 61557.5us
W20260709 13:07:26.266147 14524 consensus_peers.cc:597] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:26.316461 13987 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.260545 21473 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:26.263646 21467 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:26.263794 21357 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:26.286592 21363 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:26.287508 21365 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:26.309000 13989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.316537 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 50469us
I20260709 13:07:26.321424 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:26.233594 21468 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:26.331713 14524 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:26.332661 14524 leader_election.cc:336] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Timed out: RequestConsensusVote RPC to 127.12.109.67:46599 timed out after 1.614s (SENT)
I20260709 13:07:26.333232 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.247949 21452 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:26.335260 12725 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 13900)
I20260709 13:07:26.340710 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:26.348560 13987 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.323908 21300 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:26.352450 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.338332 21345 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:26.368256 13987 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.336841 21322 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:07:26.367973 21357 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:26.323525 21346 consensus_queue.cc:1059] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:07:26.348300 21018 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:26.347981 21265 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:26.325522 21363 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:26.379194 13838 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:26.372073 21281 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:26.371881 21252 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:07:26.398090 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.399183 13989 leader_election.cc:336] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:26.417742 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 43781us
I20260709 13:07:26.417989 13989 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:07:26.421370 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 51096.1us
I20260709 13:07:26.392221 21345 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:26.426149 21425 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:26.425604 21018 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:26.458333 13989 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.458854 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:26.399657 21265 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:26.464506 21344 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:26.421109 21281 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:26.466511 13942 raft_consensus.cc:1240] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:26.478487 21265 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:26.480660 13765 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57626 (request call id 250) took 25460 ms (client timeout 15000 ms). Trace:
I20260709 13:07:26.476125 21464 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:26.468925 21425 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 20 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:26.493992 21464 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:26.502718 21472 consensus_queue.cc:1059] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:26.503790 21472 raft_consensus.cc:3055] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:26.503046 13767 catalog_manager.cc:5697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 12 to 13, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 13 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:26.504161 21472 raft_consensus.cc:740] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:26.468998 21363 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:26.510625 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 54008.3us
W20260709 13:07:26.485592 13765 rpcz_store.cc:269] 0709 13:07:01.031262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.032561 (+  1299us) service_pool.cc:224] Handling call
0709 13:07:26.480549 (+25447988us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:23.627956 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:23.629052 (+  1096us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:23.629129 (+    77us) write_op.cc:435] Acquired schema lock
0709 13:07:23.629150 (+    21us) tablet.cc:662] Decoding operations
0709 13:07:23.915408 (+286258us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:23.931288 (+ 15880us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:23.936718 (+  5430us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:07:24.148193 (+211475us) tablet.cc:701] Row locks acquired
0709 13:07:24.148239 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:07:24.148453 (+   214us) write_op.cc:270] Start()
0709 13:07:24.148612 (+   159us) write_op.cc:276] Timestamp: P: 1783602444148374 usec, L: 0
0709 13:07:24.148660 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 13:07:24.217647 (+ 68987us) log.cc:844] Serialized 8258 byte log entry
0709 13:07:25.109235 (+891588us) op_driver.cc:464] REPLICATION: finished
0709 13:07:25.181704 (+ 72469us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c03290ac 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c6e69ef0
0709 13:07:25.236730 (+ 55026us) write_op.cc:301] APPLY: starting
0709 13:07:25.236843 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:25.513276 (+276433us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:25.513338 (+    62us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:26.039957 (+526619us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:26.055094 (+ 15137us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:07:26.055161 (+    67us) write_op.cc:312] APPLY: finished
0709 13:07:26.055739 (+   578us) log.cc:844] Serialized 188 byte log entry
0709 13:07:26.115283 (+ 59544us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:26.126429 (+ 11146us) write_op.cc:454] Released schema lock
0709 13:07:26.126529 (+   100us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:26.127340 (+   811us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:26.127385 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2397,"reported_tablets":52,"spinlock_wait_cycles":15232,"child_traces":[["op",{"apply.queue_time_us":127183,"apply.run_cpu_time_us":64147,"apply.run_wall_time_us":890595,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91781,"num_ops":21,"prepare.queue_time_us":232717,"prepare.run_cpu_time_us":89287,"prepare.run_wall_time_us":936974,"raft.queue_time_us":277847,"raft.run_cpu_time_us":7905,"raft.run_wall_time_us":141529,"replication_time_us":958675,"spinlock_wait_cycles":5716096,"thread_start_us":802321,"threads_started":5,"wal-append.queue_time_us":581844,"wal-append.run_cpu_time_us":47797,"wal-append.run_wall_time_us":1290020}]]}
I20260709 13:07:26.513389 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:26.516515 13989 consensus_peers.cc:597] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:26.518829 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:26.493145 21409 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:26.533941 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 165759us
W20260709 13:07:26.517423 13989 consensus_peers.cc:597] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:26.550915 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.551980 21018 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:26.553505 21252 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:26.554854 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.550190 13767 catalog_manager.cc:5697] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 13 to 17, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 17 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:26.556435 13987 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.509279 21473 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:26.558804 13989 leader_election.cc:336] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:26.559712 21480 log.cc:927] Time spent T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:07:26.561344 21281 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:26.543509 21409 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:26.569662 21473 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:26.571794 21018 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:26.572875 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.572368 13767 catalog_manager.cc:5697] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:26.565985 21252 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:26.520242 21468 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:26.545024 21477 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:07:26.526384 21372 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:26.579649 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48405.9us
I20260709 13:07:26.605365 13947 raft_consensus.cc:1240] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:26.611240 13943 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:07:26.608606 13759 catalog_manager.cc:4755] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6b098cc2d51e471cb2c9c1b17eebcb57: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:07:26.556231 21344 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:07:26.616959 21300 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:26.662257 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46915us
I20260709 13:07:26.663386 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.616586 21425 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:26.670976 14559 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:07:26.667991 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:26.646579 21265 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:26.678633 21471 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:26.681396 21322 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:26.683913 21323 consensus_queue.cc:1059] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:26.685249 21068 raft_consensus.cc:3055] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:26.685665 21068 raft_consensus.cc:740] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:26.686518 21068 consensus_queue.cc:260] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:26.688045 21068 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:26.690356 21473 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:26.702824 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:26.622606 21464 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:26.706835 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:26.707576 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:26.686538 21485 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:07:26.696388 21265 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:26.640504 21472 consensus_queue.cc:260] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:26.715284 13767 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 11 to 19, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 19 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:26.723956 21265 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:26.718231 21452 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:26.724790 21472 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:26.739810 21265 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.741729 21471 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:26.782562 21265 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:26.747917 21365 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:26.762635 21409 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:26.717348 21475 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:26.765251 21165 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.768947 21357 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:26.736049 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:26.743283 21363 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:26.795224 21297 consensus_meta.cc:287] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.002s
W20260709 13:07:26.832175 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 96236.8us
I20260709 13:07:26.832782 21357 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:26.833977 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:26.834689 14018 raft_consensus.cc:2376] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e1649107c781428692458f10597c5742 in term 17. Re-sending same reply.
I20260709 13:07:26.836364 14019 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:07:26.836515 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:26.801453 21464 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:26.837297 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:26.826279 21453 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:26.838558 21453 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:26.840888 21453 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:26.843308 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:26.798480 13915 leader_election.cc:336] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:26.805229 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 95925.6us
I20260709 13:07:26.831571 21165 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:26.852617 21165 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:26.836764 14019 pending_rounds.cc:85] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 10
I20260709 13:07:26.854307 14019 pending_rounds.cc:107] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:07:26.854626 14019 raft_consensus.cc:2889] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:26.850477 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 52041.1us
W20260709 13:07:26.857676 13915 leader_election.cc:336] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.857955 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:26.858338 13915 leader_election.cc:336] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:26.859017 13915 leader_election.cc:336] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.860253 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:26.859777 13915 leader_election.cc:336] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:26.802385 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 61007us
I20260709 13:07:26.864259 21265 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:26.868628 21475 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:26.870434 13943 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:26.871765 21475 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:26.879317 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:26.888576 13949 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:07:26.892464 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:26.893923 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:26.894279 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:26.895421 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.888873 13915 consensus_peers.cc:597] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:26.883407 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:26.898619 14024 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:26.894869 14552 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 14.
I20260709 13:07:26.901314 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:26.902735 13943 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:26.906800 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:26.896288 21409 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:26.914311 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 63516us
I20260709 13:07:26.915467 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:26.916303 14019 raft_consensus.cc:2468] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 17.
I20260709 13:07:26.920027 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:26.920523 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:26.923188 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 65316.6us
W20260709 13:07:26.903785 13915 consensus_peers.cc:597] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:26.896533 21489 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.181s	user 0.004s	sys 0.000s
I20260709 13:07:26.926290 13942 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:07:26.924387 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:26.974846 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:26.904848 21488 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.904769 21357 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:26.990348 14524 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.988253 13839 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:26.904400 21300 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:26.913565 21372 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.913841 21345 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:26.998775 13945 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:27.012245 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:27.001808 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:27.011224 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:26.988511 21475 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:26.913357 21409 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.028780 14019 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 23.
I20260709 13:07:27.032279 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:27.027997 13838 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:26.976881 21494 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 13:07:26.990864 21344 consensus_queue.cc:1059] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:07:26.999874 14024 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 28.
I20260709 13:07:27.064865 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:27.066668 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:27.068815 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:27.070425 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:27.030145 21068 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:27.007718 21357 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.034317 21452 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:27.020751 21372 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.092567 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:27.029773 21363 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:27.087661 14024 raft_consensus.cc:2368] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19.
I20260709 13:07:27.005033 21488 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:27.015810 21300 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.093289 14018 raft_consensus.cc:2468] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:27.103438 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:27.104054 14018 raft_consensus.cc:2468] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:27.105531 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:27.099628 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:27.100497 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 72599.6us
W20260709 13:07:27.100332 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:27.100950 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 176640us
I20260709 13:07:27.076596 13943 raft_consensus.cc:2468] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:27.121538 14024 raft_consensus.cc:2468] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:27.122710 14024 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:27.123099 14024 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:27.086328 21068 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:27.130043 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:27.130730 14018 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:27.141032 14019 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 15: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:27.146394 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 213730us
I20260709 13:07:27.148880 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:27.150215 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:27.103322 21500 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:27.156037 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:27.135914 21068 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:27.167001 21505 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:07:27.171707 21357 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:27.173765 21372 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:27.178560 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:27.199048 14018 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:27.129228 21496 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:27.123970 14024 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.220561 21414 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:27.244345 14024 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:27.100287 21363 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.098382 21452 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:27.185794 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 164499us
I20260709 13:07:27.175746 21451 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:27.174952 21409 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:27.218982 21444 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:27.175391 21321 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:07:27.248344 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:27.266384 14018 raft_consensus.cc:2468] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:27.267642 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:27.261147 14024 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:07:27.229826 21413 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.231443 21467 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.270205 14024 pending_rounds.cc:85] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 12
I20260709 13:07:27.225860 21365 consensus_queue.cc:1059] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:07:27.281855 14024 pending_rounds.cc:107] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:07:27.295591 21363 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:27.317677 14024 raft_consensus.cc:2889] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:27.297333 21509 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:07:27.297791 21365 consensus_queue.cc:1243] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [LEADER]: Peer 3779649036364744823fc035dac892c7 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:27.301653 21357 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:27.302656 21488 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:27.305217 21452 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:27.285225 21467 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.306308 21508 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:27.306597 21451 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.320328 21451 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:27.321237 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 200583us
I20260709 13:07:27.309866 21475 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:27.311792 21300 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:27.312801 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.323597 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:27.322487 21451 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:27.315099 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:27.324823 21323 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:27.325937 21323 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:07:27.321913 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 171104us
I20260709 13:07:27.332788 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:27.335352 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:27.336337 13942 raft_consensus.cc:1240] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:07:27.336900 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:27.338207 12725 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 13900)
W20260709 13:07:27.340500 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 135638us
I20260709 13:07:27.341369 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:27.343189 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:27.351294 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:27.353415 13840 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:07:27.284843 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 134692us
I20260709 13:07:27.270375 21372 raft_consensus.cc:3055] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:27.355439 21372 raft_consensus.cc:740] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:27.355465 21496 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:07:27.356225 21496 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 28 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:27.356564 21372 consensus_queue.cc:260] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.357178 21496 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:27.351315 21323 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:27.358456 21323 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:27.358844 21323 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:27.366238 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:27.371894 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:27.373740 13915 leader_election.cc:400] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19.
I20260709 13:07:27.357921 21372 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:27.323098 21363 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:07:27.351161 21346 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:27.384722 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:27.378706 13915 leader_election.cc:403] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:07:27.283219 21413 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:27.405047 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:27.374678 21452 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:27.417306 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:27.421525 13942 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:27.423066 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:27.427466 13987 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:27.428361 21533 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 18. Current term is 19.
I20260709 13:07:27.436539 21413 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:27.437458 14559 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 13.
I20260709 13:07:27.418864 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:27.448673 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 43551.8us
I20260709 13:07:27.450284 13915 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:27.462761 13947 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:07:27.477973 13945 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:27.408313 21519 raft_consensus.cc:2804] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:27.447927 21346 raft_consensus.cc:2804] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:27.342716 21508 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:27.409308 21525 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:27.447052 21475 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.494999 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:27.452221 21527 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:27.454589 21409 raft_consensus.cc:2804] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:27.487061 13838 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:27.545133 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58220.9us
I20260709 13:07:27.543174 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:27.479049 21533 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:27.478825 21344 raft_consensus.cc:2804] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:27.454641 21265 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:07:27.482712 21519 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:27.551077 13945 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 12.
W20260709 13:07:27.572356 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 116463us
W20260709 13:07:27.575670 14524 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:27.504185 21346 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:27.555837 21529 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 13:07:27.590742 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:27.562881 21493 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:07:27.595134 13914 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:27.550607 21409 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:27.564392 21363 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:27.561548 21490 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:27.643281 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45733.8us
I20260709 13:07:27.589253 21346 raft_consensus.cc:3060] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:27.548054 21475 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.586807 21519 raft_consensus.cc:3060] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:27.595855 21533 raft_consensus.cc:697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:27.749035 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 294652us
I20260709 13:07:27.645556 21300 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.649050 21409 raft_consensus.cc:3060] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:27.648502 21453 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:07:27.752182 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 205591us
W20260709 13:07:27.730719 21068 consensus_meta.cc:287] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.008s
W20260709 13:07:27.733291 21485 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:07:27.647294 21363 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:27.732085 21529 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:07:27.655213 21527 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.745438 21467 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:27.758972 21516 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.764297 21453 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:27.792536 21363 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:27.808177 21265 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.832122 21544 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.807847 21534 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.781826 21300 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.794893 21068 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:27.847716 21541 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:07:27.850273 21535 log.cc:927] Time spent T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
W20260709 13:07:27.851420 21540 log.cc:927] Time spent T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 13:07:27.852391 21537 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:07:27.857050 21300 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:27.857980 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2975) took 1021 ms. Trace:
I20260709 13:07:27.857867 21533 consensus_queue.cc:237] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.801302 21527 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.860146 21508 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:27.861367 21534 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:27.840960 21265 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:27.863760 21522 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.325s	user 0.000s	sys 0.000s
I20260709 13:07:27.832203 21467 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:27.864773 21521 log.cc:927] Time spent T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:07:27.866921 21475 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:27.867653 21544 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.818069 21516 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.956248 21516 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:27.870170 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:27.870990 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:27.872270 21500 raft_consensus.cc:3055] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:27.960214 21500 raft_consensus.cc:740] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:27.809194 21532 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.887949 21490 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:27.891167 21363 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.829700 21453 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:27.919324 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2982) took 1070 ms. Trace:
W20260709 13:07:27.930917 14024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.003s	sys 0.000s
W20260709 13:07:27.939411 13841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.931687 14016 rpcz_store.cc:276] 0709 13:07:26.836096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.846083 (+  9987us) service_pool.cc:224] Handling call
0709 13:07:26.908280 (+ 62197us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:07:26.908522 (+   242us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.102868 (+194346us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:27.212747 (+109879us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.329432 (+116685us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.353602 (+ 24170us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.354303 (+   701us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.354544 (+   241us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:27.355039 (+   495us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.424683 (+ 69644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.424868 (+   185us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:27.857737 (+432869us) raft_consensus.cc:1685] finished
0709 13:07:27.857761 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.857913 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":164346,"prepare.queue_time_us":145492,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":113,"raft.queue_time_us":248888,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":12680,"spinlock_wait_cycles":68969088,"thread_start_us":294620,"threads_started":3,"wal-append.queue_time_us":198271}
I20260709 13:07:27.944905 21545 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:27.964325 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 432864us
I20260709 13:07:27.930512 21473 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.982152 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:27.983054 14016 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 13: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:27.981775 21473 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.981624 21545 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.985108 21473 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:27.981073 14024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.004s	sys 0.000s
I20260709 13:07:27.977525 13949 raft_consensus.cc:1240] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:07:27.985656 21545 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:27.987668 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:27.988804 14016 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:27.989826 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:27.990653 14016 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 13.
I20260709 13:07:27.991729 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:27.992479 14016 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:27.979081 14020 rpcz_store.cc:276] 0709 13:07:26.848642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.349640 (+500998us) service_pool.cc:224] Handling call
0709 13:07:27.350026 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.406718 (+ 56692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.406758 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.407243 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.516226 (+108983us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:27.589205 (+ 72979us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.918974 (+329769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.919065 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:27.919106 (+    41us) raft_consensus.cc:1685] finished
0709 13:07:27.919120 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.919259 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":409696,"prepare.queue_time_us":76671,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":262845,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":5816,"spinlock_wait_cycles":7661952,"thread_start_us":267532,"threads_started":2,"wal-append.queue_time_us":470962}
I20260709 13:07:27.974946 21532 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:27.993999 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:27.994673 14020 raft_consensus.cc:2468] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:27.996197 21532 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:27.998090 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:27.998513 21544 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:27.998699 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2969) took 1164 ms. Trace:
I20260709 13:07:27.999285 14015 rpcz_store.cc:276] 0709 13:07:26.834419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.841402 (+  6983us) service_pool.cc:224] Handling call
0709 13:07:26.873126 (+ 31724us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:26.878659 (+  5533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:26.878688 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:26.921652 (+ 42964us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.320537 (+398885us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:27.320851 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.341413 (+ 20562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.341537 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:27.589030 (+247493us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b18003c10c0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:07:27.998284 (+409254us) raft_consensus.cc:1685] finished
0709 13:07:27.998441 (+   157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.998619 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81738,"prepare.queue_time_us":2029,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":51144,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":24513,"spinlock_wait_cycles":31797376,"thread_start_us":319871,"threads_started":1,"wal-append.queue_time_us":416886}
I20260709 13:07:28.003916 21507 consensus_queue.cc:1059] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:28.006973 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.007676 13943 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:28.008481 21534 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:28.011176 21527 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.012010 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:28.013487 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.001922 21508 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.014632 21508 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.015771 21363 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:27.962280 21467 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.016983 21508 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:28.018105 21506 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:28.018576 21506 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.018767 21467 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.018966 21506 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:27.948586 21511 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.019578 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.019788 21511 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:28.022687 21514 raft_consensus.cc:3055] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:28.023164 21514 raft_consensus.cc:740] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:28.024207 21514 consensus_queue.cc:260] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.025679 13948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35894 (request call id 3490) took 1151 ms. Trace:
I20260709 13:07:28.026306 13948 rpcz_store.cc:276] 0709 13:07:26.874337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.874658 (+   321us) service_pool.cc:224] Handling call
0709 13:07:26.874981 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:26.875283 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:26.875310 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:26.897529 (+ 22219us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.314540 (+417011us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:07:27.314579 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.314649 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.025380 (+710731us) raft_consensus.cc:1685] finished
0709 13:07:28.025422 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.025599 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12019,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":11743,"thread_start_us":325739,"threads_started":1,"wal-append.queue_time_us":492228}
I20260709 13:07:27.918854 21541 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.001585 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2976) took 1155 ms. Trace:
I20260709 13:07:27.941509 21409 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.037308 21518 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:28.038018 21518 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.048619 21518 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.054421 21265 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.056340 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:28.056850 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:28.057471 14558 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:07:28.059352 21541 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.034310 14017 rpcz_store.cc:276] 0709 13:07:26.846685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.846813 (+   128us) service_pool.cc:224] Handling call
0709 13:07:26.909414 (+ 62601us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b5800193e98. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030ea22 00007fb2c9227c09 00007fb2c8381ee0 00007fb2c844b0ab 00007fb2c844b131 00007fb2c840cfc0 00007fb2c83f7929 00007fb2c83f73d2 00007fb2c196ebd4
0709 13:07:26.909639 (+   225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.369646 (+460007us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.369679 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.370178 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.460568 (+ 90390us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:27.461374 (+   806us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.621993 (+160619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.001273 (+379280us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b540023f854. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:28.001324 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.001384 (+    60us) raft_consensus.cc:1685] finished
0709 13:07:28.001398 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.001526 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":321142,"prepare.queue_time_us":184,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":359175,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":94179,"spinlock_wait_cycles":556156928,"thread_start_us":139421,"threads_started":2,"wal-append.queue_time_us":161778}
I20260709 13:07:28.063339 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:28.025840 21514 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:28.065155 14522 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:27.998827 14561 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:07:27.961223 21500 consensus_queue.cc:260] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.072368 21511 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:27.941768 21344 raft_consensus.cc:697] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:28.074973 21344 consensus_queue.cc:237] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.038955 21409 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.044999 21068 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:28.094316 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 62119.3us
I20260709 13:07:28.002925 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:28.069921 21500 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:28.104172 21323 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.745s	user 0.009s	sys 0.006s
W20260709 13:07:28.072692 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:28.105168 21323 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:28.107661 13988 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:28.111735 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.115280 21068 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:28.124979 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2981) took 1276 ms. Trace:
I20260709 13:07:28.139760 14552 raft_consensus.cc:3055] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:07:28.140349 14552 raft_consensus.cc:740] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:07:28.142380 14022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.011s	sys 0.012s
W20260709 13:07:28.126495 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:28.143801 14022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.011s	sys 0.012s
I20260709 13:07:28.127506 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:28.128988 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:28.133051 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 68028.2us
I20260709 13:07:28.140146 14024 rpcz_store.cc:276] 0709 13:07:26.848408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.318830 (+470422us) service_pool.cc:224] Handling call
0709 13:07:27.319219 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.341043 (+ 21824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.341176 (+   133us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.352126 (+ 10950us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.460768 (+108642us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b48002115d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:07:27.985959 (+525191us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:27.986328 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 13:07:28.033981 (+ 47653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.034088 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.124620 (+ 90532us) raft_consensus.cc:1685] finished
0709 13:07:28.124752 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.124901 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":205846,"prepare.queue_time_us":376,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":71243,"raft.run_cpu_time_us":1191,"raft.run_wall_time_us":116341,"spinlock_wait_cycles":10910464,"thread_start_us":621107,"threads_started":1,"wal-append.queue_time_us":696064}
I20260709 13:07:28.104686 14017 raft_consensus.cc:2410] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 13:07:28.133842 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 368695us
I20260709 13:07:28.135615 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:28.147923 14553 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:28.149005 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:28.150746 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:28.160794 14016 raft_consensus.cc:2393] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:28.145256 13988 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:28.174952 13988 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:28.160295 14552 consensus_queue.cc:260] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:28.179744 14552 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:28.186489 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:28.190877 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.207294 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.208348 13945 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:28.223218 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3508) took 1323 ms. Trace:
I20260709 13:07:28.223820 14016 rpcz_store.cc:276] 0709 13:07:26.899668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.222099 (+1322431us) service_pool.cc:224] Handling call
0709 13:07:28.222523 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.222784 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.222821 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.222881 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:28.222934 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.223004 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.223142 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.230650 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.231572 13947 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:28.144070 21281 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:28.239207 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 53045.9us
W20260709 13:07:28.234737 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 48349.5us
I20260709 13:07:28.243054 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.244586 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3509) took 1344 ms. Trace:
I20260709 13:07:28.245113 14016 rpcz_store.cc:276] 0709 13:07:26.899937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.237219 (+1337282us) service_pool.cc:224] Handling call
0709 13:07:28.237643 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.237876 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.237907 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.237967 (+    60us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:28.238011 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.244285 (+  6274us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.244510 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.243933 13947 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 18.
I20260709 13:07:28.161734 21300 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:28.210886 21372 consensus_meta.cc:287] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.785s	user 0.011s	sys 0.004s
I20260709 13:07:28.277534 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:28.248195 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3512) took 1347 ms. Trace:
I20260709 13:07:28.279749 14016 rpcz_store.cc:276] 0709 13:07:26.900695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.246369 (+1345674us) service_pool.cc:224] Handling call
0709 13:07:28.246718 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.247753 (+  1035us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.247797 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.247861 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:28.247906 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.247982 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.248128 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.220080 21473 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.226832 21357 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:28.185292 21534 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:28.284466 14024 raft_consensus.cc:2410] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:07:28.285812 14016 raft_consensus.cc:1275] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:28.320863 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:28.240295 21548 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.177623 21544 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 13:07:28.286429 21472 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.256968 21322 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:28.298975 21523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.004s	sys 0.018s
I20260709 13:07:28.235985 21265 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
W20260709 13:07:28.309378 14018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.061s	sys 0.171s
W20260709 13:07:28.338665 12725 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 13900)
W20260709 13:07:28.338949 14018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.061s	sys 0.173s
I20260709 13:07:28.339063 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.241437 21281 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.288820 21357 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.316269 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:28.309764 14017 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:07:28.321658 21522 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:07:28.354923 14017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3511) took 1454 ms. Trace:
I20260709 13:07:28.355309 14017 rpcz_store.cc:276] 0709 13:07:26.900440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.249239 (+1348799us) service_pool.cc:224] Handling call
0709 13:07:28.287011 (+ 37772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.354649 (+ 67638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.354864 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.316751 14553 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 16.
I20260709 13:07:28.161367 21346 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:28.238365 21516 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:28.367403 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 46643.6us
I20260709 13:07:28.270711 14552 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 28 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 13:07:28.350435 21523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.004s	sys 0.020s
I20260709 13:07:28.169375 21519 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:28.324143 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3514) took 1422 ms. Trace:
I20260709 13:07:28.385783 14016 rpcz_store.cc:276] 0709 13:07:26.901666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.284995 (+1383329us) service_pool.cc:224] Handling call
0709 13:07:28.285369 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.322507 (+ 37138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.324064 (+  1557us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:28.346129 14019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.009s	sys 0.022s
W20260709 13:07:28.388155 14019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.009s	sys 0.023s
I20260709 13:07:28.159677 21068 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:28.352455 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:28.324466 21496 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.033s	sys 0.065s
I20260709 13:07:28.252120 21527 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:07:28.372146 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.375314 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3502) took 1477 ms. Trace:
I20260709 13:07:28.376035 21475 raft_consensus.cc:2804] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:28.415666 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:28.358336 21357 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:28.380389 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2971) took 1545 ms. Trace:
I20260709 13:07:28.363145 21511 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:28.419690 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:28.391579 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2980) took 1543 ms. Trace:
I20260709 13:07:28.387351 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:28.353783 21281 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:28.398351 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 132654us
W20260709 13:07:28.400862 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 44732.3us
I20260709 13:07:28.304821 21473 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.323925 21534 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.354594 21548 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.424450 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:28.440125 14016 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:28.440274 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:28.440985 14024 raft_consensus.cc:2468] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:28.441131 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:28.441378 13988 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:07:28.405099 21496 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.035s	sys 0.067s
I20260709 13:07:28.420436 14019 rpcz_store.cc:276] 0709 13:07:26.848170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.300530 (+452360us) service_pool.cc:224] Handling call
0709 13:07:27.301022 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.314438 (+ 13416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.314471 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.338466 (+ 23995us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.477079 (+138613us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:27.594444 (+117365us) log.cc:844] Serialized 14 byte log entry
0709 13:07:28.391174 (+796730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.391266 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.391313 (+    47us) raft_consensus.cc:1685] finished
0709 13:07:28.391326 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.391491 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":261942,"prepare.queue_time_us":112164,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":26959,"raft.queue_time_us":216674,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":20143,"thread_start_us":730665,"threads_started":2,"wal-append.queue_time_us":466954}
I20260709 13:07:28.417482 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.353377 21322 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:28.355751 21506 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.359555 21472 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:28.456476 13759 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 179735us
I20260709 13:07:28.441817 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:28.414834 14022 rpcz_store.cc:276] 0709 13:07:26.898093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.401635 (+503542us) service_pool.cc:224] Handling call
0709 13:07:27.402114 (+   479us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:27.402525 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.402552 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:27.403252 (+   700us) log.cc:844] Serialized 52 byte log entry
0709 13:07:28.006347 (+603095us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b48002115d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:07:28.145300 (+138953us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:28.145351 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.145430 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.374961 (+229531us) raft_consensus.cc:1685] finished
0709 13:07:28.374990 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.375241 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5088,"raft.run_cpu_time_us":4955,"raft.run_wall_time_us":598186,"spinlock_wait_cycles":13760384,"thread_start_us":741623,"threads_started":1,"wal-append.queue_time_us":969227}
I20260709 13:07:28.416858 21527 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.387485 21516 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.449714 13840 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:28.455576 13839 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:28.418179 21475 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:28.418562 14023 rpcz_store.cc:276] 0709 13:07:26.834984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.838848 (+  3864us) service_pool.cc:224] Handling call
0709 13:07:26.839147 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.839367 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:26.839700 (+   333us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.303150 (+463450us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b48002115d8. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c835caa1 00007fb2c8353b37 00007fb2c83539ca 00007fb2c844b26f 00007fb2c844af37 00007fb2c845f55c 00007fb2c845f508 00007fb2c845f488 00007fb2c845f443 00007fb2c845e6ec 00007fb2c7388a96
0709 13:07:27.995105 (+691955us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b1800178240. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c2fdb4bc 00007fb2c032948b 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0343455 00007fb2c6e2b3c7 00007fb2c343d9a7 00007fb2c344d0bf
0709 13:07:27.995407 (+   302us) log.cc:844] Serialized 14 byte log entry
0709 13:07:28.379977 (+384570us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.380035 (+    58us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:28.380098 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.380198 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.380325 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76047,"prepare.queue_time_us":107212,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":4473,"raft.queue_time_us":154223,"raft.run_cpu_time_us":3765,"raft.run_wall_time_us":280565,"spinlock_wait_cycles":88707712,"thread_start_us":815948,"threads_started":4,"wal-append.queue_time_us":93721,"wal-append.run_cpu_time_us":1526,"wal-append.run_wall_time_us":1003968}
I20260709 13:07:28.471151 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:28.471856 14019 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:28.442499 21534 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:28.480016 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3504) took 1581 ms. Trace:
I20260709 13:07:28.444191 21546 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.486578 14015 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:28.480819 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:28.495678 13946 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:07:28.483772 14018 rpcz_store.cc:276] 0709 13:07:26.898620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.507990 (+609370us) service_pool.cc:224] Handling call
0709 13:07:27.508416 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.731063 (+222647us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:27.731097 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.731611 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:07:28.339672 (+608061us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:28.339738 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.339843 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.479714 (+139871us) raft_consensus.cc:1685] finished
0709 13:07:28.479746 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.479929 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":225307,"raft.queue_time_us":240155,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":20610,"spinlock_wait_cycles":1506048,"thread_start_us":542549,"threads_started":1,"wal-append.queue_time_us":714428}
I20260709 13:07:28.500643 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:28.501626 14018 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:28.455758 21514 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:28.504730 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 223373us
I20260709 13:07:28.467481 21527 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:28.429069 21511 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.461966 21494 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:28.453223 21518 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:07:28.465632 21488 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:28.471601 21475 raft_consensus.cc:3060] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:28.473210 21323 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:28.513187 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 63534.7us
I20260709 13:07:28.532109 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:28.513456 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 57971.5us
I20260709 13:07:28.545575 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:28.525733 21018 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:28.593287 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 53471.5us
I20260709 13:07:28.585357 14019 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 24.
I20260709 13:07:28.585927 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:28.472954 21545 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.055s
I20260709 13:07:28.526638 21560 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:28.600036 14015 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:28.603451 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 923) took 1699 ms. Trace:
I20260709 13:07:28.487720 21453 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:28.521009 21300 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:28.603883 14015 rpcz_store.cc:276] 0709 13:07:26.903930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.597768 (+1693838us) service_pool.cc:224] Handling call
0709 13:07:28.599840 (+  2072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.603198 (+  3358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.603386 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.600656 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 924) took 1696 ms. Trace:
I20260709 13:07:28.609776 14019 rpcz_store.cc:276] 0709 13:07:26.905208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.599693 (+1694485us) service_pool.cc:224] Handling call
0709 13:07:28.600122 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.600313 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.600339 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.600384 (+    45us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:28.600435 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.600492 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.600594 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.540112 21547 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:28.583776 21488 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.592756 21018 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.584280 21559 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.508347 21546 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:28.616680 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 71250.7us
I20260709 13:07:28.516520 21514 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.642819 21508 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:28.643440 21508 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.506429 21533 consensus_queue.cc:1048] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 13:07:28.625958 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 925) took 1720 ms. Trace:
I20260709 13:07:28.606202 21560 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.658341 21560 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:28.627842 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 926) took 1722 ms. Trace:
I20260709 13:07:28.584298 21555 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:28.638888 13943 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 13.
I20260709 13:07:28.550920 21494 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:28.648311 14015 rpcz_store.cc:276] 0709 13:07:26.905520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.624855 (+1719335us) service_pool.cc:224] Handling call
0709 13:07:28.625234 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.625464 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.625502 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.625569 (+    67us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:28.625617 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.625686 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.625807 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.650979 21322 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:28.654544 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:28.682042 21519 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:28.683485 21548 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.685940 21453 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:28.689152 21068 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.648278 21559 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:28.693703 21570 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:07:28.697253 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.639730 21547 raft_consensus.cc:697] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:28.699087 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.699173 21547 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:28.644485 21546 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.618520 21300 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.644580 21514 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:28.656452 21565 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:07:28.667645 13840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.723453 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 921) took 1820 ms. Trace:
I20260709 13:07:28.644640 21018 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:28.723970 14024 rpcz_store.cc:276] 0709 13:07:26.903412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.722519 (+1819107us) service_pool.cc:224] Handling call
0709 13:07:28.722882 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.723108 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.723147 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.723196 (+    49us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:28.723228 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.723285 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.723395 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.671135 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.725457 13949 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:28.627189 21321 raft_consensus.cc:2804] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:07:28.674019 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.633492 21372 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.700222 13941 raft_consensus.cc:2393] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:28.732923 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 919) took 1830 ms. Trace:
W20260709 13:07:28.657730 13839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:28.710292 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 275468us
W20260709 13:07:28.712913 13759 catalog_manager.cc:4755] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6b098cc2d51e471cb2c9c1b17eebcb57: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:07:28.713730 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3507) took 1814 ms. Trace:
I20260709 13:07:28.735458 14020 rpcz_store.cc:276] 0709 13:07:26.899421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.995328 (+1095907us) service_pool.cc:224] Handling call
0709 13:07:27.995687 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.017761 (+ 22074us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:28.017795 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.018377 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:07:28.362113 (+343736us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:28.362221 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.362362 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.713500 (+351138us) raft_consensus.cc:1685] finished
0709 13:07:28.713522 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.713669 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21563,"raft.queue_time_us":22905,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":333917,"threads_started":1,"wal-append.queue_time_us":485260}
I20260709 13:07:28.713824 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:28.714480 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.715157 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.668233 21473 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.664305 14019 rpcz_store.cc:276] 0709 13:07:26.905792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.626567 (+1720775us) service_pool.cc:224] Handling call
0709 13:07:28.627175 (+   608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.627413 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.627452 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.627510 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:28.627572 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.627646 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.627772 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.728328 14015 raft_consensus.cc:1240] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:28.733387 14022 rpcz_store.cc:276] 0709 13:07:26.902862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.712065 (+1809203us) service_pool.cc:224] Handling call
0709 13:07:28.732173 (+ 20108us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.732435 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.732481 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.732550 (+    69us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:28.732609 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.732695 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.732851 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.739028 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:28.740423 14024 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 13:07:28.749290 14024 pending_rounds.cc:85] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 11
I20260709 13:07:28.749580 14024 pending_rounds.cc:107] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 13:07:28.749831 14024 raft_consensus.cc:2889] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:28.737803 13945 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
I20260709 13:07:28.753652 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:07:28.755168 21569 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:07:28.757473 21472 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.731276 21372 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.760778 21475 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.675079 21346 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:28.762480 14021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49644 (request call id 3501) took 1864 ms. Trace:
I20260709 13:07:28.762876 21516 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.729905 21321 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.763757 21321 raft_consensus.cc:3060] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:28.766705 21568 log.cc:927] Time spent T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.180s	user 0.003s	sys 0.000s
I20260709 13:07:28.768169 14022 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:07:28.768770 14022 pending_rounds.cc:85] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 12
I20260709 13:07:28.769181 14022 pending_rounds.cc:107] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:07:28.769572 14022 raft_consensus.cc:2889] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:28.770550 14022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2997) took 1016 ms. Trace:
I20260709 13:07:28.771126 14022 rpcz_store.cc:276] 0709 13:07:27.753962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.748814 (+994852us) service_pool.cc:224] Handling call
0709 13:07:28.767649 (+ 18835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.770208 (+  2559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.770478 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.762876 14021 rpcz_store.cc:276] 0709 13:07:26.897690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.363778 (+466088us) service_pool.cc:224] Handling call
0709 13:07:27.396004 (+ 32226us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.466865 (+ 70861us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:27.508459 (+ 41594us) log.cc:844] Serialized 14 byte log entry
0709 13:07:28.012791 (+504332us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b180049b2e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c03463f3 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:07:28.017291 (+  4500us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b180049b2e0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0328e07 00007fb2c9183ab9 00007fb2c034569b 00007fb2c034393f 00007fb2c0345c02 00007fb2c2fcd20b 00007fb2c2fd2d23 00007fb2c2fd323a 00007fb2c343da2a
0709 13:07:28.484237 (+466946us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.485022 (+   785us) log.cc:844] Serialized 26 byte log entry
0709 13:07:28.485217 (+   195us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:28.485265 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.485363 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.762225 (+276862us) raft_consensus.cc:1685] finished
0709 13:07:28.762249 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.762412 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":235586,"prepare.queue_time_us":121746,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":138723,"raft.run_cpu_time_us":452,"raft.run_wall_time_us":55991,"spinlock_wait_cycles":114286720,"thread_start_us":448582,"threads_started":1,"wal-append.queue_time_us":1039729}
I20260709 13:07:28.721341 13838 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:28.761840 21372 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:28.798781 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 77522.6us
I20260709 13:07:28.803704 13838 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:28.825181 13838 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:28.829519 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.834673 21472 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:28.732177 21357 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:28.839905 21357 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:28.821683 21323 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:28.840765 21323 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.841166 21323 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:28.832252 21546 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:28.842623 21546 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.843003 21546 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:28.852504 14021 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 28 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 13:07:28.856606 21494 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:28.857786 14017 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:28.860270 21559 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:28.868475 21500 consensus_meta.cc:287] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.769s	user 0.015s	sys 0.000s
I20260709 13:07:28.870298 13838 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:28.875241 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 105460us
I20260709 13:07:28.886376 21281 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:28.899350 14558 raft_consensus.cc:2368] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:07:28.905346 13838 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:28.909250 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:28.873366 21068 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:28.914827 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.905247 21475 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.900143 21500 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:28.901132 21281 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:28.852383 21534 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:28.929009 13838 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:28.939564 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.941380 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:28.942282 13949 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:28.945719 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.956205 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.957368 13946 raft_consensus.cc:2393] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:28.958196 13945 raft_consensus.cc:2393] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 22: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:28.959213 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.907693 21321 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.963008 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.963534 21559 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:28.967806 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:28.935078 21548 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:28.923113 21068 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.023301 13943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35894 (request call id 3507) took 1010 ms. Trace:
W20260709 13:07:29.024029 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 96395.9us
I20260709 13:07:28.991868 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:29.023725 13943 rpcz_store.cc:276] 0709 13:07:28.013040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.013206 (+   166us) service_pool.cc:224] Handling call
0709 13:07:29.023141 (+1009935us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b5400148cd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:29.023244 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1436253056}
I20260709 13:07:29.026455 13948 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:07:28.964624 21577 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:29.037376 14016 raft_consensus.cc:1240] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 28 FOLLOWER]: Rejecting Update request from peer 4dd265d61fc141a48ca9d3d5c8f52921 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 13:07:29.038087 14016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 920) took 2134 ms. Trace:
I20260709 13:07:29.038395 14016 rpcz_store.cc:276] 0709 13:07:26.903163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.036747 (+2133584us) service_pool.cc:224] Handling call
0709 13:07:29.037140 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.037807 (+   667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.038010 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.971304 21560 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.047019 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:28.978570 21527 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:28.989993 21344 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:28.988976 21490 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 13:07:29.032864 13840 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:29.043238 13914 leader_election.cc:400] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:07:29.048261 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:29.091670 13914 leader_election.cc:403] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:07:29.049055 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:29.034652 13949 raft_consensus.cc:2468] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:29.020134 21548 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.096836 21548 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:29.057297 13948 raft_consensus.cc:1275] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:29.070335 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:29.080482 21511 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:29.093747 14016 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:29.096263 21357 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:29.099208 14017 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:29.036111 21473 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:29.111361 13944 raft_consensus.cc:2368] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19.
W20260709 13:07:29.100208 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 67477.1us
I20260709 13:07:29.112592 13840 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:07:29.101751 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 60607.6us
I20260709 13:07:29.114967 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:29.116569 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:29.117496 14017 raft_consensus.cc:2368] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:07:29.103896 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:29.109335 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:29.110183 13987 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:29.124368 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:29.100596 21586 log.cc:927] Time spent T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:07:29.020711 21559 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.135905 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:29.146260 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:29.147172 14021 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:29.108928 21533 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:29.101424 21372 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:29.109778 21475 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:29.122354 21473 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.153254 13987 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:29.047504 21577 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:29.141176 21544 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:29.143429 21511 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:29.100783 21344 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.121743 21357 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:29.202755 21578 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:07:29.203377 21534 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:29.173467 21473 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:29.207345 21559 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:29.141393 21500 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:07:29.142323 21464 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:07:29.143237 21165 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.054s
I20260709 13:07:29.197003 21511 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.194965 21582 raft_consensus.cc:2804] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:29.249665 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:29.178200 21372 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:29.255036 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 268976us
I20260709 13:07:29.248413 21511 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:29.267792 21580 log.cc:927] Time spent T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:07:29.269428 21321 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:29.271791 21491 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:29.355183 21491 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
W20260709 13:07:29.355615 21588 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:07:29.356483 21491 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:29.211434 21514 consensus_queue.cc:1048] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
W20260709 13:07:29.358417 21546 consensus_meta.cc:287] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.002s
I20260709 13:07:29.242214 21583 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:29.359046 21546 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.281791 14018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 922) took 2378 ms. Trace:
I20260709 13:07:29.237906 21544 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 24 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:29.362597 14018 rpcz_store.cc:276] 0709 13:07:26.903676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.597189 (+1693513us) service_pool.cc:224] Handling call
0709 13:07:28.597491 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.710415 (+112924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.710449 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.710968 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:07:28.794092 (+ 83124us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:29.012847 (+218755us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.013123 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.281187 (+268064us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540023c654. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c3433e01 00007fb2c343283e 00007fb2c83f7419 00007fb2c196ebd4 00007fb2c196eb62 00007fb2c196ea91 00007fb2c196ea0c 00007fb2c196dcc2 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:29.281268 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.281344 (+    76us) raft_consensus.cc:1685] finished
0709 13:07:29.281492 (+   148us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.281703 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18281,"prepare.queue_time_us":500880,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":7715,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":12440,"spinlock_wait_cycles":258623232,"thread_start_us":301091,"threads_started":2,"wal-append.queue_time_us":83191}
I20260709 13:07:29.307328 14020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2995) took 1554 ms. Trace:
I20260709 13:07:29.310976 21560 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:29.365325 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 351147us
I20260709 13:07:29.311070 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:29.350112 21357 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:29.330582 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 409534us
I20260709 13:07:29.379297 14019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2996) took 1625 ms. Trace:
I20260709 13:07:29.379851 14019 rpcz_store.cc:276] 0709 13:07:27.753663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.748415 (+994752us) service_pool.cc:224] Handling call
0709 13:07:28.748781 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.763844 (+ 15063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.763896 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.766137 (+  2241us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.017858 (+251721us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:29.367921 (+350063us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.378946 (+ 11025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.379037 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.379083 (+    46us) raft_consensus.cc:1685] finished
0709 13:07:29.379100 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.379234 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14541,"prepare.queue_time_us":288722,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":15132,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"thread_start_us":573412,"threads_started":2,"wal-append.queue_time_us":421348}
I20260709 13:07:29.333003 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:29.337344 14523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.339231 12725 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 13900)
I20260709 13:07:29.342693 21534 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.345567 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:29.256338 21582 raft_consensus.cc:697] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:29.333006 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:29.507817 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:29.508657 14553 raft_consensus.cc:2393] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:29.290632 21344 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:29.360591 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:29.369824 14020 rpcz_store.cc:276] 0709 13:07:27.753309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.745501 (+992192us) service_pool.cc:224] Handling call
0709 13:07:28.745936 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.746445 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.746485 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.747090 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.126751 (+379661us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:29.294153 (+167402us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.306811 (+ 12658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.306941 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.307011 (+    70us) raft_consensus.cc:1685] finished
0709 13:07:29.307031 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.307238 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":205474,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":12477,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":244,"thread_start_us":442702,"threads_started":2,"wal-append.queue_time_us":412763}
I20260709 13:07:29.327672 21508 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:29.330910 21577 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:29.393090 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.503430 21323 consensus_meta.cc:287] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.662s	user 0.014s	sys 0.005s
I20260709 13:07:29.364785 21491 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:29.364267 21583 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.407497 21300 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:29.519392 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:29.408010 21372 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:29.408849 21488 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:29.411221 13944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35884 (request call id 3393) took 1144 ms. Trace:
W20260709 13:07:29.539492 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 276481us
I20260709 13:07:29.413900 21556 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:29.415553 21496 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.416064 21562 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:29.564693 21562 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.567375 21562 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:29.417860 21506 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:29.418419 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:29.401012 21514 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:29.568704 21547 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:29.568912 21514 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.569641 21547 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:29.571376 21514 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:29.452262 21527 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:29.478109 21588 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:07:29.481163 21560 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:29.501989 21593 log.cc:927] Time spent T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:07:29.454432 21464 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:29.499388 21533 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851) with higher term. Message: Invalid argument: T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:07:29.520715 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:29.521351 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:29.526649 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:29.542649 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:29.589792 13945 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:07:29.550558 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:29.591674 14016 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:29.539819 13944 rpcz_store.cc:276] 0709 13:07:28.266874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.276378 (+  9504us) service_pool.cc:224] Handling call
0709 13:07:29.410983 (+1134605us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b54001478d4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339c571 00007fb2c34372fa 00007fb2c83f8b17 00007fb2c19736d4 00007fb2c1973662 00007fb2c1973591 00007fb2c197350c 00007fb2c19727c2 00007fb2c176e48c 00007fb2c176d8c6 00007fb2c176d26f
0709 13:07:29.411154 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1915060608}
I20260709 13:07:29.516903 21321 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.583027 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:29.511345 21534 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.597726 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:29.589309 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:29.519028 21323 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.602201 13944 raft_consensus.cc:2393] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:07:29.609692 14024 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.023s	sys 0.021s
W20260709 13:07:29.611423 14024 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.023s	sys 0.021s
I20260709 13:07:29.616017 13945 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:29.626703 14016 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:29.632010 14024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2998) took 1346 ms. Trace:
I20260709 13:07:29.510449 21544 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:29.581281 21548 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.577847 21496 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.574971 21556 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.592099 21560 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.657408 13948 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 14.
W20260709 13:07:29.660830 21597 log.cc:927] Time spent T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:07:29.661512 21323 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:29.664093 21496 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:29.598773 21464 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:29.601361 21321 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.664835 14015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49628 (request call id 2999) took 1347 ms. Trace:
I20260709 13:07:29.672972 14015 rpcz_store.cc:276] 0709 13:07:28.317939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.752994 (+435055us) service_pool.cc:224] Handling call
0709 13:07:28.753368 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.764653 (+ 11285us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:29.020057 (+255404us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.406933 (+386876us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.407587 (+   654us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.407770 (+   183us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:29.408304 (+   534us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.409280 (+   976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.409384 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.664597 (+255213us) raft_consensus.cc:1685] finished
0709 13:07:29.664623 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.664767 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47865,"prepare.queue_time_us":490292,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":90,"raft.queue_time_us":43636,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":292,"thread_start_us":611198,"threads_started":2,"wal-append.queue_time_us":497240}
I20260709 13:07:29.680546 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:29.681432 14015 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 14: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:29.666045 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 87420us
I20260709 13:07:29.659384 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:29.631629 21547 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:29.643213 21546 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:29.687167 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:29.697011 13945 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:29.691041 14555 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 14: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:29.692116 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:29.701381 13941 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:29.692824 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:29.703222 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:29.703672 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:29.707084 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:29.710539 13838 leader_election.cc:400] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19.
I20260709 13:07:29.710942 13838 leader_election.cc:403] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:07:29.708184 21463 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:29.714589 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 44699.3us
I20260709 13:07:29.716523 13838 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:29.694980 21585 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:29.824020 13941 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
W20260709 13:07:29.722421 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 104004us
I20260709 13:07:29.827458 13945 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
I20260709 13:07:29.714221 21392 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:07:29.829689 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 126555us
W20260709 13:07:29.833781 13915 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 154 similar messages]
W20260709 13:07:29.836654 13915 leader_election.cc:336] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:29.841899 13915 leader_election.cc:336] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:29.842577 13915 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:29.724911 21475 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:29.851255 13915 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:29.838233 21346 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:07:29.824213 21547 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:07:29.880561 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 167445us
I20260709 13:07:29.887380 21547 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 20 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
W20260709 13:07:29.888713 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 52717.1us
I20260709 13:07:29.713661 21463 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:29.854686 21533 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:29.891175 13839 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:29.630906 21582 consensus_queue.cc:237] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.851917 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:29.892076 21068 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:29.903066 21598 log.cc:927] Time spent T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
W20260709 13:07:29.897586 21604 log.cc:927] Time spent T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 13:07:29.984927 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 93914.2us
W20260709 13:07:29.988193 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 77837.7us
W20260709 13:07:29.994387 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 143155us
I20260709 13:07:29.995416 21577 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:29.996243 21068 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:29.832577 21585 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:29.632490 14024 rpcz_store.cc:276] 0709 13:07:28.317449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.750945 (+433496us) service_pool.cc:224] Handling call
0709 13:07:28.751331 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.772356 (+ 21025us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.772419 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.777595 (+  5176us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.021910 (+244315us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:29.611764 (+589854us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.631692 (+ 19928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.631775 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.631820 (+    45us) raft_consensus.cc:1685] finished
0709 13:07:29.631834 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.631951 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62826,"prepare.queue_time_us":310137,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":39305,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":17898,"thread_start_us":758221,"threads_started":2,"wal-append.queue_time_us":409823}
I20260709 13:07:30.001232 21464 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:30.003966 21463 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:30.008527 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:29.889269 21547 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:30.014488 21519 raft_consensus.cc:493] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.026403 21602 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:30.026865 21519 raft_consensus.cc:515] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:30.027078 21602 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:30.027995 21594 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.577s	user 0.002s	sys 0.002s
I20260709 13:07:30.029271 21519 leader_election.cc:290] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:30.029307 21602 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:30.029592 14023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 918) took 3127 ms. Trace:
I20260709 13:07:30.030695 14023 rpcz_store.cc:276] 0709 13:07:26.902278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.621136 (+1718858us) service_pool.cc:224] Handling call
0709 13:07:28.654514 (+ 33378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.976976 (+322462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.977028 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.991298 (+ 14270us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.491647 (+500349us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:07:29.513634 (+ 21987us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.513891 (+   257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.513985 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:30.029370 (+515385us) raft_consensus.cc:1685] finished
0709 13:07:30.029395 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.029533 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23808,"prepare.run_cpu_time_us":4566,"prepare.run_wall_time_us":28433,"raft.queue_time_us":29582,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":2927,"thread_start_us":776825,"threads_started":2,"wal-append.queue_time_us":672124}
I20260709 13:07:30.032755 21548 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:30.036443 21612 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.190s	user 0.000s	sys 0.010s
W20260709 13:07:30.013651 21603 log.cc:927] Time spent T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.344s	user 0.002s	sys 0.006s
I20260709 13:07:30.016355 14555 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:30.057261 13947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 3402) took 1462 ms. Trace:
W20260709 13:07:30.012589 13915 leader_election.cc:336] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.066941 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:30.067845 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55291.5us
W20260709 13:07:30.068518 13915 leader_election.cc:336] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.068877 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:30.056178 21496 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.071429 21496 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:30.062726 13947 rpcz_store.cc:276] 0709 13:07:28.594835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.594998 (+   163us) service_pool.cc:224] Handling call
0709 13:07:28.595456 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.925083 (+329627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.925122 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.925625 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.189239 (+263614us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:29.189284 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.189371 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:30.027453 (+838082us) spinlock_profiling.cc:243] Waited 54.5 ms on lock 0x7b4400022b4c. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c026cc85 00007fb2c026d6a2 00007fb2c2fdf1db 00007fb2c2fdaaaf 00007fb2c2fce86e 00007fb2c2fcdbb6 00007fb2c2fcd7d8 00007fb2c2fd7221 00007fb2c2fd71d9 00007fb2c2fd7169
0709 13:07:30.054687 (+ 27234us) raft_consensus.cc:1685] finished
0709 13:07:30.054713 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.057192 (+  2479us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92197,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":125428096,"thread_start_us":546007,"threads_started":2,"wal-append.queue_time_us":490144}
I20260709 13:07:30.076884 21615 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:07:30.079921 13915 leader_election.cc:336] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.080459 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:30.082414 13950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35884 (request call id 3403) took 1460 ms. Trace:
I20260709 13:07:30.082824 13950 rpcz_store.cc:276] 0709 13:07:28.622223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.622459 (+   236us) service_pool.cc:224] Handling call
0709 13:07:28.622871 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.986365 (+363494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.986418 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.986988 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.376249 (+389261us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:29.376318 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.376429 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:30.082191 (+705762us) raft_consensus.cc:1685] finished
0709 13:07:30.082218 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.082357 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":65046,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":704242,"threads_started":2,"wal-append.queue_time_us":524577}
I20260709 13:07:30.099825 13915 leader_election.cc:400] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 13 election: Vote denied by peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599) with higher term. Message: Invalid argument: T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 for earlier term 13. Current term is 14.
I20260709 13:07:30.100967 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:30.115073 14525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.096545 21602 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:30.089633 21615 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:30.122509 13990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.100569 21300 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.138793 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:30.118433 21496 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:30.161738 21613 log.cc:927] Time spent T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:07:30.159704 21518 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:30.144302 21500 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:07:30.141669 21300 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:30.188020 21579 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.005s	sys 0.015s
W20260709 13:07:30.191445 21491 consensus_meta.cc:287] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Time spent flushing consensus metadata: real 0.668s	user 0.012s	sys 0.002s
W20260709 13:07:30.196986 21511 consensus_meta.cc:287] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: Time spent flushing consensus metadata: real 0.859s	user 0.017s	sys 0.001s
W20260709 13:07:30.195726 21473 consensus_meta.cc:287] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Time spent flushing consensus metadata: real 0.949s	user 0.011s	sys 0.008s
I20260709 13:07:30.193758 21534 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:30.195510 21556 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:30.190521 21518 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:30.258175 21626 log.cc:927] Time spent T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:07:30.263159 21475 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for earlier term 18. Current term is 19.
W20260709 13:07:30.243556 21579 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.005s	sys 0.015s
I20260709 13:07:30.255188 21488 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:30.248909 21372 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:30.256014 21511 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:30.256357 21473 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:30.265264 21622 log.cc:927] Time spent T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:30.286157 21372 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:30.273943 21300 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:30.291419 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:30.294426 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:30.295792 21475 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:30.297861 21511 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:30.298981 21585 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:30.287432 21488 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:30.301692 21488 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:30.303691 21372 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:30.306180 21473 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:30.306638 21560 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:30.307614 21518 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:30.310515 21500 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:30.311153 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:30.318099 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:30.315661 21068 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:30.322515 21068 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:30.320117 21475 raft_consensus.cc:3060] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:30.323335 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:30.324890 21560 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:30.325256 21560 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.325605 21560 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:30.325613 21068 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:30.323876 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 192808us
I20260709 13:07:30.324093 21500 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:30.330945 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:30.331831 21500 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:30.340023 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:30.342895 13915 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.346297 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:30.347164 21602 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:30.347671 21068 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:30.349834 21321 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:30.350672 13915 leader_election.cc:336] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:30.351733 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 166881us
W20260709 13:07:30.352336 13915 leader_election.cc:336] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.353446 13915 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:30.354206 21602 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:30.357636 13915 consensus_peers.cc:597] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.359923 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:30.361836 14524 leader_election.cc:336] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.368294 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:30.368981 13948 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:30.370105 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.362082 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:30.371239 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.373047 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.363095 14524 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:30.375469 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.377633 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:30.378166 13945 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:30.378576 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:30.365641 13989 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.394104 13950 raft_consensus.cc:2393] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:30.394397 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:30.395455 13945 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 23.
W20260709 13:07:30.395730 13989 leader_election.cc:336] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.396109 13948 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:30.396289 13989 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:30.387250 13915 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:30.399055 13915 leader_election.cc:336] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:30.399243 13989 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.362342 21583 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:30.390329 21596 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:30.401882 21596 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:30.404199 21596 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:30.408697 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:30.399832 13915 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.459771 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:30.423710 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.459077 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.463311 13947 raft_consensus.cc:2393] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:30.463563 14021 raft_consensus.cc:2410] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:07:30.414441 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.416910 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.415664 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:30.465210 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:30.461066 13989 leader_election.cc:336] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.466856 14016 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:07:30.466784 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:30.466432 14015 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 in current term 21: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:30.430538 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.468621 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:30.469527 14024 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:30.466967 13838 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:30.470016 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:30.470690 14021 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:30.417630 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:30.471567 13950 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 21: Already voted for candidate e1649107c781428692458f10597c5742 in this term.
I20260709 13:07:30.417284 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:30.471604 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:30.472040 14018 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:07:30.472288 14021 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
W20260709 13:07:30.472651 13989 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:30.473402 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:30.468919 21547 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:30.473888 13989 leader_election.cc:336] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.474017 14015 raft_consensus.cc:3060] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:30.474149 21547 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:30.470717 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:30.475195 14024 raft_consensus.cc:2468] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:30.476120 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:30.477941 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:30.484829 21323 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:30.485476 21323 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:30.418618 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:30.488363 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 88557.4us
I20260709 13:07:30.398389 21372 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:30.419261 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.399995 21595 raft_consensus.cc:2749] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:30.414592 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:30.493089 21506 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:30.474406 13989 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:30.399155 21475 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:30.504477 12725 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 13900)
I20260709 13:07:30.399742 21488 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:30.507373 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 86668.6us
I20260709 13:07:30.513926 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:30.485791 14015 raft_consensus.cc:2468] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:30.508841 14020 raft_consensus.cc:1275] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:07:30.515605 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 48724.3us
W20260709 13:07:30.517552 13989 leader_election.cc:336] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.460454 21583 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:30.518468 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:30.524456 14022 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:07:30.520166 21562 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:30.518401 13838 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:30.522699 13989 leader_election.cc:336] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:30.526764 13989 leader_election.cc:336] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.527204 13989 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:30.528638 14016 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
W20260709 13:07:30.523586 14524 leader_election.cc:336] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.531356 14522 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:30.530262 21583 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:30.532393 14018 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 19.
W20260709 13:07:30.537225 14524 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:30.525305 21475 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:07:30.539534 13989 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.540042 13989 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:30.542155 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:30.506533 21560 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:30.525552 21544 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:30.541484 21583 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:30.537187 21488 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:30.578900 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:30.583940 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343e333726b74ed98b7e80869ce28ad1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:30.544700 21534 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:30.549425 21496 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:30.599840 21506 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:30.548386 21637 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:30.613168 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:30.622651 21516 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:30.623011 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:30.623261 21516 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:30.623811 21516 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:30.624357 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:30.624773 21601 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:30.625183 21601 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:30.625547 21601 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:30.631321 13915 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:30.632637 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.578991 21544 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 20 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:30.553658 21475 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:30.678560 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 80517.7us
I20260709 13:07:30.625561 21615 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:30.656807 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 117318us
I20260709 13:07:30.629122 21583 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:30.693710 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 62478.7us
I20260709 13:07:30.607862 21506 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:30.674121 21488 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:07:30.699591 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 92904.7us
I20260709 13:07:30.709952 13987 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:30.713490 13838 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:30.689774 21518 raft_consensus.cc:2804] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:30.737067 21547 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:30.739406 21634 log.cc:927] Time spent T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:07:30.743986 21516 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:30.694396 21643 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:30.751403 21323 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:30.725121 21644 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:30.724959 21165 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:30.752682 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:30.698354 21615 consensus_queue.cc:1048] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:07:30.765759 13838 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:30.766832 21472 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:30.767760 13838 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:30.769062 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 104211us
W20260709 13:07:30.769452 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44780.8us
I20260709 13:07:30.768894 13838 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
W20260709 13:07:30.754606 21639 log.cc:927] Time spent T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:07:30.788650 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.790086 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.793273 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:30.794600 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:30.813700 13941 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:07:30.814608 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 44576.9us
I20260709 13:07:30.817354 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:30.817775 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:30.729108 21518 raft_consensus.cc:697] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:30.780858 21472 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:30.815022 21068 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:30.876827 14022 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 23.
I20260709 13:07:30.728781 21544 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:30.849058 21413 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.871397 21472 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 23 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:30.884239 21496 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:30.896200 21582 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:30.919593 21413 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:30.927896 21518 consensus_queue.cc:237] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:30.937117 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 121200us
I20260709 13:07:30.938290 13838 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:30.897297 21068 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 21 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:30.882158 21601 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:30.950276 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 263310us
I20260709 13:07:30.967031 21068 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:30.973235 14555 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:07:30.974875 13838 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:07:30.951537 13989 leader_election.cc:336] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.965583 21601 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:30.970898 21506 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:30.982519 13989 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:30.865597 21488 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:30.982620 21656 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:30.983700 21519 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:31.002925 21650 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:30.944979 21582 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:31.019907 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 68402.5us
W20260709 13:07:31.021353 13989 leader_election.cc:336] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.996699 21472 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:31.029410 13989 leader_election.cc:336] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:31.029906 13989 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:31.031174 13989 consensus_peers.cc:597] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:31.031912 13989 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:31.033836 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:31.019091 21583 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:31.037582 13989 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:31.038218 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:31.039587 13989 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:31.035211 21648 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:31.040102 13989 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:31.036794 21645 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:31.009531 21496 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:31.042689 13989 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:31.039906 21641 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:31.047603 21583 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:31.054870 21582 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:31.114677 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 505407us
I20260709 13:07:31.127009 13771 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 9 to 10, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 10 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:31.133309 13771 catalog_manager.cc:5697] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 10 to 13, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 13 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:31.142040 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:31.170534 14024 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:31.144940 21533 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:31.147454 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:31.154011 21300 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.176311 21300 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:31.155117 21372 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:31.178635 21372 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:31.157909 21582 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:31.182813 21582 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:31.183437 21413 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:31.186983 21533 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:31.190889 21533 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:31.193857 21300 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:31.195420 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:31.196547 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:31.198273 21372 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:31.199661 21516 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:31.199747 21582 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:31.208595 21652 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.510s	user 0.002s	sys 0.001s
I20260709 13:07:31.205420 14021 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 24.
I20260709 13:07:31.206290 14561 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 24.
I20260709 13:07:31.210888 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:31.200603 21165 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:07:31.221463 13767 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54072 (request call id 142) took 33454 ms (client timeout 15000 ms). Trace:
I20260709 13:07:31.224942 21475 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:31.229995 21516 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:31.230751 21533 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.243839 13988 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:31.243708 21533 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:31.245136 21534 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:31.245590 21534 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.245924 21534 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:31.245833 21560 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:31.246433 21516 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:31.239244 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:31.229205 13771 catalog_manager.cc:5697] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 13 to 15, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 15 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:31.253779 21533 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:31.259361 21516 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:31.286248 21560 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.287484 21560 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:31.288019 13838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 298800us
I20260709 13:07:31.290452 14024 raft_consensus.cc:2410] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:07:31.291486 21500 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:31.298074 13950 raft_consensus.cc:1240] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 20. Current term is 21. Ops: []
W20260709 13:07:31.287501 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 48499.2us
I20260709 13:07:31.303104 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:31.310150 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:31.310729 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:31.311193 14558 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 14.
I20260709 13:07:31.313731 13942 raft_consensus.cc:1275] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:31.336730 13943 raft_consensus.cc:1240] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:31.330849 21500 raft_consensus.cc:3060] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:31.240949 13767 rpcz_store.cc:269] 0709 13:06:57.818225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.892896 (+ 74671us) service_pool.cc:224] Handling call
0709 13:06:59.165425 (+1272529us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b5400002490. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c01cedb1 00007fb2c030eb71 00007fb2c030f0b2 00007fb2c9288a89 00007fb2c9220da6 00007fb2c91f2088 00007fb2c92f037e 00007fb2c77b12a4 00007fb2c77b1232 00007fb2c77b1161 00007fb2c77b10dc
0709 13:07:31.221093 (+32055668us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:27.053190 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:27.053823 (+   633us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:27.053863 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:07:27.053880 (+    17us) tablet.cc:662] Decoding operations
0709 13:07:27.054620 (+   740us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:27.075485 (+ 20865us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:27.086880 (+ 11395us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:07:27.159757 (+ 72877us) tablet.cc:701] Row locks acquired
0709 13:07:27.171662 (+ 11905us) write_op.cc:260] PREPARE: finished
0709 13:07:27.171884 (+   222us) write_op.cc:270] Start()
0709 13:07:27.172058 (+   174us) write_op.cc:276] Timestamp: P: 1783602447171824 usec, L: 0
0709 13:07:27.172091 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:07:27.173032 (+   941us) log.cc:844] Serialized 1686 byte log entry
0709 13:07:28.504086 (+1331054us) op_driver.cc:464] REPLICATION: finished
0709 13:07:28.929730 (+425644us) write_op.cc:301] APPLY: starting
0709 13:07:28.929834 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:28.933269 (+  3435us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:28.933413 (+   144us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:29.326740 (+393327us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:29.327001 (+   261us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:07:29.327066 (+    65us) write_op.cc:312] APPLY: finished
0709 13:07:29.327354 (+   288us) log.cc:844] Serialized 49 byte log entry
0709 13:07:29.327562 (+   208us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:29.327680 (+   118us) write_op.cc:454] Released schema lock
0709 13:07:29.327733 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:29.328288 (+   555us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:29.328322 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":150244,"reported_tablets":24,"spinlock_wait_cycles":78337920,"child_traces":[["op",{"apply.queue_time_us":413168,"apply.run_cpu_time_us":11934,"apply.run_wall_time_us":398527,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20823,"num_ops":4,"prepare.queue_time_us":303842,"prepare.run_cpu_time_us":27913,"prepare.run_wall_time_us":275471,"raft.queue_time_us":1010793,"raft.run_cpu_time_us":32709,"raft.run_wall_time_us":509153,"replication_time_us":1291576,"thread_start_us":1701814,"threads_started":4,"wal-append.queue_time_us":265061,"wal-append.run_cpu_time_us":26002,"wal-append.run_wall_time_us":2896549}]]}
I20260709 13:07:31.371435 21545 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:07:31.374203 13914 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:31.375007 21560 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:31.375103 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 47354.2us
I20260709 13:07:31.375684 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7
W20260709 13:07:31.385671 14524 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:31.393136 21475 raft_consensus.cc:2804] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:31.393707 21475 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.394076 21475 raft_consensus.cc:3060] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:31.388437 21577 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:31.400389 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:31.401485 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:31.397099 21577 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.403458 21577 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:31.415359 21677 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.054s	user 0.000s	sys 0.002s
I20260709 13:07:31.420390 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:31.421528 21392 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:31.422021 21392 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:31.422441 21392 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:31.426213 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:31.429258 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 88893.6us
I20260709 13:07:31.437008 14019 raft_consensus.cc:1240] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:31.439960 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:31.448988 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:31.454460 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:31.455421 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:31.457607 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:31.443456 21534 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:31.465451 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:31.469256 14020 raft_consensus.cc:1275] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:31.476155 21506 consensus_queue.cc:1048] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:31.490531 21500 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:31.495819 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 53354.1us
I20260709 13:07:31.496886 13840 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:31.497392 21392 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:31.500380 14552 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:31.498636 21500 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:31.504812 21500 consensus_queue.cc:1059] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:07:31.495349 21577 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:31.508239 12725 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 13900)
I20260709 13:07:31.508687 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:07:31.509968 14524 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:31.510238 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:31.511294 14552 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:31.501430 21392 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:31.508143 21534 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 25 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:31.530265 21577 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:31.552522 14552 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 15.
W20260709 13:07:31.553282 13759 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 85969.5us
I20260709 13:07:31.556838 13914 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:31.562667 21577 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:07:31.587580 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 57900.9us
I20260709 13:07:31.589704 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:31.595260 21500 consensus_queue.cc:1059] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:31.601092 21560 raft_consensus.cc:3055] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:31.601583 21560 raft_consensus.cc:740] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:31.605734 21560 consensus_queue.cc:260] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:31.607218 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 58963.8us
I20260709 13:07:31.607462 21560 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:31.608695 21511 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:31.588672 21577 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:31.615279 21577 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:31.617883 21472 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:31.618489 13759 catalog_manager.cc:4755] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6b098cc2d51e471cb2c9c1b17eebcb57: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:07:31.618660 21472 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:31.606117 21496 consensus_queue.cc:1059] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:07:31.620888 21623 raft_consensus.cc:3055] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:31.621380 21623 raft_consensus.cc:740] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:31.621423 21472 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:31.624910 13949 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:31.626128 21602 consensus_queue.cc:1059] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:31.635946 21602 raft_consensus.cc:3055] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:31.636363 21602 raft_consensus.cc:740] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:31.641258 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:31.637351 21602 consensus_queue.cc:260] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:31.645340 21683 log.cc:927] Time spent T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:07:31.638238 21516 consensus_queue.cc:1048] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:07:31.647226 21496 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:31.649367 21475 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:31.652953 21623 consensus_queue.cc:260] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:31.654675 21623 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:31.656445 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:31.657583 14024 raft_consensus.cc:2393] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 25: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:31.658591 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:31.659209 13947 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:07:31.661031 21516 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.661742 21516 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:31.671388 13915 consensus_peers.cc:597] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:31.676916 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:31.678781 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:31.682256 21413 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:07:31.688210 13915 consensus_peers.cc:597] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:31.696525 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 64379.9us
I20260709 13:07:31.697278 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:31.704675 13950 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:07:31.708323 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:31.710779 21681 log.cc:927] Time spent T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:07:31.713124 13840 leader_election.cc:304] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:31.696532 21674 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:07:31.723555 21500 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:31.729717 14552 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:31.740594 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:31.743842 21496 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:07:31.747994 13840 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:31.749256 21548 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:31.729756 21602 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:31.758805 21475 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:31.759682 21516 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:31.732236 21547 raft_consensus.cc:2749] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:31.769042 21494 consensus_queue.cc:1059] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:31.774984 21488 raft_consensus.cc:3055] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:31.800019 21500 raft_consensus.cc:3055] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:07:31.804543 14524 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:31.802141 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:31.810392 21500 raft_consensus.cc:740] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:31.811676 21488 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:31.826311 21547 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:31.843727 13949 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:07:31.819314 21500 consensus_queue.cc:260] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:31.859251 21488 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:31.861507 21488 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:31.875375 13840 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:31.887472 21472 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:31.887818 21500 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:31.953390 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:31.954545 14021 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:31.957088 21496 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.958014 21496 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:31.997726 13915 consensus_peers.cc:597] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:32.005082 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 69645.1us
I20260709 13:07:31.978243 21496 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:32.013005 13915 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.017185 13947 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:32.016570 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:31.992635 21472 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.015988 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:31.992871 21475 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:07:32.035574 21472 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:32.040500 13913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:32.062558 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 54206us
W20260709 13:07:32.065960 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 213307us
I20260709 13:07:32.071061 21516 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:32.072824 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:32.076315 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:32.064718 13915 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.096261 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:32.096508 14023 raft_consensus.cc:1240] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:32.115036 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:32.126001 14020 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:07:32.131655 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 66986.1us
I20260709 13:07:32.134083 14017 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:07:32.132647 13915 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.135061 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:32.136579 13915 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.137156 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:32.139671 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:32.139912 13915 consensus_peers.cc:597] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.131378 21602 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:32.124492 21560 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.146401 14021 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:07:32.140585 21533 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:32.143504 21577 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:32.149911 21545 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.165717 21511 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:32.175658 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:32.182225 21472 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:32.189838 21300 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:32.190747 21300 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:32.193621 21300 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:32.202314 21511 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:32.162098 21560 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:32.217973 13839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 187979us
W20260709 13:07:32.222829 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 82065.1us
W20260709 13:07:32.224112 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 120532us
I20260709 13:07:32.228140 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:32.215674 21511 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:32.212334 21545 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:32.239614 21545 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:32.240047 21560 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:32.254817 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:32.258204 21690 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:32.258322 21511 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.258842 21690 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:32.259318 21690 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:32.259258 21511 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:32.261652 13989 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.262537 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:07:32.282003 21694 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.246s	user 0.002s	sys 0.000s
I20260709 13:07:32.299857 14020 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:32.299803 21300 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:32.318235 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:32.319432 14016 raft_consensus.cc:2410] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1649107c781428692458f10597c5742 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:07:32.302515 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:32.358889 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:32.361407 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:32.367767 21690 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:32.370710 13944 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:32.373457 21674 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:32.390151 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:32.395820 13840 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:32.400148 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 174805us
I20260709 13:07:32.392030 21511 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:32.401614 21548 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:07:32.417367 21322 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:07:32.422034 13989 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.436154 21690 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:32.421643 21674 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:32.443840 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 177941us
I20260709 13:07:32.391395 21472 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:07:32.450736 13989 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:32.454293 14561 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:32.393199 21547 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
W20260709 13:07:32.491156 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 95646.9us
W20260709 13:07:32.503126 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:32.504927 21716 log.cc:927] Time spent T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:07:32.509428 12725 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 13900)
I20260709 13:07:32.492808 21488 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:32.524097 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 102124us
W20260709 13:07:32.524887 13989 leader_election.cc:336] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.525341 13989 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:32.525890 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:32.526325 21534 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:07:32.527060 13989 leader_election.cc:336] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.527504 13989 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:32.530516 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:32.532513 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:32.586530 13840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 94452.2us
W20260709 13:07:32.591748 13771 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54072 (request call id 141) took 55213 ms (client timeout 15000 ms). Trace:
W20260709 13:07:32.599568 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 224287us
W20260709 13:07:32.612636 13987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.598696 21506 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:32.630599 14020 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:07:32.631143 14020 pending_rounds.cc:85] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting all ops after (but not including) 11
I20260709 13:07:32.631379 14020 pending_rounds.cc:107] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:07:32.631691 14020 raft_consensus.cc:2889] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:32.632447 13769 catalog_manager.cc:5697] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 9 to 19. New cstate: current_term: 19 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:07:32.592253 13771 rpcz_store.cc:269] 0709 13:06:37.367813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.367977 (+   164us) service_pool.cc:224] Handling call
0709 13:07:32.581320 (+55213343us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:31.474156 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:31.474954 (+   798us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:31.475011 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:07:31.475031 (+    20us) tablet.cc:662] Decoding operations
0709 13:07:31.475693 (+   662us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:31.475797 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:31.475837 (+    40us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:07:31.476370 (+   533us) tablet.cc:701] Row locks acquired
0709 13:07:31.476420 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:07:31.476628 (+   208us) write_op.cc:270] Start()
0709 13:07:31.476786 (+   158us) write_op.cc:276] Timestamp: P: 1783602451476555 usec, L: 0
0709 13:07:31.476827 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:07:31.477536 (+   709us) log.cc:844] Serialized 1302 byte log entry
0709 13:07:31.723842 (+246306us) op_driver.cc:464] REPLICATION: finished
0709 13:07:31.999252 (+275410us) write_op.cc:301] APPLY: starting
0709 13:07:31.999379 (+   127us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:32.124500 (+125121us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:32.138799 (+ 14299us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:32.183584 (+ 44785us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:32.183930 (+   346us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 13:07:32.184010 (+    80us) write_op.cc:312] APPLY: finished
0709 13:07:32.184323 (+   313us) log.cc:844] Serialized 41 byte log entry
0709 13:07:32.185266 (+   943us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:32.185366 (+   100us) write_op.cc:454] Released schema lock
0709 13:07:32.185431 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:32.186028 (+   597us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:32.186064 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1469140,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":264412,"apply.run_cpu_time_us":9077,"apply.run_wall_time_us":191855,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16649,"num_ops":3,"prepare.queue_time_us":130565,"prepare.run_cpu_time_us":54458,"prepare.run_wall_time_us":118056,"raft.queue_time_us":72475,"raft.run_cpu_time_us":19127,"raft.run_wall_time_us":130082,"replication_time_us":246792,"thread_start_us":295874,"threads_started":4,"wal-append.queue_time_us":113704}]]}
I20260709 13:07:32.642885 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:32.643615 14020 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:32.646152 21695 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:07:32.646564 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:32.652623 13840 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:32.652895 21496 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:32.654368 21717 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:32.655050 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 128026us
I20260709 13:07:32.655824 21496 consensus_queue.cc:1059] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:32.663543 13769 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 13 to 16, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 16 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.654012 13840 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:32.655871 13989 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.668591 13989 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:32.669615 21722 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:32.671202 13989 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.671739 13989 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:32.672499 21722 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:32.672595 21472 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:32.674017 13989 leader_election.cc:336] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.674962 21674 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:32.681901 13989 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:32.667374 13769 catalog_manager.cc:5697] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 12 to 15, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 15 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.695339 21674 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:32.699218 21650 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.699928 21650 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:32.702651 21650 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:32.705605 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:32.707015 21674 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:32.709041 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:32.710759 14020 raft_consensus.cc:2468] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:32.711925 21722 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:32.713434 13989 leader_election.cc:336] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.716817 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:32.718196 13989 leader_election.cc:336] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.718650 13989 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:32.718797 21719 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:32.699836 13769 catalog_manager.cc:5697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 15 to 16, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 16 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.723567 21548 raft_consensus.cc:3055] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:32.724020 21548 raft_consensus.cc:740] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:32.725836 21548 consensus_queue.cc:260] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:32.728428 21548 raft_consensus.cc:3060] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:32.721196 13769 catalog_manager.cc:5697] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 11 to 15, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 15 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:32.752617 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:32.747390 21494 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:32.757411 13840 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:32.754163 21547 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:32.764385 21547 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:32.766166 21719 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:32.766784 21719 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:32.769042 21719 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:32.772003 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 53866.1us
W20260709 13:07:32.773275 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:32.766784 21547 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:32.781867 13839 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:32.790675 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:32.791386 14024 raft_consensus.cc:1240] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Rejecting Update request from peer e1649107c781428692458f10597c5742 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:07:32.791791 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:32.798008 13989 leader_election.cc:336] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.798580 13989 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:32.795150 13941 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 22.
I20260709 13:07:32.801842 21473 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:32.802078 13769 catalog_manager.cc:5697] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 12 to 13, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 13 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.804227 13769 catalog_manager.cc:5697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:32.806962 14021 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:07:32.800079 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 47143.2us
I20260709 13:07:32.795750 21674 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:32.818127 21494 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:32.823295 21533 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:32.828474 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:32.822765 21650 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:32.833161 21650 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:32.836858 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:32.837592 13915 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:32.839224 21650 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:32.840200 13769 catalog_manager.cc:5697] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.843369 21714 raft_consensus.cc:2804] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:32.843618 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:32.846344 21714 raft_consensus.cc:697] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:32.846565 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:32.846565 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:32.847350 21714 consensus_queue.cc:237] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:32.848570 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 65150.7us
W20260709 13:07:32.849304 13989 leader_election.cc:336] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.849758 13989 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:32.850713 21488 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:07:32.851251 13989 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:32.855926 13915 leader_election.cc:336] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.860867 13838 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:32.869886 13989 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:32.886487 13915 consensus_peers.cc:597] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:32.887458 13915 consensus_peers.cc:597] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:32.874886 21713 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:32.889858 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:32.906745 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:32.902851 13769 catalog_manager.cc:5697] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 7 to 15. New cstate: current_term: 15 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.907603 21722 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:32.922251 21472 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:32.921628 13769 catalog_manager.cc:5697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.922519 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:32.933974 13840 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:32.936031 13840 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:32.944602 21472 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:32.932631 21488 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:32.945708 21488 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.946108 21488 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:32.959054 21547 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:32.967957 21719 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:32.968740 21719 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:32.977556 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 126349us
W20260709 13:07:32.979861 13989 leader_election.cc:336] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.980449 13989 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:32.981467 21534 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:32.992003 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:32.992908 13947 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 21: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:07:33.012020 13989 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.012638 13989 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:33.014182 21719 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:33.015722 13989 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.016438 13989 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:33.029448 13915 leader_election.cc:336] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:33.031021 13915 consensus_peers.cc:597] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:33.023718 21534 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:33.037487 21722 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:33.089128 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 74486.2us
W20260709 13:07:33.110448 13915 consensus_peers.cc:597] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:33.103418 21488 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:33.125437 21488 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:33.130549 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:33.135321 13947 raft_consensus.cc:3060] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:33.165819 13989 leader_election.cc:336] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.170432 13989 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:33.171344 13947 raft_consensus.cc:2468] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 23.
I20260709 13:07:33.178417 13987 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:33.179663 21722 raft_consensus.cc:2804] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:33.180266 21722 raft_consensus.cc:697] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:33.181272 21722 consensus_queue.cc:237] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:33.195708 21534 raft_consensus.cc:2749] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:33.204907 13989 leader_election.cc:336] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.205796 13989 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:33.206902 21534 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:33.218209 21473 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.219005 21473 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:33.221840 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:33.223848 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:33.224856 21473 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:33.225548 13839 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:33.228816 21718 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:33.235292 13989 consensus_peers.cc:597] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.236204 13915 consensus_peers.cc:597] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.237432 13989 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.246362 13989 leader_election.cc:336] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.246979 13989 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:33.249341 21534 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:33.255049 13989 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:33.256057 13989 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:33.263648 13989 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.264109 13989 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
W20260709 13:07:33.266525 13989 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.267853 21534 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:07:33.281298 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:33.282320 21534 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:33.288067 21718 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:33.286585 13989 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.291383 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:33.290515 21718 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:33.292282 21488 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:33.293929 21718 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:33.296465 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:33.294035 21475 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.305562 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:33.304230 21475 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:33.313609 13840 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:33.316843 21475 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:33.317965 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:33.315012 21473 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:33.320322 14552 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 25.
I20260709 13:07:33.321782 13839 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:33.322938 21548 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:33.323391 21548 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.323729 21548 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:07:33.333050 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 46497.1us
W20260709 13:07:33.333952 13989 leader_election.cc:336] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:33.334910 13989 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.335268 13989 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:33.339927 13989 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.340488 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:33.341288 14015 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 25.
I20260709 13:07:33.340111 21722 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:33.368137 21548 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:33.343564 13989 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:33.378756 21488 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:33.379300 13989 leader_election.cc:336] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.387810 21488 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:33.388054 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:33.388763 14019 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:33.395692 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:33.396412 14552 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:33.408862 14016 raft_consensus.cc:1275] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:07:33.409749 14552 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 26.
I20260709 13:07:33.412093 13839 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:33.412158 21516 consensus_queue.cc:1048] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:33.413038 21475 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:33.413596 21475 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 26 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:33.414503 14019 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 26.
I20260709 13:07:33.406580 21548 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:33.417058 21475 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:33.423465 21548 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.424889 21548 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:33.440644 21472 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.450160 21472 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:33.453116 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:33.453507 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:33.454027 13941 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:33.454228 14018 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:33.455988 13838 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:33.464267 21547 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:33.464716 21547 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.465037 21547 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:33.465658 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:33.469331 21718 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.469875 21718 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:33.471637 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:33.472214 21718 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:33.473965 21547 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:33.475922 13915 consensus_peers.cc:597] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:33.474172 21548 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:33.477439 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:33.478122 13941 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 21.
I20260709 13:07:33.478227 21547 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:33.478605 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:33.478966 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:33.479082 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:33.479228 13942 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:33.479518 14018 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:33.479805 14015 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 20.
I20260709 13:07:33.481106 13839 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:33.454278 21472 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:33.483242 13840 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:33.485240 21500 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:07:33.486300 21718 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:33.486676 21718 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.486999 21718 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:33.487555 13942 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
I20260709 13:07:33.494170 14018 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
I20260709 13:07:33.496060 13840 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:33.496958 21548 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:33.497452 21548 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 22 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:33.498358 21548 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:33.499514 21500 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.500278 21500 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:33.507762 14524 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:33.510082 12725 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 13900)
I20260709 13:07:33.509721 21718 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:33.518857 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:33.519167 21500 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:33.520753 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:33.523131 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:33.525705 21547 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:33.526486 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:33.527235 13942 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:33.540522 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:33.541325 14019 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:07:33.552958 13942 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
I20260709 13:07:33.555182 13838 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742; no voters: 
I20260709 13:07:33.563030 21475 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:33.563661 21475 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 22 LEADER]: Becoming Leader. State: Replica: e1649107c781428692458f10597c5742, State: Running, Role: LEADER
I20260709 13:07:33.564608 21475 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:33.541922 21718 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:33.579818 13941 raft_consensus.cc:1275] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:07:33.581743 21722 consensus_queue.cc:1048] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:07:33.614342 13772 catalog_manager.cc:5697] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 9 to 12, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 12 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:33.656674 13937 tablet_service.cc:1558] Processing DeleteTablet for tablet 02dcf286c6bd48e3a6e548def7d786cc with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 7 (current committed config index is 13)) from {username='slave'} at 127.0.0.1:36032
I20260709 13:07:33.668411 14019 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1649107c781428692458f10597c5742 in term 22.
W20260709 13:07:33.711210 13769 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57642 (request call id 276) took 50670 ms (client timeout 15000 ms). Trace:
W20260709 13:07:33.711757 13769 rpcz_store.cc:269] 0709 13:06:43.041034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.047700 (+  6666us) service_pool.cc:224] Handling call
0709 13:06:44.453734 (+1406034us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b540012acd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339d090 00007fb2c343bf3a 00007fb2c91f21cc 00007fb2c92f037e 00007fb2c77b12a4 00007fb2c77b1232 00007fb2c77b1161 00007fb2c77b10dc 00007fb2c77b0392 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:33.711124 (+49257390us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:33.080553 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:33.081267 (+   714us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:33.081313 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:07:33.081338 (+    25us) tablet.cc:662] Decoding operations
0709 13:07:33.082618 (+  1280us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:33.082694 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:33.082728 (+    34us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:07:33.083692 (+   964us) tablet.cc:701] Row locks acquired
0709 13:07:33.083717 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:07:33.083884 (+   167us) write_op.cc:270] Start()
0709 13:07:33.084018 (+   134us) write_op.cc:276] Timestamp: P: 1783602453083825 usec, L: 0
0709 13:07:33.084045 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:07:33.084812 (+   767us) log.cc:844] Serialized 4087 byte log entry
0709 13:07:33.182760 (+ 97948us) op_driver.cc:464] REPLICATION: finished
0709 13:07:33.270457 (+ 87697us) write_op.cc:301] APPLY: starting
0709 13:07:33.271164 (+   707us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:33.325534 (+ 54370us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:33.325671 (+   137us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:33.506194 (+180523us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:33.514269 (+  8075us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:07:33.514437 (+   168us) write_op.cc:312] APPLY: finished
0709 13:07:33.514792 (+   355us) log.cc:844] Serialized 97 byte log entry
0709 13:07:33.514987 (+   195us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:33.515144 (+   157us) write_op.cc:454] Released schema lock
0709 13:07:33.515196 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:33.515666 (+   470us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:33.515796 (+   130us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":520470,"reported_tablets":53,"spinlock_wait_cycles":47881984,"child_traces":[["op",{"apply.queue_time_us":85602,"apply.run_cpu_time_us":16435,"apply.run_wall_time_us":245224,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45867,"num_ops":10,"prepare.queue_time_us":138347,"prepare.run_cpu_time_us":4667,"prepare.run_wall_time_us":22143,"replication_time_us":98513,"thread_start_us":138151,"threads_started":2}]]}
I20260709 13:07:33.723618 21472 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:33.782502 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 61647.8us
I20260709 13:07:33.768859 21472 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:33.797552 21472 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:33.799978 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:33.802675 13757 catalog_manager.cc:5028] TS 3779649036364744823fc035dac892c7 (127.12.109.66:43953): tablet 02dcf286c6bd48e3a6e548def7d786cc (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:07:33.791260 21753 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:33.823537 21753 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:33.829288 13989 consensus_peers.cc:597] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:33.829614 21475 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.830264 21475 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:33.833284 21475 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:33.833864 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:33.839283 21753 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:33.842259 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:33.845323 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:33.846314 13840 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:33.848382 21753 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:33.877982 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:33.887516 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:33.888653 13839 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:33.897044 21473 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:33.934832 13839 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:33.936187 21500 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:33.940160 14552 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:07:33.940196 13937 tablet_service.cc:1558] Processing DeleteTablet for tablet 02dcf286c6bd48e3a6e548def7d786cc with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 7 (current committed config index is 13)) from {username='slave'} at 127.0.0.1:36032
W20260709 13:07:33.941263 13772 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57642 (request call id 275) took 75021 ms (client timeout 15000 ms). Trace:
I20260709 13:07:33.943418 13757 catalog_manager.cc:5028] TS 3779649036364744823fc035dac892c7 (127.12.109.66:43953): tablet 02dcf286c6bd48e3a6e548def7d786cc (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:07:33.951078 21756 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.951751 21756 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:33.956175 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:33.959888 21756 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:33.945796 21475 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.017s
W20260709 13:07:33.941668 13772 rpcz_store.cc:269] 0709 13:06:18.920273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.920496 (+   223us) service_pool.cc:224] Handling call
0709 13:06:20.087745 (+1167249us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b540012acd4. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c339d090 00007fb2c343bf3a 00007fb2c91f21cc 00007fb2c92f037e 00007fb2c77b12a4 00007fb2c77b1232 00007fb2c77b1161 00007fb2c77b10dc 00007fb2c77b0392 00007fb2c176e48c 00007fb2c176d8c6
0709 13:07:33.941168 (+73853423us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:33.785163 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:33.818633 (+ 33470us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:33.818672 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:07:33.818693 (+    21us) tablet.cc:662] Decoding operations
0709 13:07:33.819113 (+   420us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:33.819189 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:33.819224 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:07:33.819427 (+   203us) tablet.cc:701] Row locks acquired
0709 13:07:33.819446 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:07:33.819613 (+   167us) write_op.cc:270] Start()
0709 13:07:33.819746 (+   133us) write_op.cc:276] Timestamp: P: 1783602453819549 usec, L: 0
0709 13:07:33.819769 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:07:33.820518 (+   749us) log.cc:844] Serialized 541 byte log entry
0709 13:07:33.854469 (+ 33951us) op_driver.cc:464] REPLICATION: finished
0709 13:07:33.854819 (+   350us) write_op.cc:301] APPLY: starting
0709 13:07:33.854910 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:33.855804 (+   894us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:33.855898 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:33.856673 (+   775us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:33.856847 (+   174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:07:33.856894 (+    47us) write_op.cc:312] APPLY: finished
0709 13:07:33.857115 (+   221us) log.cc:844] Serialized 25 byte log entry
0709 13:07:33.857291 (+   176us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:33.857378 (+    87us) write_op.cc:454] Released schema lock
0709 13:07:33.857421 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:33.857822 (+   401us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:33.857859 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1281,"reported_tablets":53,"spinlock_wait_cycles":1965678848,"child_traces":[["op",{"apply.queue_time_us":93,"apply.run_cpu_time_us":2946,"apply.run_wall_time_us":2946,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4174,"num_ops":1,"prepare.queue_time_us":93462,"prepare.run_cpu_time_us":3175,"prepare.run_wall_time_us":46638,"replication_time_us":34391,"spinlock_wait_cycles":134528,"thread_start_us":67217,"threads_started":1}]]}
I20260709 13:07:33.965404 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:33.970664 13840 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:33.971956 21756 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:33.991971 14016 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:07:34.003346 21475 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:34.031191 13941 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:07:34.033773 21500 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:34.044237 13764 catalog_manager.cc:5697] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:34.046574 13764 catalog_manager.cc:5697] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:34.048907 13764 catalog_manager.cc:5697] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 13 to 19, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 19 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:34.054454 14015 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 26 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:07:34.056993 13764 catalog_manager.cc:5697] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:34.059638 13764 catalog_manager.cc:5697] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 15 to 17, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 17 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:34.072966 21473 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:07:34.055745 13773 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57648 (request call id 281) took 76448 ms (client timeout 15000 ms). Trace:
W20260709 13:07:34.082773 13773 rpcz_store.cc:269] 0709 13:06:17.607730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.607964 (+   234us) service_pool.cc:224] Handling call
0709 13:07:34.055494 (+76447530us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":52}
I20260709 13:07:34.112445 14016 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:07:34.114702 21754 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:34.117061 21754 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.117647 21754 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:34.103490 13764 catalog_manager.cc:5697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 13 to 14, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 14 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:34.124820 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:34.127642 21754 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:34.132179 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:34.134528 13764 catalog_manager.cc:5697] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 16 to 17, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 17 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:34.136523 13764 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 17 to 21, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 21 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:34.137823 13840 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:34.156891 21756 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:34.168834 21475 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.169641 21475 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:34.172727 21475 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:34.173408 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:34.195698 13942 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Refusing update from remote peer e1649107c781428692458f10597c5742: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:07:34.197646 21475 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:07:34.209290 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:34.144361 13764 catalog_manager.cc:5697] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 10 to 13, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 13 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:34.229226 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:34.250401 13840 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:34.234197 13764 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:34.268603 21500 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:34.269598 13764 catalog_manager.cc:5697] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:34.271246 13764 catalog_manager.cc:5697] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:34.273753 13764 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 20 to 23, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 23 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:34.281965 21547 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:34.283803 21547 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:34.285918 21547 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:34.293332 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:34.314543 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:34.334386 13915 consensus_peers.cc:597] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:34.335194 13840 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:34.338280 21473 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:34.398504 14524 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:34.408308 21756 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.408998 21756 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:34.412425 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:34.414737 21756 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:34.419871 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:34.423195 13840 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:34.424691 21473 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:07:34.472963 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:34.504563 21475 raft_consensus.cc:493] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:34.505270 21475 raft_consensus.cc:515] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } attrs { replace: true } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:34.507095 13764 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57648 (request call id 283) took 24032 ms (client timeout 15000 ms). Trace:
I20260709 13:07:34.494787 13770 catalog_manager.cc:5697] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:34.508884 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:34.510581 12725 thread.cc:527] Waited for 9000ms trying to join with heartbeat (tid 13900)
I20260709 13:07:34.512739 21475 leader_election.cc:290] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:34.515151 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:34.516877 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bba4e2cef04a418c6f660869c7e6e5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:34.518523 13839 leader_election.cc:304] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:34.520386 21473 raft_consensus.cc:2749] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:34.534202 13770 catalog_manager.cc:5697] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 reported cstate change: term changed from 16 to 19, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 19 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:34.536074 13770 catalog_manager.cc:5697] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 12 to 17, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 17 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:34.538178 13770 catalog_manager.cc:5697] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
W20260709 13:07:34.554811 13915 consensus_peers.cc:597] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:34.563953 13770 catalog_manager.cc:5697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 14 to 21, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 21 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:34.507622 13764 rpcz_store.cc:269] 0709 13:07:10.474844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.476708 (+  1864us) service_pool.cc:224] Handling call
0709 13:07:34.507000 (+24030292us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:34.298742 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:34.299291 (+   549us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:34.299326 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:07:34.299343 (+    17us) tablet.cc:662] Decoding operations
0709 13:07:34.300518 (+  1175us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:34.300575 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:34.300607 (+    32us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:07:34.301800 (+  1193us) tablet.cc:701] Row locks acquired
0709 13:07:34.301822 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:07:34.301968 (+   146us) write_op.cc:270] Start()
0709 13:07:34.302174 (+   206us) write_op.cc:276] Timestamp: P: 1783602454301925 usec, L: 0
0709 13:07:34.302200 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:07:34.309023 (+  6823us) log.cc:844] Serialized 5231 byte log entry
0709 13:07:34.315983 (+  6960us) op_driver.cc:464] REPLICATION: finished
0709 13:07:34.321184 (+  5201us) write_op.cc:301] APPLY: starting
0709 13:07:34.321275 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:34.335160 (+ 13885us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:34.335278 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:34.368000 (+ 32722us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:34.368309 (+   309us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 13:07:34.368364 (+    55us) write_op.cc:312] APPLY: finished
0709 13:07:34.368746 (+   382us) log.cc:844] Serialized 121 byte log entry
0709 13:07:34.374253 (+  5507us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:34.374441 (+   188us) write_op.cc:454] Released schema lock
0709 13:07:34.374516 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:34.374958 (+   442us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:34.374993 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1897491,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":130,"apply.run_cpu_time_us":17859,"apply.run_wall_time_us":53735,"cfile_cache_hit":52,"cfile_cache_hit_bytes":54262,"num_ops":13,"prepare.queue_time_us":138,"prepare.run_cpu_time_us":4565,"prepare.run_wall_time_us":13209,"replication_time_us":10480}]]}
I20260709 13:07:34.572413 13770 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:34.584048 13770 catalog_manager.cc:5697] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:34.588269 13770 catalog_manager.cc:5697] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 reported cstate change: term changed from 17 to 19, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 19 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:34.591352 13770 catalog_manager.cc:5697] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:34.605801 13770 catalog_manager.cc:5697] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 17 to 18, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 18 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:34.614846 13770 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 16 to 22, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 22 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:34.617274 13770 catalog_manager.cc:5697] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:34.622678 13770 catalog_manager.cc:5697] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 16 to 21, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 21 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:34.627311 13770 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 23 to 26. New cstate: current_term: 26 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:07:34.628706 13989 consensus_peers.cc:597] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:34.644114 13770 catalog_manager.cc:5697] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:34.646474 13770 catalog_manager.cc:5697] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:34.652089 13770 catalog_manager.cc:5697] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 18 to 19, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 19 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:34.654862 13770 catalog_manager.cc:5697] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 11 to 16, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 16 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
W20260709 13:07:34.663659 13915 consensus_peers.cc:597] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:34.670905 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.672101 13915 consensus_peers.cc:597] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:34.672746 13770 catalog_manager.cc:5697] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:34.718283 13770 catalog_manager.cc:5697] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:34.723287 13915 consensus_peers.cc:597] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:34.727169 13770 catalog_manager.cc:5697] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 reported cstate change: term changed from 13 to 18, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 18 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:34.728978 13770 catalog_manager.cc:5697] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 reported cstate change: term changed from 13 to 16, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 16 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:34.730475 13770 catalog_manager.cc:5697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:34.736510 13770 catalog_manager.cc:5697] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 15 to 17, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 17 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:34.738005 13770 catalog_manager.cc:5697] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
W20260709 13:07:34.885066 14524 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 203 similar messages]
W20260709 13:07:34.951942 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:34.948889 13765 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 14 to 17, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 17 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:34.971720 13765 catalog_manager.cc:5697] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:34.973495 13765 catalog_manager.cc:5697] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 15 to 22, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 22 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:34.983698 13770 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57626 (request call id 251) took 12405 ms (client timeout 15000 ms). Trace:
I20260709 13:07:34.989085 13765 catalog_manager.cc:5697] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:34.991511 13765 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 21 to 24, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 24 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:34.994112 13765 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 10 to 16, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 16 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:34.996747 13765 catalog_manager.cc:5697] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 12 to 17, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 17 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:07:34.984200 13770 rpcz_store.cc:269] 0709 13:07:22.578940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.704478 (+125538us) service_pool.cc:224] Handling call
0709 13:07:34.983614 (+12279136us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:34.780528 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:34.781185 (+   657us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:34.781226 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:07:34.781251 (+    25us) tablet.cc:662] Decoding operations
0709 13:07:34.783238 (+  1987us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:34.783308 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:34.783339 (+    31us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:07:34.784726 (+  1387us) tablet.cc:701] Row locks acquired
0709 13:07:34.784739 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:07:34.784855 (+   116us) write_op.cc:270] Start()
0709 13:07:34.784963 (+   108us) write_op.cc:276] Timestamp: P: 1783602454784817 usec, L: 0
0709 13:07:34.784978 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 13:07:34.785542 (+   564us) log.cc:844] Serialized 9810 byte log entry
0709 13:07:34.788865 (+  3323us) op_driver.cc:464] REPLICATION: finished
0709 13:07:34.789189 (+   324us) write_op.cc:301] APPLY: starting
0709 13:07:34.789280 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:34.820779 (+ 31499us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:34.820944 (+   165us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:34.858282 (+ 37338us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:34.858605 (+   323us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 13:07:34.858649 (+    44us) write_op.cc:312] APPLY: finished
0709 13:07:34.859103 (+   454us) log.cc:844] Serialized 220 byte log entry
0709 13:07:34.859961 (+   858us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:34.860208 (+   247us) write_op.cc:454] Released schema lock
0709 13:07:34.860272 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:34.870442 (+ 10170us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:34.870482 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9328,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":112,"apply.run_cpu_time_us":30092,"apply.run_wall_time_us":81171,"cfile_cache_hit":100,"cfile_cache_hit_bytes":104350,"num_ops":25,"prepare.queue_time_us":124,"prepare.run_cpu_time_us":5327,"prepare.run_wall_time_us":5347,"replication_time_us":3728,"spinlock_wait_cycles":1408}]]}
I20260709 13:07:35.019205 13765 catalog_manager.cc:5697] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 15 to 19, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 19 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:35.020933 13765 catalog_manager.cc:5697] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 14 to 19, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 19 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:35.062305 13989 consensus_peers.cc:597] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:35.066905 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:35.070140 12725 tablet_replica.cc:333] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.072095 12725 raft_consensus.cc:2243] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.074291 12725 pending_rounds.cc:70] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Trying to abort 1 pending ops.
I20260709 13:07:35.075488 21756 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.076045 21756 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:35.078354 21756 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:35.081497 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:35.082396 21473 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.082979 21473 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:35.084709 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:35.085301 21473 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:35.087128 13840 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:35.087772 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:35.087894 21756 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:35.089046 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768fb01770349dfb259ab0db91394ba" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:35.090917 13840 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:35.092037 21756 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:35.074595 12725 pending_rounds.cc:77] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742: Aborting op as it isn't in flight: id { term: 16 index: 9 } timestamp: 7305635575221952512 op_type: NO_OP noop_request { }
I20260709 13:07:35.096247 12725 raft_consensus.cc:2889] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:35.096644 12725 raft_consensus.cc:2272] T 728de4a891d847028775aa5c35ba53e1 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:35.120170 13915 consensus_peers.cc:597] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:35.122550 12725 tablet_replica.cc:333] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.130798 12725 raft_consensus.cc:2243] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.133261 12725 raft_consensus.cc:2272] T 617d39d9397c4724be821e5efacbde17 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.136492 12725 tablet_replica.cc:333] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.145006 12725 raft_consensus.cc:2243] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.150316 12725 raft_consensus.cc:2272] T 9a2c65bca5ce45d5af2ace962fa0a37f P e1649107c781428692458f10597c5742 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.154512 12725 tablet_replica.cc:333] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.161222 12725 raft_consensus.cc:2243] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.167932 12725 raft_consensus.cc:2272] T 56605211bd584b8889b54049e9f6cb4b P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.164284 13768 catalog_manager.cc:5697] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 13 to 14, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 14 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.170982 13768 catalog_manager.cc:5697] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.162663 13765 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57648 (request call id 284) took 8934 ms. Trace:
W20260709 13:07:35.183027 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:35.172175 12725 tablet_replica.cc:333] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.183215 13765 rpcz_store.cc:276] 0709 13:07:26.228705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.554371 (+325666us) service_pool.cc:224] Handling call
0709 13:07:35.162591 (+8608220us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:35.035571 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:35.036195 (+   624us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:35.036236 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:07:35.036265 (+    29us) tablet.cc:662] Decoding operations
0709 13:07:35.037282 (+  1017us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:35.037349 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:35.037381 (+    32us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:07:35.038152 (+   771us) tablet.cc:701] Row locks acquired
0709 13:07:35.038173 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:07:35.038309 (+   136us) write_op.cc:270] Start()
0709 13:07:35.038427 (+   118us) write_op.cc:276] Timestamp: P: 1783602455038265 usec, L: 0
0709 13:07:35.038453 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:07:35.040579 (+  2126us) log.cc:844] Serialized 3616 byte log entry
0709 13:07:35.043943 (+  3364us) op_driver.cc:464] REPLICATION: finished
0709 13:07:35.044433 (+   490us) write_op.cc:301] APPLY: starting
0709 13:07:35.044536 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:35.049326 (+  4790us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:35.049424 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:35.100386 (+ 50962us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:35.100626 (+   240us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 13:07:35.100671 (+    45us) write_op.cc:312] APPLY: finished
0709 13:07:35.101000 (+   329us) log.cc:844] Serialized 89 byte log entry
0709 13:07:35.101181 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:35.101333 (+   152us) write_op.cc:454] Released schema lock
0709 13:07:35.101384 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:35.120818 (+ 19434us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:35.120853 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13252,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":278,"apply.run_cpu_time_us":12975,"apply.run_wall_time_us":63217,"cfile_cache_hit":36,"cfile_cache_hit_bytes":37566,"num_ops":9,"prepare.queue_time_us":131,"prepare.run_cpu_time_us":3977,"prepare.run_wall_time_us":7421,"replication_time_us":5346}]]}
I20260709 13:07:35.182672 13768 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 15 to 16, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 16 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.211138 13768 catalog_manager.cc:5697] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 14 to 15, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 15 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.213234 13768 catalog_manager.cc:5697] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 10 to 13, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 13 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.216123 12725 raft_consensus.cc:2243] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.215896 13768 catalog_manager.cc:5697] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 17 to 19, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 19 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.223898 13768 catalog_manager.cc:5697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.227018 12725 raft_consensus.cc:2272] T 10b028ec30974d06bd9da921c268a641 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.226186 13768 catalog_manager.cc:5697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 15 to 16, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 16 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.230291 13768 catalog_manager.cc:5697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:35.234577 13915 consensus_peers.cc:597] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:35.235690 12725 tablet_replica.cc:333] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.235898 13768 catalog_manager.cc:5697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 12 to 16, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 16 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.267830 21475 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.268494 21475 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:35.271322 21475 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:35.275177 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:35.276530 12725 raft_consensus.cc:2243] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.278422 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:35.281957 13915 consensus_peers.cc:597] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:35.299690 13839 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:35.300992 21475 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:35.309731 12725 raft_consensus.cc:2272] T 343e333726b74ed98b7e80869ce28ad1 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.319908 12725 tablet_replica.cc:333] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.325440 12725 raft_consensus.cc:2243] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.327157 12725 raft_consensus.cc:2272] T 99b9907790614d49bc3adf40424fc1b6 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.342748 12725 tablet_replica.cc:333] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.343876 12725 raft_consensus.cc:2243] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.348187 12725 raft_consensus.cc:2272] T 14bba4e2cef04a418c6f660869c7e6e5 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.351764 12725 tablet_replica.cc:333] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.363166 12725 raft_consensus.cc:2243] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.364666 12725 raft_consensus.cc:2272] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.375087 12725 tablet_replica.cc:333] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.384625 12725 raft_consensus.cc:2243] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.385998 12725 raft_consensus.cc:2272] T d4033a61013848c69bbd0e79b27a8be3 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.389492 12725 tablet_replica.cc:333] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.393106 13767 catalog_manager.cc:5697] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.390762 12725 raft_consensus.cc:2243] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.395313 13767 catalog_manager.cc:5697] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 15 to 21, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 21 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:35.397708 13767 catalog_manager.cc:5697] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.404474 12725 raft_consensus.cc:2272] T 05adb1eba4fe476d8504a36537b738db P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.407791 12725 tablet_replica.cc:333] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.422822 12725 raft_consensus.cc:2243] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.405505 13767 catalog_manager.cc:5697] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:07:35.414963 13768 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57642 (request call id 277) took 24760 ms (client timeout 15000 ms). Trace:
I20260709 13:07:35.425736 13767 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 16 to 18, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 18 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:07:35.424723 13768 rpcz_store.cc:269] 0709 13:07:10.655121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.705360 (+3050239us) service_pool.cc:224] Handling call
0709 13:07:35.414888 (+21709528us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:35.270112 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:35.270785 (+   673us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:35.270828 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:07:35.270844 (+    16us) tablet.cc:662] Decoding operations
0709 13:07:35.272051 (+  1207us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:35.272111 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:35.272142 (+    31us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:07:35.273114 (+   972us) tablet.cc:701] Row locks acquired
0709 13:07:35.273131 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:07:35.273284 (+   153us) write_op.cc:270] Start()
0709 13:07:35.273403 (+   119us) write_op.cc:276] Timestamp: P: 1783602455273231 usec, L: 0
0709 13:07:35.273424 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:07:35.274252 (+   828us) log.cc:844] Serialized 4020 byte log entry
0709 13:07:35.277153 (+  2901us) op_driver.cc:464] REPLICATION: finished
0709 13:07:35.277638 (+   485us) write_op.cc:301] APPLY: starting
0709 13:07:35.277721 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:35.284263 (+  6542us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:35.284374 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:35.301932 (+ 17558us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:35.302200 (+   268us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:07:35.303443 (+  1243us) write_op.cc:312] APPLY: finished
0709 13:07:35.303825 (+   382us) log.cc:844] Serialized 97 byte log entry
0709 13:07:35.304041 (+   216us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:35.304190 (+   149us) write_op.cc:454] Released schema lock
0709 13:07:35.304266 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:35.322196 (+ 17930us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:35.322239 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11708,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":264,"apply.run_cpu_time_us":14623,"apply.run_wall_time_us":44497,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41740,"num_ops":10,"prepare.queue_time_us":137,"prepare.run_cpu_time_us":4516,"prepare.run_wall_time_us":7580,"replication_time_us":3571}]]}
I20260709 13:07:35.424187 12725 raft_consensus.cc:2272] T 92405e92560c4099974ff3e6a4564895 P e1649107c781428692458f10597c5742 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.428730 13767 catalog_manager.cc:5697] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 19 to 21, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 21 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.431265 13767 catalog_manager.cc:5697] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.432658 13767 catalog_manager.cc:5697] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 13 to 15, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 15 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.433912 13767 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 11 to 12, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 12 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:35.435331 13767 catalog_manager.cc:5697] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 17 to 22, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 22 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.436578 13767 catalog_manager.cc:5697] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 19 to 20, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 20 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:35.437986 13767 catalog_manager.cc:5697] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.440841 12725 tablet_replica.cc:333] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.441895 12725 raft_consensus.cc:2243] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.445456 13767 catalog_manager.cc:5697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 17 to 18, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 18 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.447182 13767 catalog_manager.cc:5697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.448854 13767 catalog_manager.cc:5697] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 16 to 17, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 17 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.450263 12725 raft_consensus.cc:2272] T 9a37812504ce4bdb94c7eafc67fa5aee P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.451221 13767 catalog_manager.cc:5697] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 13 to 16, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 16 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:35.452714 13767 catalog_manager.cc:5697] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 22 to 25, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 25 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:35.454577 12725 tablet_replica.cc:333] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.454185 13767 catalog_manager.cc:5697] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 18 to 22, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 22 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.455801 12725 raft_consensus.cc:2243] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.456578 12725 raft_consensus.cc:2272] T 0768fb01770349dfb259ab0db91394ba P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.460114 12725 tablet_replica.cc:333] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742: stopping tablet replica
W20260709 13:07:35.460968 13989 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:35.458482 13767 catalog_manager.cc:5697] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.462252 13767 catalog_manager.cc:5697] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.461277 12725 raft_consensus.cc:2243] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.479182 12725 raft_consensus.cc:2272] T 4daef29e1ece473f9845687b3c7eae60 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.490481 12725 tablet_replica.cc:333] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.491605 12725 raft_consensus.cc:2243] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.492838 12725 raft_consensus.cc:2272] T 9bfd3f44e83e4908b229bb57863e4080 P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.506206 12725 tablet_replica.cc:333] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.507390 12725 raft_consensus.cc:2243] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.508571 12725 raft_consensus.cc:2272] T 7f4c2ba739c04f16a8ed8c28f57fe695 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.511682 12725 tablet_replica.cc:333] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.512683 12725 raft_consensus.cc:2243] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.513875 12725 raft_consensus.cc:2272] T 693a6579cdaa4e2a977368c2c6d5267a P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.527240 12725 tablet_replica.cc:333] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.530526 12725 raft_consensus.cc:2243] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.532575 12725 raft_consensus.cc:2272] T c3f6783295084924801a104ff3faf10e P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.590965 12725 tablet_replica.cc:333] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.605480 21756 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.601838 12725 raft_consensus.cc:2243] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.609284 12725 raft_consensus.cc:2272] T 7991c821f3da4b7abc0f6bb6baf8fe99 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:35.619436 13989 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:35.623345 13766 catalog_manager.cc:5697] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 16 to 19, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 19 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:35.606181 21756 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:35.629809 21756 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:35.631997 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:35.613042 12725 tablet_replica.cc:333] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.633419 12725 raft_consensus.cc:2243] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.641402 12725 raft_consensus.cc:2272] T 34d11c517b334dd5abf16448009bdf2d P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.645341 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:35.647166 13840 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:35.649750 21756 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:35.657290 13915 consensus_peers.cc:597] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:35.674707 21473 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.675330 21473 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:35.677629 21473 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:35.679489 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:35.679947 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:35.681751 13838 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:35.690817 21756 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:35.698223 12725 tablet_replica.cc:333] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.700554 12725 raft_consensus.cc:2243] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.702124 13764 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 23 to 28, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 28 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:35.731823 13764 catalog_manager.cc:5697] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:35.733863 12725 raft_consensus.cc:2272] T 4a43a85b399e492b8391cc93e618a1fa P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.735378 13767 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57642 (request call id 278) took 8972 ms. Trace:
I20260709 13:07:35.755352 14552 consensus_queue.cc:237] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:35.778718 13947 raft_consensus.cc:1275] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:07:35.783890 21492 consensus_queue.cc:1048] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:07:35.788832 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:35.793576 12725 tablet_replica.cc:333] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742: stopping tablet replica
W20260709 13:07:35.795436 13989 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:35.803427 12725 raft_consensus.cc:2243] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.805850 12725 raft_consensus.cc:2272] T 10a0b27264ce49379f2aba40cdbc2b0f P e1649107c781428692458f10597c5742 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.767050 13767 rpcz_store.cc:276] 0709 13:07:26.763065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.464353 (+4701288us) service_pool.cc:224] Handling call
0709 13:07:35.735270 (+4270917us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:35.472294 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:35.472853 (+   559us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:35.472888 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:07:35.472905 (+    17us) tablet.cc:662] Decoding operations
0709 13:07:35.474874 (+  1969us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:35.474953 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:35.474981 (+    28us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:07:35.483086 (+  8105us) tablet.cc:701] Row locks acquired
0709 13:07:35.483106 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:07:35.483279 (+   173us) write_op.cc:270] Start()
0709 13:07:35.483408 (+   129us) write_op.cc:276] Timestamp: P: 1783602455483224 usec, L: 0
0709 13:07:35.483441 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:07:35.484210 (+   769us) log.cc:844] Serialized 7875 byte log entry
0709 13:07:35.493358 (+  9148us) op_driver.cc:464] REPLICATION: finished
0709 13:07:35.494264 (+   906us) write_op.cc:301] APPLY: starting
0709 13:07:35.494350 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:35.517266 (+ 22916us) spinlock_profiling.cc:243] Waited 8.3 ms on lock 0x7b180023aca0. stack: 00007fb2c0314343 00007fb2c0314240 00007fb2bfca31d4 00007fb2c99e1304 00007fb2c99e1299 00007fb2c920ede1 00007fb2c921b5b1 00007fb2c0346522 00007fb2c03487c1 00007fb2c0348779 00007fb2c0348709 00007fb2c03486d1 00007fb2c03479cd 000000000038f354 000000000038f199 00007fb2c032983a
0709 13:07:35.519631 (+  2365us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:35.519661 (+    30us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:35.533028 (+ 13367us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:35.533366 (+   338us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 13:07:35.533419 (+    53us) write_op.cc:312] APPLY: finished
0709 13:07:35.534365 (+   946us) log.cc:844] Serialized 180 byte log entry
0709 13:07:35.535264 (+   899us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:35.535497 (+   233us) write_op.cc:454] Released schema lock
0709 13:07:35.535569 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:35.581400 (+ 45831us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:35.581457 (+    57us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":35318,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":642,"apply.run_cpu_time_us":24531,"apply.run_wall_time_us":87083,"cfile_cache_hit":80,"cfile_cache_hit_bytes":83480,"num_ops":20,"prepare.queue_time_us":104,"prepare.run_cpu_time_us":5614,"prepare.run_wall_time_us":35835,"replication_time_us":9770,"spinlock_wait_cycles":19086080}]]}
I20260709 13:07:35.807369 21473 raft_consensus.cc:493] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.830597 21473 raft_consensus.cc:515] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:35.831411 13766 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54072 (request call id 143) took 15918 ms (client timeout 15000 ms). Trace:
I20260709 13:07:35.801777 13764 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 12 to 15, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 15 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:35.836033 13764 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 23 to 26, leader changed from 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 26 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:35.836184 21492 raft_consensus.cc:2955] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Committing config change with OpId 21.16: config changed from index 4 to 16, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:35.840071 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:35.842460 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d53d5ea0e8450e97d9e321677e63d5" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:35.843181 13942 raft_consensus.cc:2955] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index 4 to 16, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:07:35.844702 13759 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 90448.2us
W20260709 13:07:35.831903 13766 rpcz_store.cc:269] 0709 13:07:19.913438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.658852 (+2745414us) service_pool.cc:224] Handling call
0709 13:07:35.665819 (+13006967us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:35.665853 (+    34us) write_op.cc:454] Released schema lock
0709 13:07:35.831335 (+165482us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:35.652614 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:35.653212 (+   598us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:35.653268 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:07:35.653289 (+    21us) tablet.cc:662] Decoding operations
0709 13:07:35.653681 (+   392us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:35.653738 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:35.653784 (+    46us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:07:35.653979 (+   195us) tablet.cc:701] Row locks acquired
0709 13:07:35.654004 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:07:35.654173 (+   169us) write_op.cc:270] Start()
0709 13:07:35.654282 (+   109us) write_op.cc:276] Timestamp: P: 1783602455654131 usec, L: 0
0709 13:07:35.654301 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:07:35.654959 (+   658us) log.cc:844] Serialized 536 byte log entry
0709 13:07:35.658314 (+  3355us) op_driver.cc:464] REPLICATION: finished
0709 13:07:35.658754 (+   440us) write_op.cc:301] APPLY: starting
0709 13:07:35.658841 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:35.659804 (+   963us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:35.659909 (+   105us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:35.660694 (+   785us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:35.660870 (+   176us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:07:35.660925 (+    55us) write_op.cc:312] APPLY: finished
0709 13:07:35.661135 (+   210us) log.cc:844] Serialized 25 byte log entry
0709 13:07:35.661283 (+   148us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:35.661370 (+    87us) write_op.cc:454] Released schema lock
0709 13:07:35.661410 (+    40us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":60507,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":239,"apply.run_cpu_time_us":2983,"apply.run_wall_time_us":2983,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4174,"num_ops":1,"prepare.queue_time_us":118,"prepare.run_cpu_time_us":2669,"prepare.run_wall_time_us":2669,"replication_time_us":3885}]]}
I20260709 13:07:35.848392 13838 leader_election.cc:304] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:35.839112 21473 leader_election.cc:290] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:35.820277 12725 tablet_replica.cc:333] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742: stopping tablet replica
W20260709 13:07:35.853415 14524 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:35.859735 21475 raft_consensus.cc:2749] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:35.864064 13764 catalog_manager.cc:5697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 20 to 22, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 22 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:35.866540 12725 raft_consensus.cc:2243] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 17 LEADER]: Raft consensus shutting down.
W20260709 13:07:35.871009 13759 catalog_manager.cc:4755] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6b098cc2d51e471cb2c9c1b17eebcb57: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:07:35.881531 12725 raft_consensus.cc:2272] T f2e6672bab624b20b38e363591856fc0 P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:35.890111 13989 consensus_peers.cc:597] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:35.905164 12725 tablet_replica.cc:333] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.914420 12725 raft_consensus.cc:2243] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.914687 13758 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 14bba4e2cef04a418c6f660869c7e6e5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:35.915081 12725 pending_rounds.cc:70] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Trying to abort 2 pending ops.
I20260709 13:07:35.915357 12725 pending_rounds.cc:77] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Aborting op as it isn't in flight: id { term: 19 index: 11 } timestamp: 7305635619816181760 op_type: NO_OP noop_request { }
I20260709 13:07:35.915735 12725 raft_consensus.cc:2889] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:35.916044 12725 pending_rounds.cc:77] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742: Aborting op as it isn't in flight: id { term: 16 index: 10 } timestamp: 7305635498778071040 op_type: NO_OP noop_request { }
I20260709 13:07:35.916393 12725 raft_consensus.cc:2889] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:35.916745 12725 raft_consensus.cc:2272] T f50263f80db844239948e5e67ef3b987 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.919809 12725 tablet_replica.cc:333] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.920714 12725 raft_consensus.cc:2243] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.921926 12725 raft_consensus.cc:2272] T 02dcf286c6bd48e3a6e548def7d786cc P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.951977 12725 tablet_replica.cc:333] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742: stopping tablet replica
W20260709 13:07:35.953064 13915 consensus_peers.cc:597] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:35.972054 13766 catalog_manager.cc:5697] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 19 to 20, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 20 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.973876 13766 catalog_manager.cc:5697] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 22 to 23, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 23 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.975437 13766 catalog_manager.cc:5697] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.978785 13766 catalog_manager.cc:5697] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 17 to 22, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 22 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.980073 13766 catalog_manager.cc:5697] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 21 to 23, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 23 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.981326 13766 catalog_manager.cc:5697] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.982343 12725 raft_consensus.cc:2243] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.983729 12725 raft_consensus.cc:2272] T a168d82765704a87b2c60aeb7b23e8b0 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.983163 13766 catalog_manager.cc:5697] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 17 to 19, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 19 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.987097 12725 tablet_replica.cc:333] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:35.989832 12725 raft_consensus.cc:2243] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.990447 21475 raft_consensus.cc:493] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:35.991070 12725 raft_consensus.cc:2272] T 3730fc366f4349dea87ae85e1aed546c P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.990991 21475 raft_consensus.cc:515] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:35.991510 13766 catalog_manager.cc:5697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.987776 13764 catalog_manager.cc:5697] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 4 to 16, VOTER d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) evicted. New cstate: current_term: 21 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:35.993916 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:35.993716 13766 catalog_manager.cc:5697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 19 to 21, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 21 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:35.997098 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6c4cc997504124a8bb7fa9c7f10e34" candidate_uuid: "e1649107c781428692458f10597c5742" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:35.999152 13839 leader_election.cc:304] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1649107c781428692458f10597c5742; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:36.000002 21473 raft_consensus.cc:2749] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:35.997567 21475 leader_election.cc:290] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:36.003499 13766 catalog_manager.cc:5697] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.005465 13766 catalog_manager.cc:5697] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.006990 13766 catalog_manager.cc:5697] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 19 to 22, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 22 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:36.008435 12725 tablet_replica.cc:333] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.008611 13766 catalog_manager.cc:5697] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 16 to 17, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to e1649107c781428692458f10597c5742 (127.12.109.65). New cstate: current_term: 17 leader_uuid: "e1649107c781428692458f10597c5742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:36.011947 14012 tablet_service.cc:1558] Processing DeleteTablet for tablet 14bba4e2cef04a418c6f660869c7e6e5 with delete_type TABLET_DATA_TOMBSTONED (TS d1b4fd04200247d99ba9ad1fbeb42b42 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:34700
I20260709 13:07:36.013796 13766 catalog_manager.cc:5697] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } }
I20260709 13:07:36.022123 13766 catalog_manager.cc:5697] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 17 to 20, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 20 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.010423 12725 raft_consensus.cc:2243] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.028393 12725 raft_consensus.cc:2272] T 765348f2471a493d92e998132c0c746b P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.052255 12725 tablet_replica.cc:333] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.054579 12725 raft_consensus.cc:2243] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.055053 21773 tablet_replica.cc:333] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:36.055728 12725 raft_consensus.cc:2272] T 5d6ce914003a44f2b5a485ae7962383a P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.056353 21773 raft_consensus.cc:2243] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.057315 21773 raft_consensus.cc:2272] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.059104 12725 tablet_replica.cc:333] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.060112 12725 raft_consensus.cc:2243] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.061347 12725 raft_consensus.cc:2272] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.064601 12725 tablet_replica.cc:333] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.071376 12725 raft_consensus.cc:2243] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.072122 12725 pending_rounds.cc:70] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Trying to abort 1 pending ops.
I20260709 13:07:36.072438 12725 pending_rounds.cc:77] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742: Aborting op as it isn't in flight: id { term: 27 index: 20 } timestamp: 7305635614843559936 op_type: NO_OP noop_request { }
I20260709 13:07:36.072876 12725 raft_consensus.cc:2889] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:36.073293 12725 raft_consensus.cc:2272] T 435cf49d11154354987bf559d46d3c3a P e1649107c781428692458f10597c5742 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.076555 12725 tablet_replica.cc:333] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.077445 12725 raft_consensus.cc:2243] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.078177 12725 pending_rounds.cc:70] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Trying to abort 2 pending ops.
I20260709 13:07:36.078487 12725 pending_rounds.cc:77] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Aborting op as it isn't in flight: id { term: 19 index: 13 } timestamp: 7305635539104911360 op_type: NO_OP noop_request { }
I20260709 13:07:36.078915 12725 raft_consensus.cc:2889] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:36.079303 12725 pending_rounds.cc:77] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742: Aborting op as it isn't in flight: id { term: 17 index: 12 } timestamp: 7305635483683590144 op_type: NO_OP noop_request { }
I20260709 13:07:36.079702 12725 raft_consensus.cc:2889] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:36.080173 12725 raft_consensus.cc:2272] T 8dfe02d97ee6481298ba92b29baa32b7 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.080595 21773 ts_tablet_manager.cc:1916] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:36.083657 12725 tablet_replica.cc:333] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.084702 12725 raft_consensus.cc:2243] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.085856 12725 raft_consensus.cc:2272] T 17624553356541c69c1644ca409df1e9 P e1649107c781428692458f10597c5742 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.088954 12725 tablet_replica.cc:333] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742: stopping tablet replica
W20260709 13:07:36.090783 13915 consensus_peers.cc:597] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:36.093348 12725 raft_consensus.cc:2243] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.095580 12725 raft_consensus.cc:2272] T f9158487769746e2a7f4f283dda5abf3 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.099234 12725 tablet_replica.cc:333] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.100144 12725 raft_consensus.cc:2243] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.101027 12725 raft_consensus.cc:2272] T 82051b9bd85845d6b73773b21ccc6d6c P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.122309 12725 tablet_replica.cc:333] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.138507 12725 raft_consensus.cc:2243] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.140268 12725 raft_consensus.cc:2272] T dcd5a97e61554ed698b00fdf13294a7b P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.167356 12725 tablet_replica.cc:333] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.168421 12725 raft_consensus.cc:2243] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.174090 12725 raft_consensus.cc:2272] T 403f6d16105d4532ae1de441c32f632a P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.178779 12725 tablet_replica.cc:333] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.179670 12725 raft_consensus.cc:2243] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.180990 12725 raft_consensus.cc:2272] T be9dcc1a27af4a9a91a283a728d4132d P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.184399 12725 tablet_replica.cc:333] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.185367 12725 raft_consensus.cc:2243] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.186264 12725 raft_consensus.cc:2272] T ef6c4cc997504124a8bb7fa9c7f10e34 P e1649107c781428692458f10597c5742 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.193358 12725 tablet_replica.cc:333] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.194844 12725 raft_consensus.cc:2243] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.196405 21492 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.201683 21492 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:36.204098 21492 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.206239 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:36.208244 14524 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:36.208688 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:36.195744 12725 raft_consensus.cc:2272] T a3d53d5ea0e8450e97d9e321677e63d5 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.210115 21492 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:36.214004 12725 tablet_replica.cc:333] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.216998 12725 raft_consensus.cc:2243] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.218210 12725 raft_consensus.cc:2272] T 545fc4aeb71d491985e296a0721d905d P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.225018 12725 tablet_replica.cc:333] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.213430 21773 ts_tablet_manager.cc:1929] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 21.15
I20260709 13:07:36.226470 21773 log.cc:1199] T 14bba4e2cef04a418c6f660869c7e6e5 P d1b4fd04200247d99ba9ad1fbeb42b42: Deleting WAL directory at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0/minicluster-data/ts-2-root/wals/14bba4e2cef04a418c6f660869c7e6e5
I20260709 13:07:36.225960 12725 raft_consensus.cc:2243] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.227315 12725 raft_consensus.cc:2272] T 4e20f6d8df1f4522b10fd2dc803e0e75 P e1649107c781428692458f10597c5742 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.229085 13759 catalog_manager.cc:5028] TS d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): tablet 14bba4e2cef04a418c6f660869c7e6e5 (table test-workload [id=23a417905cf742398ff129309a2ade2b]) successfully deleted
I20260709 13:07:36.230530 12725 tablet_replica.cc:333] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.231758 12725 raft_consensus.cc:2243] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.232987 12725 raft_consensus.cc:2272] T 1ec9c4d3cc3a47bd838d18fe7da650fc P e1649107c781428692458f10597c5742 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.236498 12725 tablet_replica.cc:333] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.237314 12725 raft_consensus.cc:2243] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.237985 12725 raft_consensus.cc:2272] T 76071b78766a43e0b5097f615de4c569 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.241410 12725 tablet_replica.cc:333] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.242307 12725 raft_consensus.cc:2243] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.243526 12725 raft_consensus.cc:2272] T 47fee46f08eb4928939d6ac41891db38 P e1649107c781428692458f10597c5742 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.246651 12725 tablet_replica.cc:333] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.247481 12725 raft_consensus.cc:2243] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.248085 12725 raft_consensus.cc:2272] T f5c93fb008bd4a4f8db94cdc59c65d93 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.258809 12725 tablet_replica.cc:333] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.262637 12725 raft_consensus.cc:2243] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.263373 12725 raft_consensus.cc:2272] T fdef734205364543b02ed4158216bf36 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.266335 12725 tablet_replica.cc:333] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.267119 12725 raft_consensus.cc:2243] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.267385 21750 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.267784 12725 raft_consensus.cc:2272] T 481a7f1a133247ebb81e1be603cbdd57 P e1649107c781428692458f10597c5742 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.268078 21750 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.271268 12725 tablet_replica.cc:333] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742: stopping tablet replica
I20260709 13:07:36.272014 12725 raft_consensus.cc:2243] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.273262 12725 raft_consensus.cc:2272] T 17bd99d4be2c4073bd867c161fa52482 P e1649107c781428692458f10597c5742 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.276261 12725 tablet_replica.cc:333] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742: stopping tablet replica
W20260709 13:07:36.276757 13989 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:36.277004 12725 raft_consensus.cc:2243] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.277714 12725 raft_consensus.cc:2272] T 6d6d5ae5feb04bda8f546e7e4ca80503 P e1649107c781428692458f10597c5742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.278398 21750 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:36.280893 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:36.282606 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:36.283695 21750 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:36.443131 21769 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.443773 21769 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:36.446346 21769 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:36.448200 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:36.454598 13915 leader_election.cc:336] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:36.455106 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:36.456667 21748 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:36.457445 21722 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.458073 21722 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.462459 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:36.463527 21722 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:36.471932 13989 leader_election.cc:336] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:36.472429 13989 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:36.483758 21750 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:36.487314 21750 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.487865 21750 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.491427 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:36.495298 21748 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.495843 21748 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.503017 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:36.503823 14015 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 22.
W20260709 13:07:36.508584 13989 connection.cc:573] client connection to 127.12.109.65:43607 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:07:36.509510 13989 consensus_peers.cc:597] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:36.502496 21769 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.497975 21748 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.504876 21750 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:36.511057 21769 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.519358 21769 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.519510 21722 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.520406 21722 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.522838 21722 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:36.524729 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:36.533505 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:36.535501 21769 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:36.535964 21769 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.536367 21769 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:36.553532 13989 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.554232 13989 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:36.555140 21722 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:36.569561 13989 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.570283 13989 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:36.571898 21722 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:36.575510 21774 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.576225 21774 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.578027 21748 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.578614 21748 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.580694 21748 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.584410 21774 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.589097 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:36.589807 14022 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
I20260709 13:07:36.594189 21769 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.596434 21769 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.599895 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:36.602461 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:36.604764 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:36.605394 14022 raft_consensus.cc:3060] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:36.610656 21722 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.611330 21722 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:36.613821 21722 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:36.614797 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:36.615497 14552 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 26.
I20260709 13:07:36.615832 21769 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.616402 21769 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.618662 21769 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.620253 13988 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:36.621150 21722 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:36.623318 21722 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 26 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.623750 21722 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:36.626207 21769 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.626400 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:36.627000 21769 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.630455 21748 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:36.630966 21748 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.632378 21748 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:36.631191 14022 raft_consensus.cc:2468] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 23.
I20260709 13:07:36.631898 21769 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.638764 21750 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.639369 21750 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.644264 13915 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:36.652604 21769 raft_consensus.cc:2804] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:36.653188 21769 raft_consensus.cc:697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 23 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:36.654219 21769 consensus_queue.cc:237] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.641870 21750 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:36.668408 21722 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:36.682448 21757 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.683166 21757 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:36.690197 21722 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 27 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:36.677372 21748 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:36.695747 13989 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.698405 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:36.691924 21778 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:36.699764 13915 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.699120 13947 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 22.
I20260709 13:07:36.701889 21778 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.702737 21780 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.703316 21780 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.705437 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:36.706660 21750 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:36.707082 21750 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.707443 21750 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:36.711144 13989 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:36.712060 13989 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.714658 21757 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:36.716776 13989 leader_election.cc:336] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.716743 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:36.718793 13947 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
I20260709 13:07:36.720669 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:36.721618 13947 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:07:36.722926 13987 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:36.720069 14522 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:36.726576 21780 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.725097 21722 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:36.728295 21722 raft_consensus.cc:493] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.728721 21722 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:36.723564 21778 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:36.723912 21748 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.732748 21757 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:36.737748 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:36.739517 14015 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:36.737653 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:36.743403 21778 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.744014 21778 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.744973 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:36.745632 14016 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:36.748605 21778 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:36.750520 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:36.752261 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:36.752978 14015 raft_consensus.cc:3060] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:36.754585 21778 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:36.755198 13989 leader_election.cc:336] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.755170 21778 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:36.755911 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:36.757905 21778 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:36.758518 21778 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:36.761524 13915 leader_election.cc:336] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.739285 21757 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.762495 21757 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:36.762573 13915 leader_election.cc:336] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.763285 13915 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:36.764379 21779 raft_consensus.cc:2749] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:36.767279 13915 leader_election.cc:336] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.767742 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:36.768517 21779 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:36.770077 13915 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.771059 13765 catalog_manager.cc:5697] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 22 to 23, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 23 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:36.775384 21779 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.776005 21779 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.778343 21779 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.780478 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:36.781225 21779 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:36.781581 21779 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.781953 21779 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:36.786741 14015 raft_consensus.cc:2468] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:36.793051 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:36.793715 14552 raft_consensus.cc:3060] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:36.798738 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:36.799464 14561 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:36.799866 21779 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.805691 21722 raft_consensus.cc:515] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.806154 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:36.802899 14552 raft_consensus.cc:2468] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 27.
I20260709 13:07:36.807098 21778 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:36.792527 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:36.807852 21778 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.808310 21778 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:36.812923 21748 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:36.813432 21748 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.813840 21748 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:36.815922 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:36.818444 21779 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.824353 21757 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:36.828598 21779 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.831998 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4c2ba739c04f16a8ed8c28f57fe695" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:36.847908 13947 raft_consensus.cc:3060] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:36.847782 21779 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.831410 21722 leader_election.cc:290] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:36.829535 21778 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:36.843894 21769 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.844841 13915 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:36.845918 21748 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.857631 21779 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.857856 21748 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:36.847033 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 91873.2us
I20260709 13:07:36.789237 21750 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.863085 13947 raft_consensus.cc:2468] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:36.859822 13988 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:36.867972 14022 raft_consensus.cc:2393] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 23: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:36.876891 13987 leader_election.cc:304] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:36.881135 21778 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:36.882836 21778 raft_consensus.cc:2804] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:07:36.866866 21748 raft_consensus.cc:2804] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:36.865075 21779 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.863842 21769 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.889403 21788 raft_consensus.cc:2804] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:36.889648 21750 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:36.889976 21757 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:36.891937 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:36.892546 13947 raft_consensus.cc:3060] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:36.901448 13947 raft_consensus.cc:2468] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 19.
I20260709 13:07:36.883370 21778 raft_consensus.cc:697] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 27 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:36.904498 14522 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:36.906147 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:36.907202 13942 raft_consensus.cc:3060] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:36.888831 13915 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:36.913450 13915 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:36.913441 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 53702.8us
W20260709 13:07:36.929925 13915 leader_election.cc:336] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.890069 21788 raft_consensus.cc:697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:07:36.931042 13915 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.888068 21779 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.938663 21778 consensus_queue.cc:237] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:36.953792 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 55246.4us
W20260709 13:07:36.954602 14524 leader_election.cc:336] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.958261 21769 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:36.932425 21785 raft_consensus.cc:2804] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:36.961880 21785 raft_consensus.cc:697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:36.962872 21785 consensus_queue.cc:237] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:36.965433 21788 consensus_queue.cc:237] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.948894 13942 raft_consensus.cc:2468] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 23.
I20260709 13:07:36.970686 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:36.971377 14552 raft_consensus.cc:3060] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:36.972791 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.973361 14524 leader_election.cc:336] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.977734 14552 raft_consensus.cc:2468] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
W20260709 13:07:36.981601 13915 consensus_peers.cc:597] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:36.982378 13915 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:36.982894 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:36.988030 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:36.990465 21750 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:36.991087 21750 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:36.991637 21780 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:36.993297 21750 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:37.002538 13989 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.003731 13989 leader_election.cc:336] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.004719 13989 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.005446 13989 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.007579 21750 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.008162 21750 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.009934 13988 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.010785 21750 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.011380 21750 raft_consensus.cc:2804] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:37.011850 21750 raft_consensus.cc:697] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:37.013597 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:37.015678 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 69244.4us
I20260709 13:07:37.023032 21789 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.023615 21789 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:37.007197 21748 raft_consensus.cc:697] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 21 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:37.020594 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 116198us
I20260709 13:07:37.027961 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:37.029778 13947 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 15.
I20260709 13:07:37.018968 21774 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:37.044021 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:37.085243 13942 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:37.090444 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:37.091131 14015 raft_consensus.cc:3060] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:37.031081 21750 consensus_queue.cc:237] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:37.094843 13766 catalog_manager.cc:5697] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 26 to 27, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 27 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.096552 13766 catalog_manager.cc:5697] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 20 to 21, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 21 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:37.094236 21792 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.107295 21792 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:37.110462 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 91956us
I20260709 13:07:37.080919 21786 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.115396 13987 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.115633 21786 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:37.116984 13987 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:37.117873 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:37.118633 14019 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:37.109762 21792 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.126636 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:37.127993 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:37.134899 14022 raft_consensus.cc:3060] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:37.137384 13987 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:37.130172 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:37.097375 21778 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.138896 14552 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 28.
I20260709 13:07:37.138447 21793 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:37.148679 21793 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.140035 21722 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:37.149338 21722 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.149149 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:37.149897 21722 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:37.150246 13947 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:07:37.120080 21789 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:37.149140 21793 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:37.135746 21783 raft_consensus.cc:2804] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:37.142292 14015 raft_consensus.cc:2468] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 19.
I20260709 13:07:37.141142 13764 catalog_manager.cc:5697] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 18 to 19, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 19 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:37.152858 21778 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.155346 21778 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.157529 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:37.117946 21786 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:37.152168 21779 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:37.152712 21783 raft_consensus.cc:697] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:37.170670 21783 consensus_queue.cc:237] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:37.143617 13915 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.174398 13915 leader_election.cc:336] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.174921 13915 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.175457 13915 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.118230 21790 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.168247 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:37.160964 14016 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:37.169569 21789 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 13:07:37.184254 13989 leader_election.cc:336] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.179430 14022 raft_consensus.cc:2468] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 22.
I20260709 13:07:37.184273 21790 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.192019 21790 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:37.192203 14524 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.194402 13915 consensus_peers.cc:597] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:37.198822 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.182651 21789 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 28 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:37.216276 13915 consensus_peers.cc:597] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:37.211557 21778 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.176590 21748 consensus_queue.cc:237] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.226271 21778 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.228489 21778 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.230312 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:37.232882 21791 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:37.233369 21791 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.233733 21791 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:37.234953 21722 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.235812 21793 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.237257 21722 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.229533 21789 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:07:37.241029 21793 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:37.243371 13915 consensus_peers.cc:597] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:37.243933 13915 consensus_peers.cc:597] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:37.244462 13915 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.249198 14016 raft_consensus.cc:2393] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 16: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:37.249193 13989 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.244637 21750 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:37.250370 13989 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.251047 21750 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.253322 21750 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.254781 21779 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.255378 21779 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.256801 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:37.257423 13942 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
I20260709 13:07:37.257681 21779 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:37.261686 13915 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.262468 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:37.262913 13915 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.264321 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:07:37.266171 13989 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.230522 21783 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.271519 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:37.272346 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:37.273496 13947 raft_consensus.cc:3060] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:37.272814 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:37.275768 13946 raft_consensus.cc:3060] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:37.277891 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:37.279017 13987 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:37.282392 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:37.275619 21780 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:37.284188 21780 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.284682 21780 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:37.285585 21793 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:37.293684 21798 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:37.353155 21798 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.268018 21774 raft_consensus.cc:2804] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:37.353592 21798 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:37.353855 21779 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.354514 21779 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.356688 21779 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:37.267449 21799 raft_consensus.cc:2804] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:37.357702 21799 raft_consensus.cc:697] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:37.357779 21769 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.358408 21769 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.358705 21799 consensus_queue.cc:237] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.344837 21793 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.360431 21769 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:37.360596 21793 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:37.354518 21774 raft_consensus.cc:697] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 22 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:37.363137 21774 consensus_queue.cc:237] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.352363 13946 raft_consensus.cc:2468] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 16.
I20260709 13:07:37.348232 13947 raft_consensus.cc:2468] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:37.350955 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:37.351962 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:37.372655 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 89718.2us
I20260709 13:07:37.373406 21803 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.374150 21803 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.377240 21793 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.368259 21778 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.377983 21803 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:37.371628 21782 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.399005 21782 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:37.371838 21792 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:37.401369 21782 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.384971 13947 raft_consensus.cc:1275] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:07:37.386091 21798 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.389215 21791 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.391922 21789 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:37.398347 21786 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.398849 21778 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.401355 21793 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.381906 21780 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.404047 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:37.405468 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 126823us
I20260709 13:07:37.406569 21786 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:37.416466 21786 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.408217 14017 raft_consensus.cc:2393] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 29: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:37.409334 21778 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.278492 21783 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.407320 21782 consensus_queue.cc:1048] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.004s
I20260709 13:07:37.409855 21789 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 29 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:37.418884 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:37.419658 14524 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:37.419746 13989 consensus_peers.cc:597] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:37.418089 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:37.423245 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 156364us
I20260709 13:07:37.423738 13946 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 22.
W20260709 13:07:37.423857 13989 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.426179 21780 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:37.425705 21783 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.427551 21798 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.429150 21780 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:37.424440 13989 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:37.434944 13989 leader_election.cc:336] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.438340 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:37.439988 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:37.440407 14552 raft_consensus.cc:3060] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:07:37.440968 13766 catalog_manager.cc:5697] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 19 to 20, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 20 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.439661 21780 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.409387 21791 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:37.433444 13915 consensus_peers.cc:597] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:37.452169 14552 raft_consensus.cc:2468] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 29.
I20260709 13:07:37.444206 13766 catalog_manager.cc:5697] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 22 to 23, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 23 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:37.445712 21783 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:37.444012 13989 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.440979 21782 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:37.455797 21782 raft_consensus.cc:493] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.456264 21782 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:37.454560 14524 leader_election.cc:336] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.458676 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:37.461833 21492 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:37.450654 21780 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.466409 21780 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:37.467552 14524 leader_election.cc:336] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.471293 13988 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:07:37.492488 13989 leader_election.cc:336] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.486330 21798 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:37.500850 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:37.517437 13989 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.518862 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:37.490789 21783 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.490468 21789 raft_consensus.cc:2804] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:07:37.504012 21774 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:37.519274 13989 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.521915 13915 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.522514 13915 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.523281 13915 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.523862 13915 leader_election.cc:336] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.524250 13915 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.524629 13915 leader_election.cc:336] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.525817 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:37.526643 13941 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 22: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:37.527804 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:37.528419 14016 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 20.
W20260709 13:07:37.528724 12725 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 13896)
I20260709 13:07:37.529428 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86de6a18f3bd4b23a8b66dea2cf5c6a9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:37.530028 14022 raft_consensus.cc:3060] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:37.529336 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:37.531610 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:37.532409 14015 raft_consensus.cc:2393] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 22: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:37.533281 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:37.533414 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:37.527165 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:37.535145 14017 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:37.537708 14018 raft_consensus.cc:1275] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:07:37.534013 14016 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:37.543226 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:37.544596 21799 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:37.544209 13764 catalog_manager.cc:5697] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 20 to 21, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 21 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.546324 13764 catalog_manager.cc:5697] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 21 to 22, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 22 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.548013 13764 catalog_manager.cc:5697] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 18 to 19, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 19 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.521164 21798 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:37.552299 21798 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:37.552713 13989 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.525425 21774 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:37.553418 13989 leader_election.cc:336] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.522801 21795 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:07:37.553977 13989 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.554915 21774 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:37.558632 14524 consensus_peers.cc:597] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:37.559486 13989 leader_election.cc:336] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.560106 13989 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.521378 21783 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.563726 14019 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 22.
I20260709 13:07:37.564644 21783 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.571286 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:37.572065 13942 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 23.
I20260709 13:07:37.573171 13944 raft_consensus.cc:1275] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:07:37.573868 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:37.575654 13947 raft_consensus.cc:3060] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:37.577306 21782 raft_consensus.cc:515] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:37.579584 21782 leader_election.cc:290] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.580830 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee2fe56c0b4dd5bf2a9c07597c79ba" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:37.581485 13944 raft_consensus.cc:3060] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:37.586004 14524 leader_election.cc:336] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.587092 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:37.587785 13941 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:37.594534 13944 raft_consensus.cc:2468] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 23.
I20260709 13:07:37.596576 13947 raft_consensus.cc:2468] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 21.
W20260709 13:07:37.597700 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 64505.2us
I20260709 13:07:37.598371 13987 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.598412 14022 raft_consensus.cc:2468] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 22.
I20260709 13:07:37.599682 21790 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:37.595726 14522 leader_election.cc:304] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.600409 21789 raft_consensus.cc:697] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 29 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:37.601334 21789 consensus_queue.cc:237] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:37.603011 21782 raft_consensus.cc:2804] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:37.603637 21782 raft_consensus.cc:697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:37.604616 21782 consensus_queue.cc:237] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:37.605144 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:37.605237 21722 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.605856 21722 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.607939 21722 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.614526 14552 raft_consensus.cc:1275] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
W20260709 13:07:37.617816 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:37.618599 21792 consensus_queue.cc:1048] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 13:07:37.619119 21750 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.619719 21750 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:37.621709 13989 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.621886 21750 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.626430 21774 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.626184 13989 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:37.627992 14552 raft_consensus.cc:1275] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:07:37.632685 13989 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.633208 13987 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.634300 21810 raft_consensus.cc:2804] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:37.634480 21783 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:37.626960 21774 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.635128 21810 raft_consensus.cc:697] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:37.636116 21810 consensus_queue.cc:237] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.636725 21774 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:37.634318 21778 consensus_queue.cc:1048] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
W20260709 13:07:37.643029 13989 consensus_peers.cc:597] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:37.643733 13989 leader_election.cc:336] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.645826 13989 consensus_peers.cc:597] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:37.647452 13987 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:37.655228 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:07:37.656332 13915 leader_election.cc:336] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.659435 13987 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.660553 21793 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:37.661038 21793 raft_consensus.cc:493] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.661397 21793 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:37.642419 21750 raft_consensus.cc:2804] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:37.664760 21750 raft_consensus.cc:697] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:37.665748 21750 consensus_queue.cc:237] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.666174 21790 consensus_queue.cc:1048] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:37.667514 13987 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.672014 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.674615 13915 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:37.677593 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:37.679785 13942 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 23.
I20260709 13:07:37.681586 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:37.683859 13944 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 22.
I20260709 13:07:37.686740 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:37.687448 13948 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 22.
I20260709 13:07:37.666992 21783 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:37.687935 21783 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.688347 21783 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:37.689239 21798 raft_consensus.cc:2804] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:37.689795 21798 raft_consensus.cc:697] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:37.690735 21798 consensus_queue.cc:237] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.657528 21778 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:37.697546 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.700192 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.695871 21780 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:37.702725 21780 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.703166 21748 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:37.703752 21748 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.704154 21748 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:37.702769 21492 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.704703 21492 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:37.703260 21780 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:37.707576 21492 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:37.710253 21779 consensus_queue.cc:1048] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:37.711186 21799 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.711773 21799 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:37.714393 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.698555 21791 raft_consensus.cc:2749] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:37.728444 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:37.730469 14524 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.729592 13764 catalog_manager.cc:5697] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 22 to 23, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 23 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:37.732643 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:37.734786 13989 consensus_peers.cc:597] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:37.726549 21799 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:37.742195 14017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:37.742518 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:37.743115 14018 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 22.
I20260709 13:07:37.731144 14555 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:37.720698 21774 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:37.746273 13765 catalog_manager.cc:5697] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 28 to 29, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 29 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.737618 13915 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
W20260709 13:07:37.739383 13989 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:37.749464 13989 consensus_peers.cc:597] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:37.749693 13915 leader_election.cc:336] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.745004 13914 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:37.746423 21800 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:37.742908 14017 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 16.
I20260709 13:07:37.747650 21774 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.767741 13987 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.753155 21799 raft_consensus.cc:2804] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:37.751379 21800 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.766726 21807 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:37.778237 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.790882 13987 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:37.798660 21788 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:37.770479 21774 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:37.797777 21778 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:37.787485 21800 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:37.799315 21788 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.786983 21807 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
W20260709 13:07:37.836730 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 77072.6us
I20260709 13:07:37.801239 21818 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:37.821174 21778 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.834411 21788 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:37.838666 21807 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:37.894690 13988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 44060.4us
I20260709 13:07:37.843454 21818 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:37.846599 21778 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:37.918861 21814 log.cc:927] Time spent T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:07:37.926604 21820 log.cc:927] Time spent T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:07:37.937688 21793 raft_consensus.cc:515] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.950119 21780 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.951674 21793 leader_election.cc:290] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:37.952263 21780 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:37.910120 21818 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:07:37.979931 13989 leader_election.cc:336] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:37.966391 21748 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.993942 21748 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:37.995173 21819 log.cc:927] Time spent T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: Append to log took a long time: real 0.144s	user 0.000s	sys 0.004s
W20260709 13:07:38.007117 13989 consensus_peers.cc:597] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:38.008659 21807 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:38.013270 21778 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:37.946203 21799 raft_consensus.cc:697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 22 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:38.024766 21783 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:38.028982 13989 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:38.042639 21799 consensus_queue.cc:237] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:38.048393 13946 raft_consensus.cc:1275] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:07:38.060626 21757 consensus_queue.cc:1048] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:38.065044 21818 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:38.066184 13989 consensus_peers.cc:597] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:38.066963 21807 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:38.068077 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bd99d4be2c4073bd867c161fa52482" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:38.068708 14552 raft_consensus.cc:3060] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:38.069243 21774 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:38.071821 21774 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:38.016038 21788 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:38.078388 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 285994us
I20260709 13:07:38.084453 14022 raft_consensus.cc:2393] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 24: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:38.088395 14552 raft_consensus.cc:2468] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 22.
I20260709 13:07:38.089751 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:38.091163 13914 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:07:38.097239 13989 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:38.102716 21800 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:38.103677 21788 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:38.106204 21818 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:38.106444 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcd5a97e61554ed698b00fdf13294a7b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:38.106914 13948 raft_consensus.cc:1275] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:07:38.140014 13944 raft_consensus.cc:3060] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:38.109664 21783 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:38.111011 21778 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:38.119006 13764 catalog_manager.cc:5697] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 15 to 16, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 16 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:38.129190 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 220633us
I20260709 13:07:38.144172 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:38.155279 21774 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:38.155771 21774 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:38.156186 21774 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:38.157181 21800 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:38.140796 21786 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:38.158720 21786 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e1649107c781428692458f10597c5742)
I20260709 13:07:38.159099 21786 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:38.168522 13989 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:38.169188 13989 leader_election.cc:336] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:38.170919 13989 leader_election.cc:336] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:38.143245 13764 catalog_manager.cc:5697] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 17 to 18, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 18 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:38.176468 13764 catalog_manager.cc:5697] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 20 to 21, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 21 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:38.185262 13766 catalog_manager.cc:5697] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 21 to 22, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 22 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.186451 21835 raft_consensus.cc:2804] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:07:38.190121 13915 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:38.191514 14552 raft_consensus.cc:1275] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 29 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 13:07:38.194710 21835 raft_consensus.cc:697] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 22 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:38.195703 21835 consensus_queue.cc:237] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:38.194006 13944 raft_consensus.cc:2468] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:38.204394 21774 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:38.191767 21786 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:38.211558 21786 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:38.223453 13915 leader_election.cc:336] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:38.224596 13915 consensus_peers.cc:597] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.225255 13915 consensus_peers.cc:597] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:38.230870 14524 consensus_peers.cc:597] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:38.238152 21832 consensus_queue.cc:1048] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.008s
I20260709 13:07:38.246431 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:38.246701 21774 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:38.247176 14016 raft_consensus.cc:3060] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:38.250736 14017 raft_consensus.cc:1275] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:07:38.253612 14524 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:38.258665 13989 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:38.261127 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:38.261785 14019 raft_consensus.cc:3060] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:38.262418 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec9c4d3cc3a47bd838d18fe7da650fc" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:38.263034 14024 raft_consensus.cc:3060] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:38.264770 14017 raft_consensus.cc:1275] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:38.261976 14018 raft_consensus.cc:1275] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:07:38.267053 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:38.268010 13915 consensus_peers.cc:597] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.271672 13915 consensus_peers.cc:597] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:38.271762 14015 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 23: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:38.272342 13915 consensus_peers.cc:597] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:38.272814 13915 consensus_peers.cc:597] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.273259 13915 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:38.273700 14020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:38.273854 13915 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:38.274448 13915 leader_election.cc:336] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:38.274446 14020 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 23: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
W20260709 13:07:38.275266 13915 leader_election.cc:336] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:38.275769 13915 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:38.276526 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:38.277246 13944 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 23: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
W20260709 13:07:38.280907 13915 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:38.281708 13989 consensus_peers.cc:597] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:38.285979 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:38.286741 13944 raft_consensus.cc:3060] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:38.286955 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:38.287771 21774 raft_consensus.cc:3060] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:38.296028 13942 raft_consensus.cc:1275] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:38.299882 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:38.300109 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:38.300997 13948 raft_consensus.cc:2393] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 23: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:38.304563 13947 raft_consensus.cc:1275] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:38.309087 13766 catalog_manager.cc:5697] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 21 to 22, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 22 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.325213 14019 raft_consensus.cc:2468] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:38.335839 14024 raft_consensus.cc:2468] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:38.343344 14016 raft_consensus.cc:2468] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 21.
I20260709 13:07:38.346122 13944 raft_consensus.cc:2468] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 24.
I20260709 13:07:38.343988 21774 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:38.355752 13946 raft_consensus.cc:2468] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 24.
I20260709 13:07:38.365732 21835 consensus_queue.cc:1048] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:38.370114 21835 consensus_queue.cc:1048] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:38.381793 21774 consensus_queue.cc:1048] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:38.383292 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:38.391076 21835 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:38.385604 13946 raft_consensus.cc:1275] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:07:38.395560 13987 leader_election.cc:304] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:38.395925 21818 consensus_queue.cc:1048] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
W20260709 13:07:38.403143 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:38.422443 14524 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:38.423503 13989 consensus_peers.cc:597] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:38.426338 21824 raft_consensus.cc:2804] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:38.426926 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:38.426986 21824 raft_consensus.cc:697] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:38.428715 21824 consensus_queue.cc:237] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:38.428532 13915 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:38.441321 21774 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:38.436851 13987 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:38.434727 21810 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:38.436681 21832 consensus_queue.cc:1048] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:07:38.462450 21818 raft_consensus.cc:2749] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:38.467989 21832 consensus_queue.cc:1048] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:38.470578 13987 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:38.471520 21810 raft_consensus.cc:2804] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:38.472251 21810 raft_consensus.cc:697] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:38.473914 13987 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:38.473837 21810 consensus_queue.cc:237] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:38.512714 21778 raft_consensus.cc:2804] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:38.513387 21778 raft_consensus.cc:697] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:38.514405 21778 consensus_queue.cc:237] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:38.518718 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 44887.3us
I20260709 13:07:38.521615 13764 catalog_manager.cc:5697] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 17 to 18, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 18 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:38.529122 12725 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 13896)
I20260709 13:07:38.530421 21832 consensus_queue.cc:1048] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:07:38.535076 13989 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:38.536475 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:38.537333 14022 raft_consensus.cc:3060] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.545917 14022 raft_consensus.cc:2468] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:07:38.555240 14524 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:38.556161 13915 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:38.557384 13915 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:38.558687 21782 raft_consensus.cc:2804] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:38.559300 21782 raft_consensus.cc:697] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:38.559880 21807 raft_consensus.cc:2804] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:38.560451 21807 raft_consensus.cc:697] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:38.560262 21782 consensus_queue.cc:237] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:38.561300 21807 consensus_queue.cc:237] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:38.565994 13915 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:38.570353 21799 raft_consensus.cc:2804] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:07:38.572504 13915 consensus_peers.cc:597] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:38.579730 21774 raft_consensus.cc:2804] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:38.606468 13764 catalog_manager.cc:5697] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 23 to 24, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 24 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:38.619881 13764 catalog_manager.cc:5697] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 23 to 24, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 24 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:38.590708 21799 raft_consensus.cc:697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:38.629900 21774 raft_consensus.cc:697] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 21 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:38.661217 21799 consensus_queue.cc:237] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:38.708573 13989 consensus_peers.cc:597] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.735658 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 64326.5us
I20260709 13:07:38.754809 14016 raft_consensus.cc:1275] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:07:38.761073 21774 consensus_queue.cc:237] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:38.786231 21850 log.cc:927] Time spent T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 13:07:38.787947 21847 log.cc:927] Time spent T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 13:07:38.831809 13989 consensus_peers.cc:597] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:38.845296 21849 log.cc:927] Time spent T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.098s	user 0.001s	sys 0.001s
I20260709 13:07:38.855966 14552 raft_consensus.cc:1275] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
W20260709 13:07:38.878784 21846 log.cc:927] Time spent T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.141s	user 0.002s	sys 0.000s
I20260709 13:07:38.892967 13766 catalog_manager.cc:5697] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 16 to 17, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 17 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:38.922928 13915 consensus_peers.cc:597] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:38.923708 13915 consensus_peers.cc:597] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:38.951869 21856 consensus_queue.cc:1048] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:07:38.943298 21861 consensus_queue.cc:1048] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.066s
I20260709 13:07:39.008737 13944 raft_consensus.cc:1275] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:07:38.975564 13764 catalog_manager.cc:5697] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 17 to 18, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 18 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:39.011917 13944 raft_consensus.cc:1275] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:07:39.027102 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 56183.5us
I20260709 13:07:39.071170 13948 raft_consensus.cc:1275] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:07:39.082082 21874 consensus_queue.cc:1048] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:07:39.076176 21790 consensus_queue.cc:1048] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:07:39.114704 21750 consensus_queue.cc:1048] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
W20260709 13:07:39.139225 13989 consensus_peers.cc:597] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.139914 13989 consensus_peers.cc:597] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.142477 14524 consensus_peers.cc:597] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.163367 13915 consensus_peers.cc:597] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.226220 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 150382us
W20260709 13:07:39.242925 13915 consensus_peers.cc:597] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:39.245831 13989 consensus_peers.cc:597] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.255116 13989 consensus_peers.cc:597] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:39.267676 13915 consensus_peers.cc:597] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.312784 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:39.330271 13766 catalog_manager.cc:5697] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 20 to 21, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 21 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:39.337051 13766 catalog_manager.cc:5697] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 17 to 18, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 18 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:39.345268 14020 raft_consensus.cc:1275] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:07:39.331142 13915 consensus_peers.cc:597] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:39.386299 14015 raft_consensus.cc:1275] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:39.388216 14017 raft_consensus.cc:1275] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:07:39.444203 21795 consensus_queue.cc:1048] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:07:39.483773 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:39.498557 21778 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.499266 21778 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:39.501507 21778 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:39.509793 13989 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:39.518795 21774 consensus_queue.cc:1048] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
W20260709 13:07:39.529747 12725 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 13896)
I20260709 13:07:39.535825 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:39.536602 13941 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 22.
I20260709 13:07:39.538545 21856 consensus_queue.cc:1048] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:39.563138 13987 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:39.564163 21862 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:39.564606 21862 raft_consensus.cc:493] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.564952 21862 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:39.593362 13915 consensus_peers.cc:597] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:39.610852 14022 raft_consensus.cc:1275] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:07:39.647822 21807 consensus_queue.cc:1048] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
W20260709 13:07:39.739621 14524 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:39.751385 21862 raft_consensus.cc:515] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:39.795946 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7991c821f3da4b7abc0f6bb6baf8fe99" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:39.796646 13948 raft_consensus.cc:3060] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:39.768519 13915 consensus_peers.cc:597] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:39.814706 13948 raft_consensus.cc:2468] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 23.
I20260709 13:07:39.817706 13987 leader_election.cc:304] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:39.825259 21868 raft_consensus.cc:2804] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:07:39.823730 13989 leader_election.cc:336] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:39.827705 21868 raft_consensus.cc:697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:39.828673 21868 consensus_queue.cc:237] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:39.840296 13915 consensus_peers.cc:597] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:39.809468 21862 leader_election.cc:290] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:39.888302 13915 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111) [suppressed 390 similar messages]
I20260709 13:07:39.910334 13766 catalog_manager.cc:5697] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 21 to 23, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 23 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:39.972637 13915 consensus_peers.cc:597] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:39.995960 21862 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.996565 21862 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:40.002825 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:40.011972 13989 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:40.018154 21862 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:40.020821 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:40.021643 13946 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 23.
I20260709 13:07:40.022927 13987 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:40.024199 21862 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:40.024623 21862 raft_consensus.cc:493] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.024982 21862 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:07:40.053479 13915 consensus_peers.cc:597] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:40.098276 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:40.111786 13915 consensus_peers.cc:597] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:40.120625 21862 raft_consensus.cc:515] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:40.148479 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9dcc1a27af4a9a91a283a728d4132d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:40.149240 13946 raft_consensus.cc:3060] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:07:40.203050 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 59936.1us
I20260709 13:07:40.144733 21862 leader_election.cc:290] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:40.530421 12725 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 13896)
I20260709 13:07:40.542068 13946 raft_consensus.cc:2468] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 24.
I20260709 13:07:40.788780 21880 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.266887 21905 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.267804 21905 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.271044 21905 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:40.778837 13987 leader_election.cc:304] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:40.797664 21880 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.320670 21880 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.378772 21785 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.380887 21785 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.386140 21785 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.398649 21785 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.400017 21785 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.403843 21785 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:41.431237 14522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.038s	user 0.011s	sys 0.024s
W20260709 13:07:41.432003 14522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.039s	user 0.011s	sys 0.024s
I20260709 13:07:41.445796 21914 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.446945 21914 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.449710 21785 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.450402 21914 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.453058 21785 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.457249 21858 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.458177 21858 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.473279 21858 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.471486 21785 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.495386 21785 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.497138 21785 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:41.498127 13989 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.004s	sys 0.008s
W20260709 13:07:41.499248 13989 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.004s	sys 0.010s
W20260709 13:07:41.499903 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 778434us
I20260709 13:07:41.505336 21785 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:41.512080 13990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.006s	sys 0.025s
W20260709 13:07:41.512629 13990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.006s	sys 0.025s
W20260709 13:07:41.531605 12725 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 13896)
I20260709 13:07:41.538298 21914 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.539407 21914 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:41.554741 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 775798us
I20260709 13:07:41.567298 21874 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.567456 21899 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.568322 21874 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.569868 21912 raft_consensus.cc:2804] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:07:41.570478 13915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.178s	user 0.012s	sys 0.007s
I20260709 13:07:41.568506 21899 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.570749 21912 raft_consensus.cc:697] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:41.573444 21899 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:41.574569 13913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.183s	user 0.023s	sys 0.049s
W20260709 13:07:41.575038 13913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.184s	user 0.023s	sys 0.049s
I20260709 13:07:41.575603 21874 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.582568 21914 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:41.592607 14524 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.887s	user 0.009s	sys 0.005s
W20260709 13:07:41.593297 14524 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.010s	sys 0.006s
W20260709 13:07:41.570963 13915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.179s	user 0.012s	sys 0.007s
W20260709 13:07:41.596748 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 1.39215e+06us
I20260709 13:07:41.598465 21785 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.599551 21785 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.602607 21907 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.602715 21911 raft_consensus.cc:493] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.603788 21911 raft_consensus.cc:515] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.606254 21785 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.606467 21903 raft_consensus.cc:493] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.607489 21903 raft_consensus.cc:515] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.603708 21907 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.612313 21907 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.613306 21861 raft_consensus.cc:493] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:41.614420 21807 raft_consensus.cc:493] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:41.614512 21853 raft_consensus.cc:493] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.615564 21807 raft_consensus.cc:515] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.615670 21853 raft_consensus.cc:515] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:41.589727 13989 leader_election.cc:336] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:41.620157 21861 raft_consensus.cc:515] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.621275 21874 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.627897 21874 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.634776 21862 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.635509 21862 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.638319 21874 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.639267 21861 leader_election.cc:290] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:41.641856 21769 raft_consensus.cc:493] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:41.654667 21912 consensus_queue.cc:237] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.665066 21769 raft_consensus.cc:515] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.668262 21769 leader_election.cc:290] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:41.671463 21899 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.672397 21899 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.672904 21911 leader_election.cc:290] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.673923 21903 leader_election.cc:290] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.674407 21868 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.675508 21868 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.676466 21900 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.678728 21901 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.679440 21899 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.679517 21901 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.680469 21899 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.682374 21881 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:41.683216 21868 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:41.681391 21899 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.683364 21881 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.688472 21778 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.689221 21778 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.690853 21853 leader_election.cc:290] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.692943 21807 leader_election.cc:290] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:41.677376 21900 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.691936 21778 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.637995 21862 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.712155 21750 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.712764 21899 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.715554 21901 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.715224 21750 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.698452 21880 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.707808 21874 raft_consensus.cc:493] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:41.710902 21868 raft_consensus.cc:493] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:41.732013 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 80209.2us
I20260709 13:07:41.722265 21900 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.737164 21906 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.734000 21874 raft_consensus.cc:515] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } attrs { replace: true } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.746333 21880 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.748732 21874 leader_election.cc:290] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:41.748757 21880 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.760267 21880 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.761181 21880 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.764360 21874 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.773484 21880 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.773363 21874 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.731923 21881 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:41.762542 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 172911us
I20260709 13:07:41.737948 21906 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.776028 21861 raft_consensus.cc:493] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:41.777635 21807 raft_consensus.cc:493] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:41.777904 21880 raft_consensus.cc:493] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:41.787299 21750 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.786211 21868 raft_consensus.cc:515] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.790421 21880 raft_consensus.cc:515] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.792187 21868 leader_election.cc:290] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:41.794739 21861 raft_consensus.cc:515] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.795620 21911 raft_consensus.cc:493] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.796331 21911 raft_consensus.cc:515] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.797528 21861 leader_election.cc:290] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:41.798764 21911 leader_election.cc:290] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.798632 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05adb1eba4fe476d8504a36537b738db" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:41.800037 21906 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.801028 21880 leader_election.cc:290] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:41.801506 21874 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.798640 21921 raft_consensus.cc:493] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.802830 21921 raft_consensus.cc:515] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.805267 21921 leader_election.cc:290] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.784226 21853 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.811631 21853 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.814113 21853 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:41.807475 21807 raft_consensus.cc:515] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.817036 21903 raft_consensus.cc:493] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.817870 21903 raft_consensus.cc:515] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.818215 21919 raft_consensus.cc:493] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
W20260709 13:07:41.818989 13804 debug-util.cc:398] Leaking SignalData structure 0x7b080033e860 after lost signal to thread 12728
I20260709 13:07:41.820364 21903 leader_election.cc:290] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:41.820768 13804 debug-util.cc:398] Leaking SignalData structure 0x7b0800195360 after lost signal to thread 13807
I20260709 13:07:41.820912 21903 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.819056 21919 raft_consensus.cc:515] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.823191 21903 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.825582 21903 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:41.825882 13804 debug-util.cc:398] Leaking SignalData structure 0x7b08002fa6e0 after lost signal to thread 13974
W20260709 13:07:41.828873 13804 debug-util.cc:398] Leaking SignalData structure 0x7b080003cb60 after lost signal to thread 14048
I20260709 13:07:41.830225 21911 raft_consensus.cc:493] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.826713 21807 leader_election.cc:290] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:41.831000 13804 debug-util.cc:398] Leaking SignalData structure 0x7b0800287500 after lost signal to thread 14585
W20260709 13:07:41.833437 13804 debug-util.cc:398] Leaking SignalData structure 0x7b08003a94a0 after lost signal to thread 21621
I20260709 13:07:41.829705 21861 raft_consensus.cc:493] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.834168 21861 raft_consensus.cc:515] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.836570 21861 leader_election.cc:290] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.838406 21880 raft_consensus.cc:493] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
W20260709 13:07:41.838995 13804 debug-util.cc:398] Leaking SignalData structure 0x7b0800388fc0 after lost signal to thread 21727
I20260709 13:07:41.831936 21874 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.839155 21880 raft_consensus.cc:515] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.839684 21874 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.832576 21906 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:41.842698 21906 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.845923 21906 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:41.839461 13948 raft_consensus.cc:1275] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:07:41.841903 21874 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:41.830921 21911 raft_consensus.cc:515] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.832382 21919 leader_election.cc:290] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.846392 21880 leader_election.cc:290] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
W20260709 13:07:41.851491 13804 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1e60 after lost signal to thread 21737
I20260709 13:07:41.854964 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403f6d16105d4532ae1de441c32f632a" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:41.856986 13804 debug-util.cc:398] Leaking SignalData structure 0x7b0800257180 after lost signal to thread 21774
W20260709 13:07:41.857399 13804 debug-util.cc:398] Leaking SignalData structure 0x7b0800021820 after lost signal to thread 21776
W20260709 13:07:41.857640 13804 debug-util.cc:398] Leaking SignalData structure 0x7b0800356fa0 after lost signal to thread 21777
I20260709 13:07:41.856418 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:41.859128 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c93fb008bd4a4f8db94cdc59c65d93" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:41.860387 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.861047 13948 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:07:41.850725 21853 raft_consensus.cc:493] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.862205 21853 raft_consensus.cc:515] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:41.862533 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.862591 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e20f6d8df1f4522b10fd2dc803e0e75" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.863245 13945 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 23.
I20260709 13:07:41.864485 21853 leader_election.cc:290] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.865453 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a168d82765704a87b2c60aeb7b23e8b0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.866709 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.867328 13949 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 24.
I20260709 13:07:41.868237 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.877050 21911 leader_election.cc:290] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.878706 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:41.879168 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:41.879503 14555 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:41.879817 14561 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:41.880932 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435cf49d11154354987bf559d46d3c3a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:41.882341 14552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:41.885519 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.887692 14522 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:07:41.888595 13804 debug-util.cc:398] Leaking SignalData structure 0x7b08003a1e00 after lost signal to thread 21876
I20260709 13:07:41.889585 21785 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:41.903163 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b028ec30974d06bd9da921c268a641" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.903439 21785 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.903991 21785 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:41.890316 14522 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:41.895043 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2e6672bab624b20b38e363591856fc0" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:41.903509 13804 debug-util.cc:398] Leaking SignalData structure 0x7b080017f640 after lost signal to thread 21883
I20260709 13:07:41.896906 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.898550 13914 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
W20260709 13:07:41.922256 13804 debug-util.cc:398] Leaking SignalData structure 0x7b08002b2900 after lost signal to thread 21885
I20260709 13:07:41.900205 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728de4a891d847028775aa5c35ba53e1" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.923311 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "617d39d9397c4724be821e5efacbde17" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.923648 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50263f80db844239948e5e67ef3b987" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.924849 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9907790614d49bc3adf40424fc1b6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.925486 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.927449 13947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d11c517b334dd5abf16448009bdf2d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.902432 13945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76071b78766a43e0b5097f615de4c569" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.914608 13950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.915508 21905 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:41.928692 21905 raft_consensus.cc:493] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.928779 13914 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:41.929162 21905 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:41.930874 21919 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:41.931264 21919 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.931650 21919 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:41.916215 13946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9158487769746e2a7f4f283dda5abf3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.934792 21911 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:41.935304 21911 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:41.935706 21911 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:41.918975 13949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd3f44e83e4908b229bb57863e4080" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.896178 13948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.899338 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a0b27264ce49379f2aba40cdbc2b0f" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.901786 13942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4033a61013848c69bbd0e79b27a8be3" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:41.939074 13804 debug-util.cc:398] Leaking SignalData structure 0x7b0800302820 after lost signal to thread 21886
W20260709 13:07:41.939380 13804 debug-util.cc:398] Leaking SignalData structure 0x7b0800317420 after lost signal to thread 21887
I20260709 13:07:41.925942 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:41.902235 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56605211bd584b8889b54049e9f6cb4b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
W20260709 13:07:41.947981 13989 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:41.951627 21785 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.954933 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:41.956019 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3f6783295084924801a104ff3faf10e" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:41.962877 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:41.964726 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2c65bca5ce45d5af2ace962fa0a37f" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:41.971789 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:41.973539 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:41.973609 21785 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:41.974408 14015 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 15.
I20260709 13:07:41.975721 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02dcf286c6bd48e3a6e548def7d786cc" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:41.977317 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:41.977407 13989 leader_election.cc:336] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.978019 13989 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.978580 13989 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:41.979064 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4daef29e1ece473f9845687b3c7eae60" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:41.984786 13989 leader_election.cc:336] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:41.984656 21911 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:41.985411 13989 leader_election.cc:336] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.985944 13989 consensus_peers.cc:597] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.986398 13989 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.986872 13989 leader_election.cc:336] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.987336 13989 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.987808 13989 leader_election.cc:336] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.988257 13989 leader_election.cc:336] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.988718 13989 leader_election.cc:336] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.989163 13989 leader_election.cc:336] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.989624 13989 leader_election.cc:336] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.992017 13989 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:41.991878 21880 consensus_queue.cc:1048] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:07:41.992635 13989 leader_election.cc:336] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.993197 13989 leader_election.cc:336] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.993732 13989 leader_election.cc:336] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:41.994289 13989 consensus_peers.cc:597] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:41.994760 13989 consensus_peers.cc:597] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:41.995486 13989 consensus_peers.cc:597] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.996047 13989 consensus_peers.cc:597] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:42.001531 13989 consensus_peers.cc:597] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:42.002259 13989 consensus_peers.cc:597] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:42.005362 13766 catalog_manager.cc:5697] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 22 to 24, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 24 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:42.006597 13989 leader_election.cc:336] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:42.007143 13989 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:41.993270 21919 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:41.981210 21905 raft_consensus.cc:515] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:42.016185 21905 leader_election.cc:290] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:42.016989 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:42.017129 21919 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:42.017036 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:42.018872 14553 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:42.017197 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.024514 14552 raft_consensus.cc:1240] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:42.033499 14559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545fc4aeb71d491985e296a0721d905d" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:42.030262 13989 consensus_peers.cc:597] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:42.031213 14553 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:42.031454 14551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:42.035346 21880 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:42.032531 14555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:42.035007 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:42.027287 14558 raft_consensus.cc:1240] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:42.037528 21912 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:42.037976 21912 raft_consensus.cc:493] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.038347 21912 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:42.039068 13914 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:42.041138 14561 raft_consensus.cc:1240] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Rejecting Update request from peer d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:42.043128 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:42.044027 14553 raft_consensus.cc:2393] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 18: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:42.049096 21912 raft_consensus.cc:515] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:42.050258 21919 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:42.035017 21911 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:42.051723 21919 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:42.052876 21919 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:42.054209 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:42.058369 13989 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.065557 21912 leader_election.cc:290] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: Requested vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:42.066309 21874 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:42.070139 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47fee46f08eb4928939d6ac41891db38" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:42.072124 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b098cc2d51e471cb2c9c1b17eebcb57" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:42.075359 14554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6d5ae5feb04bda8f546e7e4ca80503" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:42.075717 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
W20260709 13:07:42.079643 14524 consensus_peers.cc:597] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:42.092549 13988 leader_election.cc:304] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:42.093561 21906 consensus_queue.cc:1059] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:07:42.093918 21868 raft_consensus.cc:2749] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:42.094884 21750 raft_consensus.cc:3055] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:42.095275 21750 raft_consensus.cc:740] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:42.096311 21750 consensus_queue.cc:260] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:42.097767 21750 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:42.103144 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.106652 21880 consensus_queue.cc:1059] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:42.107038 13988 leader_election.cc:304] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:42.107769 21912 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:42.108626 21880 consensus_queue.cc:1059] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:42.109295 13988 leader_election.cc:304] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:42.109709 21880 raft_consensus.cc:3055] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:42.110113 21880 raft_consensus.cc:740] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:42.110293 21906 raft_consensus.cc:2749] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:42.111099 21906 raft_consensus.cc:3055] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:42.111547 21906 raft_consensus.cc:740] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:42.111140 21880 consensus_queue.cc:260] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:42.112859 21880 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:42.112474 21906 consensus_queue.cc:260] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:42.114251 21901 raft_consensus.cc:2749] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:42.117177 21906 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:42.127084 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.132308 13764 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 16 to 17, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 17 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:42.131099 13989 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:42.134737 21750 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:42.135516 13987 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:42.146888 21750 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:42.147456 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.149916 13987 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.150846 21750 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:42.151273 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.154266 13987 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.155411 13987 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:42.155781 21881 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:42.156180 21901 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:42.157444 13987 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.150962 21880 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:42.158253 21906 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:42.161360 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.163089 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.164495 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.165747 13987 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:42.166203 21750 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:42.162176 21901 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:42.166021 21906 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:42.166605 21880 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:42.168090 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:42.169013 21908 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:42.169441 21908 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:42.169612 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:42.169838 21908 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:42.174661 13987 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.175639 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:42.174450 13915 consensus_peers.cc:597] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:42.176239 21785 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:42.178148 13987 leader_election.cc:304] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:42.178876 21901 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:42.179442 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.179795 21901 raft_consensus.cc:2749] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:42.180312 21881 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:42.176251 21880 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:42.177419 21906 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:42.183342 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:42.184242 14015 raft_consensus.cc:2393] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3779649036364744823fc035dac892c7 in current term 24: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:42.185926 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45103ab867574475a587979f8ea5fa4e" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:42.187620 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a43a85b399e492b8391cc93e618a1fa" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:42.189113 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92405e92560c4099974ff3e6a4564895" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:42.190349 13915 consensus_peers.cc:597] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:42.190768 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef734205364543b02ed4158216bf36" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:42.193806 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eeee4c6df564edcb3c96e2887208cc4" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:42.195333 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481a7f1a133247ebb81e1be603cbdd57" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:42.197132 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:42.197562 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dfe02d97ee6481298ba92b29baa32b7" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:42.197769 14024 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
I20260709 13:07:42.197868 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d6ce914003a44f2b5a485ae7962383a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:42.198918 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:42.199553 14015 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:42.199604 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a37812504ce4bdb94c7eafc67fa5aee" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:42.202409 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "693a6579cdaa4e2a977368c2c6d5267a" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:42.206635 14021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:42.207199 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:42.215301 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17624553356541c69c1644ca409df1e9" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:42.190948 13915 consensus_peers.cc:597] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:42.216387 14524 leader_election.cc:336] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.216909 14524 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:42.215849 14019 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
W20260709 13:07:42.216619 13915 consensus_peers.cc:597] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:42.222261 13915 leader_election.cc:336] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.222630 14021 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 18.
W20260709 13:07:42.223104 13915 leader_election.cc:336] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.222209 13944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3730fc366f4349dea87ae85e1aed546c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:42.225512 13944 raft_consensus.cc:3060] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:07:42.225605 13915 leader_election.cc:336] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.227578 13915 leader_election.cc:304] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:42.228492 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "765348f2471a493d92e998132c0c746b" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:42.230593 14524 leader_election.cc:336] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.227020 21785 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:42.234781 13915 consensus_peers.cc:597] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:42.236851 21919 raft_consensus.cc:2749] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:42.241804 13989 consensus_peers.cc:597] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:42.231727 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:42.235436 13915 leader_election.cc:336] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:42.251396 14524 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.252058 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:42.253167 21785 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:42.259341 13944 raft_consensus.cc:2468] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 24.
W20260709 13:07:42.261384 13915 consensus_peers.cc:597] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:42.262918 21785 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:42.264370 21906 raft_consensus.cc:493] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dd265d61fc141a48ca9d3d5c8f52921)
I20260709 13:07:42.269546 14022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:42.279505 14022 raft_consensus.cc:2393] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 18: Already voted for candidate 3779649036364744823fc035dac892c7 in this term.
I20260709 13:07:42.279174 21906 raft_consensus.cc:515] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:42.281917 21906 leader_election.cc:290] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:42.270573 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8d34f260fb43229460064c72d795e8" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:42.273097 14524 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.273474 13949 raft_consensus.cc:1275] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 13:07:42.282501 13915 leader_election.cc:336] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.283313 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:42.284559 21785 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:07:42.283735 13915 leader_election.cc:336] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.286408 13915 leader_election.cc:304] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:42.281642 21908 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:42.287588 21919 raft_consensus.cc:2749] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:42.290778 21908 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:42.292838 13943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7"
I20260709 13:07:42.313710 13987 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: e1649107c781428692458f10597c5742
I20260709 13:07:42.314725 14018 raft_consensus.cc:2468] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 25.
I20260709 13:07:42.314903 21901 raft_consensus.cc:2804] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:42.315570 21901 raft_consensus.cc:697] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
W20260709 13:07:42.288218 14524 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.316548 21901 consensus_queue.cc:237] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:42.319581 13989 consensus_peers.cc:597] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:42.324795 21881 consensus_queue.cc:1048] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:42.317727 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:42.326746 21908 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:42.330375 14524 consensus_peers.cc:597] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:42.334017 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 46167.6us
I20260709 13:07:42.347625 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:42.366004 21908 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:07:42.394738 13989 leader_election.cc:336] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.401381 14023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:42.433467 13915 leader_election.cc:304] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:42.439312 21919 raft_consensus.cc:2749] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:42.448431 14553 raft_consensus.cc:1240] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Rejecting Update request from peer 3779649036364744823fc035dac892c7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:42.450348 21903 consensus_queue.cc:1059] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:42.451452 21903 raft_consensus.cc:3055] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:42.451818 21903 raft_consensus.cc:740] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:42.452930 21903 consensus_queue.cc:260] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:42.454535 13989 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:42.455123 13989 consensus_peers.cc:597] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:42.456041 21903 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:42.458904 13989 consensus_peers.cc:597] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:42.445331 13764 catalog_manager.cc:5697] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 22 to 24, leader changed from e1649107c781428692458f10597c5742 (127.12.109.65) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 24 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:42.483836 14524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 136301us
I20260709 13:07:42.486944 14524 leader_election.cc:304] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:42.494608 13915 leader_election.cc:304] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:42.523531 13915 leader_election.cc:304] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:42.523766 21905 raft_consensus.cc:2804] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:42.524418 21905 raft_consensus.cc:697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:42.524823 13915 leader_election.cc:304] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:42.516832 13989 consensus_peers.cc:597] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:42.526095 13915 leader_election.cc:304] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:42.526501 21853 raft_consensus.cc:2749] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:42.525471 21905 consensus_queue.cc:237] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:42.531009 21919 raft_consensus.cc:2749] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:42.530079 21911 raft_consensus.cc:2749] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:42.530825 13915 leader_election.cc:304] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:42.537817 21919 raft_consensus.cc:2749] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:42.523556 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
W20260709 13:07:42.532462 12725 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 13896)
I20260709 13:07:42.526680 21944 raft_consensus.cc:2749] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:42.550588 21908 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:42.552137 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82051b9bd85845d6b73773b21ccc6d6c" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:42.564283 13988 leader_election.cc:304] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:42.566243 21868 raft_consensus.cc:2749] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:42.588613 14553 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:42.590409 21853 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:07:42.607446 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:46599, user_credentials={real_user=slave}} blocked reactor thread for 65000.3us
W20260709 13:07:42.606042 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:42.612676 13915 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 4dd265d61fc141a48ca9d3d5c8f52921
W20260709 13:07:42.620594 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 121054us
I20260709 13:07:42.620555 21919 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:42.624030 21919 raft_consensus.cc:697] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
W20260709 13:07:42.624797 13915 leader_election.cc:336] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.625340 13915 leader_election.cc:304] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:42.625083 21919 consensus_queue.cc:237] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:42.628131 21946 raft_consensus.cc:2749] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:42.632332 13915 leader_election.cc:336] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.632927 13915 leader_election.cc:304] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:42.636157 21946 raft_consensus.cc:2749] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:42.637221 13915 leader_election.cc:336] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.637737 13915 leader_election.cc:304] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:42.650975 13915 leader_election.cc:336] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.651686 13915 leader_election.cc:304] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:42.654024 13915 leader_election.cc:336] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.655241 13915 leader_election.cc:304] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: 4dd265d61fc141a48ca9d3d5c8f52921, e1649107c781428692458f10597c5742
I20260709 13:07:42.657565 21903 raft_consensus.cc:2749] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:07:42.659691 13915 leader_election.cc:336] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.660120 13915 leader_election.cc:304] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:42.660995 13915 leader_election.cc:336] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.663419 21946 raft_consensus.cc:2749] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:42.654747 21853 raft_consensus.cc:2749] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:42.655151 21911 raft_consensus.cc:2749] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:42.664577 13764 catalog_manager.cc:5697] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 24 to 25, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 25 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:42.668371 14524 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:42.661388 13915 leader_election.cc:304] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
W20260709 13:07:42.692029 13915 leader_election.cc:336] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:42.692546 13915 leader_election.cc:304] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3779649036364744823fc035dac892c7; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:42.697244 21853 raft_consensus.cc:2749] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:42.730353 21919 raft_consensus.cc:2749] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:42.730243 13766 catalog_manager.cc:5697] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 17 to 18, leader changed from d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67) to 3779649036364744823fc035dac892c7 (127.12.109.66). New cstate: current_term: 18 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:42.744504 13989 consensus_peers.cc:597] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:42.748464 13989 consensus_peers.cc:597] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:42.749099 13989 consensus_peers.cc:597] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:42.755800 14022 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:42.794719 13943 raft_consensus.cc:1275] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:07:42.796748 21750 consensus_queue.cc:1048] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:42.799309 21853 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:07:42.877537 14524 consensus_peers.cc:597] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:42.883632 13915 consensus_peers.cc:597] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:42.889016 13989 consensus_peers.cc:597] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:42.923741 13915 consensus_peers.cc:597] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:43.002763 14023 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:07:43.009766 21905 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:07:43.019259 13943 raft_consensus.cc:3060] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:43.076395 13943 raft_consensus.cc:1275] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:07:43.078892 21945 consensus_queue.cc:1048] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:07:43.200289 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:43.255065 21954 log.cc:927] Time spent T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:07:43.264199 14553 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:07:43.266147 21903 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:43.309075 14024 raft_consensus.cc:1275] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
W20260709 13:07:43.369107 13915 consensus_peers.cc:597] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:43.369908 13915 consensus_peers.cc:597] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:43.406203 21903 consensus_queue.cc:1048] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:07:43.430517 13915 consensus_peers.cc:597] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:43.457001 13915 consensus_peers.cc:597] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:43.543133 12725 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 13896)
W20260709 13:07:43.548143 13915 consensus_peers.cc:597] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:43.552769 13915 consensus_peers.cc:597] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:43.593286 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:43.657738 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:43.666363 13915 consensus_peers.cc:597] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:43.668002 21911 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.668749 21911 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:43.672761 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:43.673435 14553 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 16.
I20260709 13:07:43.675037 13914 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:43.675951 21958 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:43.676371 21958 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.676704 21958 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:43.683626 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:43.712181 21881 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:43.712823 21881 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:43.715124 21881 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:43.718865 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:43.720181 14553 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 16: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
W20260709 13:07:43.722273 13989 consensus_peers.cc:597] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:43.737861 13941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3779649036364744823fc035dac892c7" is_pre_election: true
I20260709 13:07:43.746212 21911 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:43.781350 21958 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:43.782846 13941 raft_consensus.cc:2368] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17.
I20260709 13:07:43.785447 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:43.786176 14553 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:43.793010 13915 consensus_peers.cc:597] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:43.795830 13987 leader_election.cc:400] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17.
I20260709 13:07:43.796284 13987 leader_election.cc:403] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:43.796918 21868 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:07:43.806417 21868 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953) with higher term. Message: Invalid argument: T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 for earlier term 16. Current term is 17.
W20260709 13:07:43.817655 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:43.833719 21958 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Requested vote from peers d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:43.852541 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "3779649036364744823fc035dac892c7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:43.860888 14018 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:43.863204 14553 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3779649036364744823fc035dac892c7 in term 17.
I20260709 13:07:43.864579 13914 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 
I20260709 13:07:43.865453 21919 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:43.865948 21919 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Becoming Leader. State: Replica: 3779649036364744823fc035dac892c7, State: Running, Role: LEADER
I20260709 13:07:43.870954 21919 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:43.925756 13765 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3779649036364744823fc035dac892c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:44.008677 14524 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:44.016997 13989 consensus_peers.cc:597] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.128945 13989 consensus_peers.cc:597] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.156325 13989 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:44.164194 13989 consensus_peers.cc:597] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.178299 13759 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 72332.9us
W20260709 13:07:44.181631 13759 catalog_manager.cc:4755] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6b098cc2d51e471cb2c9c1b17eebcb57: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:44.195428 13989 consensus_peers.cc:597] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:44.238346 13915 consensus_peers.cc:597] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:44.239166 13915 consensus_peers.cc:597] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.264850 13915 consensus_peers.cc:597] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:44.396052 13915 consensus_peers.cc:597] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.449916 13915 consensus_peers.cc:597] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.461567 13989 consensus_peers.cc:597] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:44.480340 14553 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:44.482937 21911 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:07:44.483884 14524 consensus_peers.cc:597] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.486123 13989 consensus_peers.cc:597] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:44.502285 14524 consensus_peers.cc:597] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:44.504324 14024 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 3779649036364744823fc035dac892c7: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:44.510564 21962 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:07:44.519012 13989 consensus_peers.cc:597] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.528692 13915 consensus_peers.cc:597] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:44.542987 13989 consensus_peers.cc:597] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.543701 12725 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 13896)
W20260709 13:07:44.595620 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:44.614171 13915 consensus_peers.cc:597] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:44.647776 13915 consensus_peers.cc:597] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.677701 13989 consensus_peers.cc:597] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:44.763588 13989 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:44.797569 13989 consensus_peers.cc:597] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:44.847152 13989 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:44.869858 13989 consensus_peers.cc:597] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.875577 13915 consensus_peers.cc:597] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:44.889348 13915 consensus_peers.cc:597] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:44.898326 13989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111) [suppressed 395 similar messages]
W20260709 13:07:44.968607 13989 consensus_peers.cc:597] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.970460 14524 consensus_peers.cc:597] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:45.086514 13989 consensus_peers.cc:597] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:45.128665 13915 consensus_peers.cc:597] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:45.207208 13989 consensus_peers.cc:597] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:45.333360 13989 consensus_peers.cc:597] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:45.391964 13915 consensus_peers.cc:597] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:45.465161 13989 consensus_peers.cc:597] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:45.546952 12725 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 13896)
W20260709 13:07:45.566358 13989 consensus_peers.cc:597] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:45.593230 14524 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:45.611873 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 45574.5us
W20260709 13:07:45.704540 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:45.873742 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 67800.4us
W20260709 13:07:45.917477 13915 consensus_peers.cc:597] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:45.951056 13915 consensus_peers.cc:597] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:45.984642 13915 consensus_peers.cc:597] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:46.007004 13915 consensus_peers.cc:597] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:46.015057 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:46.102371 13915 consensus_peers.cc:597] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:46.119607 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:46.122738 13989 consensus_peers.cc:597] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:46.307308 13915 consensus_peers.cc:597] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:46.317844 13915 consensus_peers.cc:597] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:46.316967 13989 consensus_peers.cc:597] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:46.427644 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:46.458945 13915 consensus_peers.cc:597] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:46.496079 13915 consensus_peers.cc:597] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:46.521037 13989 consensus_peers.cc:597] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:46.527546 13915 consensus_peers.cc:597] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:46.547575 12725 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 13896)
W20260709 13:07:46.611059 14524 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:46.758731 13989 consensus_peers.cc:597] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:46.759532 13762 maintenance_manager.cc:419] P 91c63734839347c4aee340a59c8e3221: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033394
W20260709 13:07:46.772230 13989 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:46.810115 13989 consensus_peers.cc:597] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:46.823716 13761 maintenance_manager.cc:643] P 91c63734839347c4aee340a59c8e3221: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.061s	user 0.037s	sys 0.004s Metrics: {"bytes_written":7549,"delete_count":0,"lbm_write_time_us":1124,"lbm_writes_lt_1ms":7,"reinsert_count":0,"thread_start_us":15597,"threads_started":1,"update_count":313,"wal-append.queue_time_us":15557}
I20260709 13:07:46.826546 13762 maintenance_manager.cc:419] P 91c63734839347c4aee340a59c8e3221: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.357205
W20260709 13:07:46.883457 13915 consensus_peers.cc:597] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:46.939145 13915 consensus_peers.cc:597] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:46.942405 13915 consensus_peers.cc:597] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:46.971664 13989 consensus_peers.cc:597] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:46.987591 14524 consensus_peers.cc:597] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.005301 13915 consensus_peers.cc:597] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.016705 13989 consensus_peers.cc:597] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.019441 13989 consensus_peers.cc:597] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.096909 14524 consensus_peers.cc:597] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.105109 13989 consensus_peers.cc:597] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:47.117823 13989 consensus_peers.cc:597] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.148561 13989 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:47.177896 13915 consensus_peers.cc:597] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:47.206632 13989 consensus_peers.cc:597] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.260373 13915 consensus_peers.cc:597] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:47.282605 13761 maintenance_manager.cc:643] P 91c63734839347c4aee340a59c8e3221: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.454s	user 0.116s	sys 0.016s Metrics: {"cfile_cache_miss":10,"cfile_cache_miss_bytes":28744,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":53896,"dirs.run_cpu_time_us":592,"dirs.run_wall_time_us":2378,"lbm_read_time_us":15377,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":24,"lbm_write_time_us":1884,"lbm_writes_lt_1ms":12,"peak_mem_usage":315485,"reinsert_count":0,"thread_start_us":61033,"threads_started":2,"update_count":322}
I20260709 13:07:47.288638 13762 maintenance_manager.cc:419] P 91c63734839347c4aee340a59c8e3221: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:07:47.289500 13761 maintenance_manager.cc:643] P 91c63734839347c4aee340a59c8e3221: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
W20260709 13:07:47.293081 13989 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:47.295756 13989 consensus_peers.cc:597] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:47.306301 14524 consensus_peers.cc:597] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.363114 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:47.429630 13915 consensus_peers.cc:597] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:47.468197 13989 consensus_peers.cc:597] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:47.548441 12725 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 13896)
W20260709 13:07:47.638212 13989 consensus_peers.cc:597] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.651752 13989 consensus_peers.cc:597] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.675585 13915 consensus_peers.cc:597] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:47.711903 13989 consensus_peers.cc:597] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.842613 13915 consensus_peers.cc:597] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:47.848697 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 152879us
W20260709 13:07:47.898707 13915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 56193.6us
W20260709 13:07:47.967583 13989 consensus_peers.cc:597] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:48.015117 13915 consensus_peers.cc:597] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:48.219167 13989 consensus_peers.cc:597] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:48.327775 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:48.367738 13915 consensus_peers.cc:597] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:48.412642 13915 consensus_peers.cc:597] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:48.458767 14524 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:48.491458 13989 consensus_peers.cc:597] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:48.538007 13915 consensus_peers.cc:597] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:48.549010 12725 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 13896)
W20260709 13:07:48.563812 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:48.623376 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:48.735013 13915 consensus_peers.cc:597] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:48.802690 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:48.809571 13915 consensus_peers.cc:597] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:48.832259 13915 consensus_peers.cc:597] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:48.858153 13915 consensus_peers.cc:597] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:48.882766 13915 consensus_peers.cc:597] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:49.088142 13989 consensus_peers.cc:597] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:49.214454 13989 consensus_peers.cc:597] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:49.215651 13915 consensus_peers.cc:597] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:49.230786 13989 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:49.344729 13915 consensus_peers.cc:597] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:49.366602 13989 consensus_peers.cc:597] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:49.413170 13915 consensus_peers.cc:597] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:49.425060 13915 consensus_peers.cc:597] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:49.445742 13989 consensus_peers.cc:597] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:49.479931 13989 consensus_peers.cc:597] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:49.523962 13915 consensus_peers.cc:597] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:49.524868 14524 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:49.549623 12725 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 13896)
W20260709 13:07:49.547509 13915 consensus_peers.cc:597] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:49.568819 13989 consensus_peers.cc:597] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:49.578953 13989 consensus_peers.cc:597] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:49.588547 13989 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:49.602896 13915 consensus_peers.cc:597] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:49.605283 13989 consensus_peers.cc:597] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:49.640722 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:49.693149 14524 consensus_peers.cc:597] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:49.710111 13989 consensus_peers.cc:597] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:49.724300 13989 consensus_peers.cc:597] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:49.756510 14524 consensus_peers.cc:597] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:49.825788 13989 consensus_peers.cc:597] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:49.898860 13989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111) [suppressed 493 similar messages]
W20260709 13:07:49.934187 13989 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:49.941941 13989 consensus_peers.cc:597] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:49.984053 13989 consensus_peers.cc:597] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:50.033236 13915 consensus_peers.cc:597] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:50.064903 13915 consensus_peers.cc:597] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:50.093447 13989 consensus_peers.cc:597] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:50.138626 14524 consensus_peers.cc:597] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:50.173483 13915 consensus_peers.cc:597] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:50.296226 13915 consensus_peers.cc:597] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:50.321195 13989 consensus_peers.cc:597] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:50.349283 13915 consensus_peers.cc:597] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:50.461136 13989 consensus_peers.cc:597] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:50.550117 12725 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 13896)
W20260709 13:07:50.679988 13915 consensus_peers.cc:597] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:50.734015 13989 consensus_peers.cc:597] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:50.874518 13915 consensus_peers.cc:597] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:51.006740 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:51.014412 13915 consensus_peers.cc:597] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:51.064199 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:51.105706 13989 consensus_peers.cc:597] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:51.121134 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:51.133261 13915 consensus_peers.cc:597] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:51.147653 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:51.270452 13915 consensus_peers.cc:597] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:51.321447 14524 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:51.354590 13915 consensus_peers.cc:597] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:51.464021 13989 consensus_peers.cc:597] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:51.467310 13989 consensus_peers.cc:597] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:51.550948 12725 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 13896)
W20260709 13:07:51.589476 13915 consensus_peers.cc:597] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:51.590433 13915 consensus_peers.cc:597] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:51.607174 13989 consensus_peers.cc:597] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:51.709916 13915 consensus_peers.cc:597] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:51.774724 13915 consensus_peers.cc:597] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:51.811647 13915 consensus_peers.cc:597] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:51.870393 13915 consensus_peers.cc:597] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:51.883644 13915 consensus_peers.cc:597] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:51.970405 13989 consensus_peers.cc:597] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:51.973519 13989 consensus_peers.cc:597] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:52.003698 13989 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:52.048739 13989 consensus_peers.cc:597] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:52.090579 14524 consensus_peers.cc:597] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:52.091778 13989 consensus_peers.cc:597] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:52.115422 13989 consensus_peers.cc:597] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:52.119230 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:52.142582 13989 consensus_peers.cc:597] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:52.148088 14524 consensus_peers.cc:597] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:52.165139 13915 consensus_peers.cc:597] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:52.243250 13989 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:52.250942 13915 consensus_peers.cc:597] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:52.257135 14524 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:52.286002 13915 consensus_peers.cc:597] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:52.299590 13989 consensus_peers.cc:597] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:52.360697 13989 consensus_peers.cc:597] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:52.456420 13989 consensus_peers.cc:597] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:52.456904 13915 consensus_peers.cc:597] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:52.489701 13989 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:52.523703 13915 consensus_peers.cc:597] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:52.554219 13989 consensus_peers.cc:597] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:52.595811 14524 consensus_peers.cc:597] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:52.638787 13915 consensus_peers.cc:597] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:52.691856 13989 consensus_peers.cc:597] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:52.692621 13915 consensus_peers.cc:597] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:52.735000 13989 consensus_peers.cc:597] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:53.242247 13915 consensus_peers.cc:597] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:53.343890 13915 consensus_peers.cc:597] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:53.354216 13989 consensus_peers.cc:597] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:53.378914 13915 consensus_peers.cc:597] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:53.474500 13915 consensus_peers.cc:597] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:53.479151 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:07:53.483213 13989 consensus_peers.cc:597] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:53.540426 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:07:53.557325 13914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:42851, user_credentials={real_user=slave}} blocked reactor thread for 126886us
W20260709 13:07:53.618598 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:07:53.626650 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:07:53.627319 13989 consensus_peers.cc:597] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:53.675724 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 64053.8us
W20260709 13:07:53.678470 13989 consensus_peers.cc:597] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:53.749558 13915 consensus_peers.cc:597] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:53.831920 13915 consensus_peers.cc:597] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:53.966110 13915 consensus_peers.cc:597] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:53.995874 13915 consensus_peers.cc:597] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:54.031996 13915 consensus_peers.cc:597] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:54.173954 14524 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:54.194957 13989 consensus_peers.cc:597] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:54.236837 13915 consensus_peers.cc:597] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:54.264611 13989 consensus_peers.cc:597] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:54.339628 14524 consensus_peers.cc:597] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:54.342351 13915 consensus_peers.cc:597] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:54.483566 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:54.515786 14524 consensus_peers.cc:597] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:54.538355 13915 consensus_peers.cc:597] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:54.581954 13989 consensus_peers.cc:597] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:54.587881 13989 consensus_peers.cc:597] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:54.662823 12725 tablet_server.cc:196] TabletServer@127.12.109.65:0 shutdown complete.
W20260709 13:07:54.698833 13989 consensus_peers.cc:597] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:54.699584 13989 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:54.700409 13989 consensus_peers.cc:597] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:54.700969 13989 consensus_peers.cc:597] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:54.739527 13989 consensus_peers.cc:597] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:54.742354 13989 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:54.761270 13989 consensus_peers.cc:597] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:54.763752 13987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 46457.5us
W20260709 13:07:54.790494 14524 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:07:54.793195 13915 consensus_peers.cc:597] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:54.825273 13989 consensus_peers.cc:597] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:54.874543 13915 consensus_peers.cc:597] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:54.886847 13915 consensus_peers.cc:597] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:54.903822 13915 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111) [suppressed 506 similar messages]
W20260709 13:07:54.924719 13915 consensus_peers.cc:597] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:54.929144 13989 consensus_peers.cc:597] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:54.995752 13915 consensus_peers.cc:597] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:55.003131 12725 tablet_server.cc:179] TabletServer@127.12.109.66:0 shutting down...
W20260709 13:07:55.015451 14522 consensus_peers.cc:597] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.064780 13989 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:55.072463 13987 consensus_peers.cc:597] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.109148 13915 consensus_peers.cc:597] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:55.116628 13987 consensus_peers.cc:597] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.130245 13987 consensus_peers.cc:597] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.136559 13987 consensus_peers.cc:597] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.170877 14522 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.174543 13987 consensus_peers.cc:597] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.186380 13987 consensus_peers.cc:597] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.200623 13989 consensus_peers.cc:597] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:55.204454 14522 consensus_peers.cc:597] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.208063 13987 consensus_peers.cc:597] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.215338 13915 consensus_peers.cc:597] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:55.227133 14522 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.228871 14522 consensus_peers.cc:597] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.257697 14522 consensus_peers.cc:597] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.258416 13987 consensus_peers.cc:597] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.261749 13987 consensus_peers.cc:597] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.269809 14522 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.278724 13987 consensus_peers.cc:597] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.279491 13987 consensus_peers.cc:597] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.287385 13987 consensus_peers.cc:597] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.300987 13989 consensus_peers.cc:597] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:55.303364 13987 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.307421 14522 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.323475 14524 consensus_peers.cc:597] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:55.327289 13987 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.331202 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:55.332435 12725 tablet_replica.cc:333] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.333650 12725 raft_consensus.cc:2243] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.335253 12725 raft_consensus.cc:2272] T 56605211bd584b8889b54049e9f6cb4b P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.338820 12725 tablet_replica.cc:333] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.339851 12725 raft_consensus.cc:2243] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.341562 12725 raft_consensus.cc:2272] T 728de4a891d847028775aa5c35ba53e1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.354193 12725 tablet_replica.cc:333] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.360750 12725 raft_consensus.cc:2243] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.365092 12725 raft_consensus.cc:2272] T 617d39d9397c4724be821e5efacbde17 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:55.369505 14522 consensus_peers.cc:597] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.379202 14522 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.377084 12725 tablet_replica.cc:333] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.385141 12725 raft_consensus.cc:2243] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.386574 12725 raft_consensus.cc:2272] T 10b028ec30974d06bd9da921c268a641 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.392985 12725 tablet_replica.cc:333] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.393994 12725 raft_consensus.cc:2243] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.395490 12725 raft_consensus.cc:2272] T 567aed48a57a41e5820f0b937f2de9be P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.399350 12725 tablet_replica.cc:333] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.400269 12725 raft_consensus.cc:2243] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.401661 12725 raft_consensus.cc:2272] T d4033a61013848c69bbd0e79b27a8be3 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.406188 12725 tablet_replica.cc:333] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.407166 12725 raft_consensus.cc:2243] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.408141 12725 raft_consensus.cc:2272] T 343e333726b74ed98b7e80869ce28ad1 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.411146 12725 tablet_replica.cc:333] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.412011 12725 raft_consensus.cc:2243] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.413323 12725 raft_consensus.cc:2272] T 14bba4e2cef04a418c6f660869c7e6e5 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:55.416455 13987 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.419741 14522 consensus_peers.cc:597] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.421573 12725 tablet_replica.cc:333] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.422693 12725 raft_consensus.cc:2243] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.424011 12725 raft_consensus.cc:2272] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.427618 12725 tablet_replica.cc:333] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.428589 12725 raft_consensus.cc:2243] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.429241 12725 raft_consensus.cc:2272] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.432485 12725 tablet_replica.cc:333] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.433452 12725 raft_consensus.cc:2243] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.434249 12725 raft_consensus.cc:2272] T 05adb1eba4fe476d8504a36537b738db P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.441355 12725 tablet_replica.cc:333] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.442337 12725 raft_consensus.cc:2243] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.443229 12725 raft_consensus.cc:2272] T 4eeee4c6df564edcb3c96e2887208cc4 P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.446156 12725 tablet_replica.cc:333] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.447081 12725 raft_consensus.cc:2243] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.455016 12725 raft_consensus.cc:2272] T f2e6672bab624b20b38e363591856fc0 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.458257 12725 tablet_replica.cc:333] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.459133 12725 raft_consensus.cc:2243] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.461066 12725 raft_consensus.cc:2272] T f50263f80db844239948e5e67ef3b987 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.464395 12725 tablet_replica.cc:333] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.465245 12725 raft_consensus.cc:2243] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.466653 12725 raft_consensus.cc:2272] T 0768fb01770349dfb259ab0db91394ba P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.469602 12725 tablet_replica.cc:333] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.470629 12725 raft_consensus.cc:2243] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.471853 12725 raft_consensus.cc:2272] T 9a37812504ce4bdb94c7eafc67fa5aee P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.474911 12725 tablet_replica.cc:333] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.476003 12725 raft_consensus.cc:2243] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.477092 12725 raft_consensus.cc:2272] T 0e8d34f260fb43229460064c72d795e8 P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.480417 12725 tablet_replica.cc:333] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.481313 12725 raft_consensus.cc:2243] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.482877 12725 raft_consensus.cc:2272] T 9bfd3f44e83e4908b229bb57863e4080 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.485754 12725 tablet_replica.cc:333] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.486678 12725 raft_consensus.cc:2243] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.487387 12725 raft_consensus.cc:2272] T 7f4c2ba739c04f16a8ed8c28f57fe695 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.490711 12725 tablet_replica.cc:333] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.491569 12725 raft_consensus.cc:2243] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.492404 12725 raft_consensus.cc:2272] T 693a6579cdaa4e2a977368c2c6d5267a P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.495815 12725 tablet_replica.cc:333] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.496701 12725 raft_consensus.cc:2243] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.497440 12725 raft_consensus.cc:2272] T 7991c821f3da4b7abc0f6bb6baf8fe99 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.505103 12725 tablet_replica.cc:333] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.506035 12725 raft_consensus.cc:2243] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.507369 12725 raft_consensus.cc:2272] T 34d11c517b334dd5abf16448009bdf2d P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.511272 12725 tablet_replica.cc:333] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.512141 12725 raft_consensus.cc:2243] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:07:55.512923 13987 consensus_peers.cc:597] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.513386 12725 raft_consensus.cc:2272] T a168d82765704a87b2c60aeb7b23e8b0 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.517132 12725 tablet_replica.cc:333] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.518159 12725 raft_consensus.cc:2243] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.519030 12725 raft_consensus.cc:2272] T 45103ab867574475a587979f8ea5fa4e P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.521914 12725 tablet_replica.cc:333] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.523272 12725 raft_consensus.cc:2243] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.524559 12725 raft_consensus.cc:2272] T 99b9907790614d49bc3adf40424fc1b6 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.527534 12725 tablet_replica.cc:333] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.528358 12725 raft_consensus.cc:2243] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.529280 12725 raft_consensus.cc:2272] T 4a43a85b399e492b8391cc93e618a1fa P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.532014 12725 tablet_replica.cc:333] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.532825 12725 raft_consensus.cc:2243] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.534006 12725 raft_consensus.cc:2272] T 10a0b27264ce49379f2aba40cdbc2b0f P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.536917 12725 tablet_replica.cc:333] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.537797 12725 raft_consensus.cc:2243] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.538731 12725 raft_consensus.cc:2272] T 92405e92560c4099974ff3e6a4564895 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.541540 12725 tablet_replica.cc:333] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.542407 12725 raft_consensus.cc:2243] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.543061 12725 raft_consensus.cc:2272] T 3730fc366f4349dea87ae85e1aed546c P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:55.545943 13989 consensus_peers.cc:597] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:55.547014 12725 tablet_replica.cc:333] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.547940 12725 raft_consensus.cc:2243] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.550638 12725 raft_consensus.cc:2272] T f9158487769746e2a7f4f283dda5abf3 P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:55.553974 13987 consensus_peers.cc:597] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.554922 12725 tablet_replica.cc:333] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.555859 12725 raft_consensus.cc:2243] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.563621 12725 raft_consensus.cc:2272] T dcd5a97e61554ed698b00fdf13294a7b P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.566788 12725 tablet_replica.cc:333] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.567664 12725 raft_consensus.cc:2243] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.568342 12725 raft_consensus.cc:2272] T 765348f2471a493d92e998132c0c746b P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.571026 12725 tablet_replica.cc:333] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.571663 12725 raft_consensus.cc:2243] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.572356 12725 raft_consensus.cc:2272] T 435cf49d11154354987bf559d46d3c3a P 3779649036364744823fc035dac892c7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.576534 12725 tablet_replica.cc:333] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.577726 12725 raft_consensus.cc:2243] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.578405 12725 raft_consensus.cc:2272] T 8dfe02d97ee6481298ba92b29baa32b7 P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.586858 12725 tablet_replica.cc:333] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.587742 12725 raft_consensus.cc:2243] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.588943 12725 raft_consensus.cc:2272] T 17624553356541c69c1644ca409df1e9 P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.598214 12725 tablet_replica.cc:333] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.599445 12725 raft_consensus.cc:2243] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.609059 12725 raft_consensus.cc:2272] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 3779649036364744823fc035dac892c7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.612753 12725 tablet_replica.cc:333] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.614280 12725 raft_consensus.cc:2243] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.614953 12725 raft_consensus.cc:2272] T 5d6ce914003a44f2b5a485ae7962383a P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.618199 12725 tablet_replica.cc:333] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.619055 12725 raft_consensus.cc:2243] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.621205 12725 raft_consensus.cc:2272] T 403f6d16105d4532ae1de441c32f632a P 3779649036364744823fc035dac892c7 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:55.623148 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08002a6e40 after lost signal to thread 12728
W20260709 13:07:55.624567 14045 debug-util.cc:398] Leaking SignalData structure 0x7b0800376160 after lost signal to thread 13807
W20260709 13:07:55.626350 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08002e2780 after lost signal to thread 13974
W20260709 13:07:55.627840 14045 debug-util.cc:398] Leaking SignalData structure 0x7b0800238c80 after lost signal to thread 14048
W20260709 13:07:55.629360 14045 debug-util.cc:398] Leaking SignalData structure 0x7b0800141c80 after lost signal to thread 14585
I20260709 13:07:55.669039 12725 tablet_replica.cc:333] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.670342 12725 raft_consensus.cc:2243] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.673367 12725 raft_consensus.cc:2272] T be9dcc1a27af4a9a91a283a728d4132d P 3779649036364744823fc035dac892c7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.686358 12725 tablet_replica.cc:333] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.687413 12725 raft_consensus.cc:2243] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.688167 12725 raft_consensus.cc:2272] T a3d53d5ea0e8450e97d9e321677e63d5 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.697666 12725 tablet_replica.cc:333] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.698750 12725 raft_consensus.cc:2243] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.700415 12725 raft_consensus.cc:2272] T 4e20f6d8df1f4522b10fd2dc803e0e75 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.704147 12725 tablet_replica.cc:333] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.715631 12725 raft_consensus.cc:2243] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.721067 12725 raft_consensus.cc:2272] T 1ec9c4d3cc3a47bd838d18fe7da650fc P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.728400 12725 tablet_replica.cc:333] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.729350 12725 raft_consensus.cc:2243] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.735056 12725 raft_consensus.cc:2272] T 208a6eaac64647a59ee173ef6726ae8b P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.738983 12725 tablet_replica.cc:333] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.741786 12725 raft_consensus.cc:2243] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.757529 12725 raft_consensus.cc:2272] T ef6c4cc997504124a8bb7fa9c7f10e34 P 3779649036364744823fc035dac892c7 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:55.776289 13915 consensus_peers.cc:597] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:07:55.803803 13989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:43607, user_credentials={real_user=slave}} blocked reactor thread for 63472.4us
I20260709 13:07:55.826915 12725 tablet_replica.cc:333] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.830839 12725 raft_consensus.cc:2243] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.833187 12725 raft_consensus.cc:2272] T efeae9bc4fd84c4487b09362929c2fa6 P 3779649036364744823fc035dac892c7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.836139 12725 tablet_replica.cc:333] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.836920 12725 raft_consensus.cc:2243] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.838531 12725 raft_consensus.cc:2272] T 9f8599b3d11641199dc0043a592857fb P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.841631 12725 tablet_replica.cc:333] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.842483 12725 raft_consensus.cc:2243] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.843739 12725 raft_consensus.cc:2272] T 545fc4aeb71d491985e296a0721d905d P 3779649036364744823fc035dac892c7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.846796 12725 tablet_replica.cc:333] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.847681 12725 raft_consensus.cc:2243] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.848752 12725 raft_consensus.cc:2272] T fdef734205364543b02ed4158216bf36 P 3779649036364744823fc035dac892c7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.851547 12725 tablet_replica.cc:333] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.852411 12725 raft_consensus.cc:2243] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.857050 12725 raft_consensus.cc:2272] T 17bd99d4be2c4073bd867c161fa52482 P 3779649036364744823fc035dac892c7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.859956 12725 tablet_replica.cc:333] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.860728 12725 raft_consensus.cc:2243] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.862013 12725 raft_consensus.cc:2272] T 76071b78766a43e0b5097f615de4c569 P 3779649036364744823fc035dac892c7 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:55.876639 13989 consensus_peers.cc:597] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:55.888878 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:07:55.921303 12725 tablet_replica.cc:333] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.922273 12725 raft_consensus.cc:2243] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.922963 12725 raft_consensus.cc:2272] T 481a7f1a133247ebb81e1be603cbdd57 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.925748 12725 tablet_replica.cc:333] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7: stopping tablet replica
I20260709 13:07:55.926478 12725 raft_consensus.cc:2243] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.930817 12725 raft_consensus.cc:2272] T f5c93fb008bd4a4f8db94cdc59c65d93 P 3779649036364744823fc035dac892c7 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:56.015879 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:07:56.176081 13989 consensus_peers.cc:597] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:56.209997 13987 connection.cc:573] client connection to 127.12.109.66:43953 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:07:56.298367 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:07:56.326478 13989 consensus_peers.cc:597] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:56.339808 13989 consensus_peers.cc:597] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:56.340220 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:07:56.440817 22004 raft_consensus.cc:493] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.441522 22004 raft_consensus.cc:515] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.444295 22004 leader_election.cc:290] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:56.451668 13989 leader_election.cc:336] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:56.453627 13987 leader_election.cc:336] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.454119 13987 leader_election.cc:304] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.454881 22004 raft_consensus.cc:2749] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:56.503980 21999 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.504673 21999 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:56.510124 22004 raft_consensus.cc:493] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.510803 22004 raft_consensus.cc:515] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:56.521883 13989 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:56.523334 13987 leader_election.cc:336] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.523777 13987 leader_election.cc:304] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.524704 22020 raft_consensus.cc:2749] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:07:56.525496 14522 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.526368 22004 leader_election.cc:290] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:56.528345 21999 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:56.529685 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:56.531497 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:56.532647 21999 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:56.544970 22020 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.545614 22020 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:56.550534 22024 raft_consensus.cc:493] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.551054 22020 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:56.551134 22024 raft_consensus.cc:515] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.553573 22024 leader_election.cc:290] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:56.554407 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:56.555153 14553 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 19.
I20260709 13:07:56.556691 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
W20260709 13:07:56.561630 13989 leader_election.cc:336] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:56.563815 22020 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:56.564220 22020 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.564574 22020 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:56.565743 13987 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
W20260709 13:07:56.573172 13987 leader_election.cc:336] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.575806 13987 leader_election.cc:304] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.576992 22024 raft_consensus.cc:2749] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:56.605259 22020 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:56.608881 22020 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:56.612599 14558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "567aed48a57a41e5820f0b937f2de9be" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:56.613287 14558 raft_consensus.cc:3060] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:56.615936 14524 consensus_peers.cc:597] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:56.618642 13987 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.628612 14558 raft_consensus.cc:2468] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 20.
I20260709 13:07:56.631568 13988 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:56.632346 22004 raft_consensus.cc:2804] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:56.632830 22004 raft_consensus.cc:697] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:56.633834 22004 consensus_queue.cc:237] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:56.638207 13989 consensus_peers.cc:597] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:07:56.646662 22020 raft_consensus.cc:493] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.647315 22020 raft_consensus.cc:515] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:56.655174 13987 leader_election.cc:336] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.656064 22024 raft_consensus.cc:493] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.656579 22024 raft_consensus.cc:515] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.658740 22024 leader_election.cc:290] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:56.666157 13987 leader_election.cc:336] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.669871 13766 catalog_manager.cc:5697] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 19 to 20, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 20 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.674295 22020 leader_election.cc:290] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:56.676057 13989 leader_election.cc:336] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:56.676574 13989 leader_election.cc:304] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.677363 22020 raft_consensus.cc:2749] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:56.680603 13989 leader_election.cc:336] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:56.681162 13989 leader_election.cc:304] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.681931 22020 raft_consensus.cc:2749] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:56.692217 22020 raft_consensus.cc:493] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.692873 22020 raft_consensus.cc:515] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.695150 22020 leader_election.cc:290] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:56.699380 13989 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:56.705186 13989 consensus_peers.cc:597] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:56.706197 13987 leader_election.cc:336] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.707682 13987 leader_election.cc:304] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.709618 22020 raft_consensus.cc:2749] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:07:56.710788 14524 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:07:56.743774 22020 raft_consensus.cc:493] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.744437 22020 raft_consensus.cc:515] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.745052 22024 raft_consensus.cc:493] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.745638 22024 raft_consensus.cc:515] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.747474 22004 raft_consensus.cc:493] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.747895 22024 leader_election.cc:290] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:56.748044 22004 raft_consensus.cc:515] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.749214 22025 raft_consensus.cc:493] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:56.749313 13989 consensus_peers.cc:597] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:07:56.749841 22025 raft_consensus.cc:515] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.751315 22004 leader_election.cc:290] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:56.755278 13989 leader_election.cc:336] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:56.752225 22025 leader_election.cc:290] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:56.755369 22020 leader_election.cc:290] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:56.756291 13989 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:56.759969 13989 leader_election.cc:336] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:56.760545 13989 leader_election.cc:336] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:56.761034 13987 leader_election.cc:336] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.761524 13987 leader_election.cc:304] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:56.762758 13987 leader_election.cc:336] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.763239 22020 raft_consensus.cc:2749] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:56.764118 13987 leader_election.cc:304] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.764950 22020 raft_consensus.cc:2749] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:56.769425 13987 leader_election.cc:336] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.772339 22020 raft_consensus.cc:493] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.775841 22024 raft_consensus.cc:493] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.776603 22024 raft_consensus.cc:515] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.775975 22020 raft_consensus.cc:515] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.779175 22024 leader_election.cc:290] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:56.779687 22020 leader_election.cc:290] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:56.774755 13987 leader_election.cc:304] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.783495 22020 raft_consensus.cc:2749] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:56.783782 13987 leader_election.cc:336] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.784263 13987 leader_election.cc:304] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.784957 22020 raft_consensus.cc:2749] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:56.790257 13989 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:56.794572 22020 raft_consensus.cc:493] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
W20260709 13:07:56.794742 13989 leader_election.cc:336] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:56.795208 22020 raft_consensus.cc:515] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.797559 22020 leader_election.cc:290] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:56.799930 22004 raft_consensus.cc:493] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.800451 22004 raft_consensus.cc:515] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:56.806959 13989 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:56.807889 22004 leader_election.cc:290] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:56.808336 22024 raft_consensus.cc:493] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.811919 22024 raft_consensus.cc:515] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
W20260709 13:07:56.812731 14524 consensus_peers.cc:597] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:07:56.813928 22024 leader_election.cc:290] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:56.815011 13989 leader_election.cc:336] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:56.817656 13989 leader_election.cc:336] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:56.819917 13987 leader_election.cc:336] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.821710 13987 leader_election.cc:304] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.822580 22024 raft_consensus.cc:2749] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:56.824527 13987 leader_election.cc:336] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.824985 13987 leader_election.cc:304] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.825034 14558 consensus_queue.cc:237] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:56.825912 22024 raft_consensus.cc:2749] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:56.832679 13987 leader_election.cc:336] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.833282 13987 leader_election.cc:304] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.834162 22024 raft_consensus.cc:2749] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:56.835465 13987 leader_election.cc:336] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.835945 13987 leader_election.cc:304] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.836774 22024 raft_consensus.cc:2749] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:56.838415 13987 leader_election.cc:336] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:56.839144 13987 leader_election.cc:304] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:56.839910 22024 raft_consensus.cc:2749] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:56.860836 22020 raft_consensus.cc:493] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.861436 22020 raft_consensus.cc:515] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.863656 22020 leader_election.cc:290] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:56.870157 13989 leader_election.cc:336] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:56.874131 22024 raft_consensus.cc:493] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.875058 22024 raft_consensus.cc:515] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:56.877456 22024 leader_election.cc:290] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:56.886165 22024 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:56.886849 22024 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:56.887671 13989 leader_election.cc:336] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:57.000346 13987 leader_election.cc:336] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:57.001036 13987 leader_election.cc:304] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:57.004149 22004 raft_consensus.cc:2749] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:57.010846 14015 raft_consensus.cc:1275] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:57.021157 22024 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:57.031097 21999 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:57.031823 21999 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:57.029407 13987 leader_election.cc:336] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:57.034881 13987 leader_election.cc:304] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:57.035691 22024 raft_consensus.cc:2749] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:56.912775 14522 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:57.037267 14522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:43953, user_credentials={real_user=slave}} blocked reactor thread for 124579us
I20260709 13:07:57.044704 21999 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:57.045445 21999 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:57.045958 21999 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:57.048002 21999 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:57.053297 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:57.054112 14553 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 17.
I20260709 13:07:57.054416 22026 consensus_queue.cc:1048] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:57.063932 13988 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:57.065550 22026 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:57.065666 22020 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:57.072389 22020 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:57.072821 22020 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:57.075414 22004 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:57.076043 22004 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:57.078039 13987 leader_election.cc:336] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:57.078521 22004 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:57.080725 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
W20260709 13:07:57.066606 14524 leader_election.cc:336] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:57.079591 22026 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:57.095757 22026 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:57.097482 22024 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:57.099699 22024 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:57.090559 22020 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:57.103353 14018 raft_consensus.cc:2393] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 18: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:57.103659 22020 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:57.104604 14024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:57.105266 14024 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
I20260709 13:07:57.106384 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:57.101835 22024 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:57.107052 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" is_pre_election: true
I20260709 13:07:57.107769 14553 raft_consensus.cc:2468] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:57.111914 13988 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:57.112890 22004 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:57.113291 22004 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:57.113696 22004 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:57.114650 14553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8599b3d11641199dc0043a592857fb" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
W20260709 13:07:57.115126 13987 leader_election.cc:336] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:57.115208 14553 raft_consensus.cc:3060] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:57.115798 13987 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:57.116566 22020 raft_consensus.cc:2749] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:57.123859 14553 raft_consensus.cc:2468] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in term 18.
I20260709 13:07:57.125195 13988 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:57.125905 22020 raft_consensus.cc:2804] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:57.126291 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 
I20260709 13:07:57.126513 22020 raft_consensus.cc:697] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Becoming Leader. State: Replica: d1b4fd04200247d99ba9ad1fbeb42b42, State: Running, Role: LEADER
I20260709 13:07:57.127048 21999 raft_consensus.cc:2804] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:57.127403 21999 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:57.127720 21999 raft_consensus.cc:3060] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:57.127578 22020 consensus_queue.cc:237] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:57.135233 21999 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:57.137190 21999 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:57.142161 14553 raft_consensus.cc:1275] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:07:57.144773 14522 leader_election.cc:336] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:57.153115 14522 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:57.150266 13987 consensus_peers.cc:597] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:57.154129 13987 leader_election.cc:336] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:57.158473 21999 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:57.161768 14522 leader_election.cc:336] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
W20260709 13:07:57.162166 13989 consensus_peers.cc:597] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:07:57.162990 14522 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:57.163255 22020 consensus_queue.cc:1048] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:57.163964 21999 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:57.167501 13987 leader_election.cc:336] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:57.171293 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
W20260709 13:07:57.173326 13989 consensus_peers.cc:597] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:07:57.174121 13989 consensus_peers.cc:597] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:57.177420 14522 leader_election.cc:336] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
W20260709 13:07:57.180496 13989 consensus_peers.cc:597] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:57.181885 14522 leader_election.cc:336] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:57.173326 22004 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:57.187237 22026 raft_consensus.cc:2955] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Committing config change with OpId 17.14: config changed from index 5 to 14, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
W20260709 13:07:57.197767 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:07:57.201198 14015 raft_consensus.cc:2393] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dd265d61fc141a48ca9d3d5c8f52921 in current term 19: Already voted for candidate d1b4fd04200247d99ba9ad1fbeb42b42 in this term.
I20260709 13:07:57.210235 14524 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:57.211159 14015 raft_consensus.cc:2955] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 5 to 14, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } } }
I20260709 13:07:57.214218 21999 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:57.221364 22035 raft_consensus.cc:493] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:57.221989 22035 raft_consensus.cc:515] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } }
I20260709 13:07:57.221737 22034 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:57.224264 22034 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:57.228377 22034 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:57.231428 22035 leader_election.cc:290] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
I20260709 13:07:57.232081 14561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efeae9bc4fd84c4487b09362929c2fa6" candidate_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "4dd265d61fc141a48ca9d3d5c8f52921"
I20260709 13:07:57.229214 13766 catalog_manager.cc:5697] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 reported cstate change: term changed from 17 to 18, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67). New cstate: current_term: 18 leader_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.232880 14561 raft_consensus.cc:2393] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d1b4fd04200247d99ba9ad1fbeb42b42 in current term 19: Already voted for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in this term.
I20260709 13:07:57.234237 22004 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68:42851)
I20260709 13:07:57.242072 13758 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 545fc4aeb71d491985e296a0721d905d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 15)
W20260709 13:07:57.244268 14522 leader_election.cc:336] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
W20260709 13:07:57.251051 13989 leader_election.cc:336] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:57.254947 14018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" is_pre_election: true
I20260709 13:07:57.255689 14018 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 17.
I20260709 13:07:57.266369 13764 catalog_manager.cc:5697] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: config changed from index 5 to 14, VOTER e1649107c781428692458f10597c5742 (127.12.109.65) evicted. New cstate: current_term: 17 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:57.271903 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
W20260709 13:07:57.272328 13987 leader_election.cc:336] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:57.272822 13987 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, 4dd265d61fc141a48ca9d3d5c8f52921
I20260709 13:07:57.273006 21999 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:57.273365 21999 raft_consensus.cc:493] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:57.273728 21999 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:57.273841 13987 leader_election.cc:336] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:57.274363 13987 leader_election.cc:304] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:57.275866 22034 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:57.276414 22034 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:57.280586 22034 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:57.284571 14522 leader_election.cc:336] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
W20260709 13:07:57.298842 14524 leader_election.cc:336] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:57.300655 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:57.301661 22034 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:57.302726 22035 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:57.303645 22035 raft_consensus.cc:2749] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:57.293267 21999 raft_consensus.cc:515] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:57.339061 13759 catalog_manager.cc:4755] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 545fc4aeb71d491985e296a0721d905d: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:57.343992 14522 leader_election.cc:336] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
W20260709 13:07:57.345201 14524 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:07:57.349332 21999 leader_election.cc:290] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: Requested vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:57.352521 14015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208a6eaac64647a59ee173ef6726ae8b" candidate_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42"
I20260709 13:07:57.353204 14015 raft_consensus.cc:3060] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:57.355660 13989 consensus_peers.cc:597] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:57.361389 14015 raft_consensus.cc:2468] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd265d61fc141a48ca9d3d5c8f52921 in term 18.
W20260709 13:07:57.373250 13989 consensus_peers.cc:597] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:07:57.376618 14524 leader_election.cc:304] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921, d1b4fd04200247d99ba9ad1fbeb42b42; no voters: 3779649036364744823fc035dac892c7
I20260709 13:07:57.377605 21999 raft_consensus.cc:2804] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:57.378153 21999 raft_consensus.cc:697] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 LEADER]: Becoming Leader. State: Replica: 4dd265d61fc141a48ca9d3d5c8f52921, State: Running, Role: LEADER
I20260709 13:07:57.379101 21999 consensus_queue.cc:237] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:57.422555 13989 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:07:57.440526 13766 catalog_manager.cc:5697] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 reported cstate change: term changed from 17 to 18, leader changed from 3779649036364744823fc035dac892c7 (127.12.109.66) to 4dd265d61fc141a48ca9d3d5c8f52921 (127.12.109.68). New cstate: current_term: 18 leader_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:57.473245 13989 consensus_peers.cc:597] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:57.524263 13987 consensus_peers.cc:597] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:57.529557 12725 tablet_server.cc:196] TabletServer@127.12.109.66:0 shutdown complete.
W20260709 13:07:57.532832 13987 consensus_peers.cc:597] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.557435 14522 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.569489 13987 consensus_peers.cc:597] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.573050 13987 consensus_peers.cc:597] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.617452 14522 consensus_peers.cc:597] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.620839 13989 consensus_peers.cc:597] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:57.643210 13987 consensus_peers.cc:597] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:57.649461 14522 consensus_peers.cc:597] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:57.664350 14553 raft_consensus.cc:1275] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Refusing update from remote peer d1b4fd04200247d99ba9ad1fbeb42b42: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:57.669544 22035 consensus_queue.cc:1048] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:07:57.671434 13987 consensus_peers.cc:597] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.678265 13989 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:07:57.699671 13987 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.704516 13989 consensus_peers.cc:597] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:07:57.753325 14522 consensus_peers.cc:597] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:57.768841 14015 raft_consensus.cc:1275] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Refusing update from remote peer 4dd265d61fc141a48ca9d3d5c8f52921: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:07:57.777453 13987 consensus_peers.cc:597] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:57.780690 21999 consensus_queue.cc:1048] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:57.782524 12725 tablet_server.cc:179] TabletServer@127.12.109.67:0 shutting down...
W20260709 13:07:57.789857 13987 consensus_peers.cc:597] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.799005 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Couldn't send request to peer d1b4fd04200247d99ba9ad1fbeb42b42. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:57.799746 14524 consensus_peers.cc:597] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Couldn't send request to peer d1b4fd04200247d99ba9ad1fbeb42b42. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:57.800390 14524 consensus_peers.cc:597] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Couldn't send request to peer d1b4fd04200247d99ba9ad1fbeb42b42. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:57.801988 14524 consensus_peers.cc:597] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Couldn't send request to peer d1b4fd04200247d99ba9ad1fbeb42b42. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:57.828527 14524 consensus_peers.cc:597] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Couldn't send request to peer d1b4fd04200247d99ba9ad1fbeb42b42. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:57.835049 14524 consensus_peers.cc:597] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:57.851311 13989 consensus_peers.cc:597] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:07:57.859138 14524 consensus_peers.cc:597] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Couldn't send request to peer d1b4fd04200247d99ba9ad1fbeb42b42. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:57.878131 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:07:57.878021 13987 consensus_peers.cc:597] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:57.879426 12725 tablet_replica.cc:333] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:57.880591 12725 raft_consensus.cc:2243] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:07:57.882510 12725 raft_consensus.cc:2272] T 9a2c65bca5ce45d5af2ace962fa0a37f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:57.887324 13987 consensus_peers.cc:597] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.888036 13987 consensus_peers.cc:597] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:57.888173 12725 tablet_replica.cc:333] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:57.889379 12725 raft_consensus.cc:2243] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:57.890199 12725 raft_consensus.cc:2272] T 728de4a891d847028775aa5c35ba53e1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:57.893507 12725 tablet_replica.cc:333] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:57.894632 12725 raft_consensus.cc:2243] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:57.895404 12725 raft_consensus.cc:2272] T 56605211bd584b8889b54049e9f6cb4b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:57.898463 12725 tablet_replica.cc:333] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:57.899430 12725 raft_consensus.cc:2243] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:57.900260 12725 raft_consensus.cc:2272] T 10b028ec30974d06bd9da921c268a641 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:57.903246 12725 tablet_replica.cc:333] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:57.904320 12725 raft_consensus.cc:2243] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:57.905239 12725 raft_consensus.cc:2272] T 617d39d9397c4724be821e5efacbde17 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:57.908372 12725 tablet_replica.cc:333] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
W20260709 13:07:57.903303 13989 consensus_peers.cc:597] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:07:57.909339 12725 raft_consensus.cc:2243] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:07:57.910820 12725 raft_consensus.cc:2272] T 343e333726b74ed98b7e80869ce28ad1 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:57.913903 12725 tablet_replica.cc:333] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:57.916217 12725 raft_consensus.cc:2243] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:57.917136 12725 raft_consensus.cc:2272] T d4033a61013848c69bbd0e79b27a8be3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:57.918211 14045 debug-util.cc:398] Leaking SignalData structure 0x7b0800151fc0 after lost signal to thread 12728
W20260709 13:07:57.919706 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08000cede0 after lost signal to thread 13807
I20260709 13:07:57.920527 12725 tablet_replica.cc:333] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
W20260709 13:07:57.921442 14045 debug-util.cc:398] Leaking SignalData structure 0x7b08003aea80 after lost signal to thread 14048
I20260709 13:07:57.921481 12725 raft_consensus.cc:2243] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:57.922600 12725 raft_consensus.cc:2272] T 0768fb01770349dfb259ab0db91394ba P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:57.925863 12725 tablet_replica.cc:333] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:57.926836 12725 raft_consensus.cc:2243] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:57.928148 12725 raft_consensus.cc:2272] T 9a37812504ce4bdb94c7eafc67fa5aee P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:57.929590 14045 debug-util.cc:398] Leaking SignalData structure 0x7b0800358dc0 after lost signal to thread 14585
I20260709 13:07:57.931401 12725 tablet_replica.cc:333] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
W20260709 13:07:57.932315 14522 consensus_peers.cc:597] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.953779 14522 consensus_peers.cc:597] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:57.957844 12725 raft_consensus.cc:2243] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:57.960536 12725 raft_consensus.cc:2272] T 4daef29e1ece473f9845687b3c7eae60 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:57.963928 12725 tablet_replica.cc:333] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:57.964916 12725 raft_consensus.cc:2243] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:57.965656 12725 raft_consensus.cc:2272] T 0e8d34f260fb43229460064c72d795e8 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:57.969003 12725 tablet_replica.cc:333] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
W20260709 13:07:57.969528 14522 consensus_peers.cc:597] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.970162 14522 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:57.970163 12725 raft_consensus.cc:2243] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:07:57.972121 12725 raft_consensus.cc:2272] T 92405e92560c4099974ff3e6a4564895 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:57.975246 12725 tablet_replica.cc:333] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:57.976199 12725 raft_consensus.cc:2243] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:07:57.977595 12725 raft_consensus.cc:2272] T 02dcf286c6bd48e3a6e548def7d786cc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:57.981029 13987 consensus_peers.cc:597] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.991881 14522 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.991922 13987 consensus_peers.cc:597] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:57.996989 13987 consensus_peers.cc:597] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:57.996455 12725 tablet_replica.cc:333] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:57.998380 12725 raft_consensus.cc:2243] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:57.999269 12725 raft_consensus.cc:2272] T a168d82765704a87b2c60aeb7b23e8b0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:58.002225 14522 consensus_peers.cc:597] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:58.004912 12725 tablet_replica.cc:333] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
W20260709 13:07:58.005854 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Couldn't send request to peer d1b4fd04200247d99ba9ad1fbeb42b42. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:58.006196 12725 raft_consensus.cc:2243] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.007210 12725 raft_consensus.cc:2272] T 45103ab867574475a587979f8ea5fa4e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.011854 12725 tablet_replica.cc:333] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.012799 12725 raft_consensus.cc:2243] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.013633 12725 raft_consensus.cc:2272] T 99b9907790614d49bc3adf40424fc1b6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.017014 12725 tablet_replica.cc:333] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.018007 12725 raft_consensus.cc:2243] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.020319 12725 raft_consensus.cc:2272] T 4a43a85b399e492b8391cc93e618a1fa P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.023758 12725 tablet_replica.cc:333] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.024611 12725 raft_consensus.cc:2243] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.025843 12725 raft_consensus.cc:2272] T 10a0b27264ce49379f2aba40cdbc2b0f P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:58.026651 13987 consensus_peers.cc:597] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:58.028960 12725 tablet_replica.cc:333] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.029944 12725 raft_consensus.cc:2243] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.030956 12725 raft_consensus.cc:2272] T 6b098cc2d51e471cb2c9c1b17eebcb57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:58.039145 13987 consensus_peers.cc:597] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:58.040747 12725 tablet_replica.cc:333] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.041853 12725 raft_consensus.cc:2243] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.043068 12725 raft_consensus.cc:2272] T 86de6a18f3bd4b23a8b66dea2cf5c6a9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.046123 12725 tablet_replica.cc:333] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.047072 12725 raft_consensus.cc:2243] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.047765 12725 raft_consensus.cc:2272] T 4eeee4c6df564edcb3c96e2887208cc4 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.050935 12725 tablet_replica.cc:333] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.051919 12725 raft_consensus.cc:2243] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.053045 12725 raft_consensus.cc:2272] T f2e6672bab624b20b38e363591856fc0 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.056185 12725 tablet_replica.cc:333] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.057147 12725 raft_consensus.cc:2243] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.057940 12725 raft_consensus.cc:2272] T f50263f80db844239948e5e67ef3b987 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.060890 12725 tablet_replica.cc:333] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.061810 12725 raft_consensus.cc:2243] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.063436 12725 raft_consensus.cc:2272] T 693a6579cdaa4e2a977368c2c6d5267a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.066399 12725 tablet_replica.cc:333] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.067341 12725 raft_consensus.cc:2243] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.070863 12725 raft_consensus.cc:2272] T c3f6783295084924801a104ff3faf10e P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:58.073527 14524 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Couldn't send request to peer d1b4fd04200247d99ba9ad1fbeb42b42. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:58.074802 12725 tablet_replica.cc:333] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.075969 12725 raft_consensus.cc:2243] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.077350 12725 raft_consensus.cc:2272] T 7991c821f3da4b7abc0f6bb6baf8fe99 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.080942 12725 tablet_replica.cc:333] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.084367 12725 raft_consensus.cc:2243] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.086175 12725 raft_consensus.cc:2272] T 9bfd3f44e83e4908b229bb57863e4080 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.089512 12725 tablet_replica.cc:333] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.090587 12725 raft_consensus.cc:2243] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.092207 12725 raft_consensus.cc:2272] T 7f4c2ba739c04f16a8ed8c28f57fe695 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.095243 12725 tablet_replica.cc:333] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.096131 12725 raft_consensus.cc:2243] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.097090 12725 raft_consensus.cc:2272] T 34d11c517b334dd5abf16448009bdf2d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.100322 12725 tablet_replica.cc:333] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.101190 12725 raft_consensus.cc:2243] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.102501 12725 raft_consensus.cc:2272] T 3730fc366f4349dea87ae85e1aed546c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.105701 12725 tablet_replica.cc:333] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.106590 12725 raft_consensus.cc:2243] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.107404 12725 raft_consensus.cc:2272] T f9158487769746e2a7f4f283dda5abf3 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.110193 12725 tablet_replica.cc:333] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.111052 12725 raft_consensus.cc:2243] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.112022 12725 raft_consensus.cc:2272] T 82051b9bd85845d6b73773b21ccc6d6c P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.114969 12725 tablet_replica.cc:333] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.115906 12725 raft_consensus.cc:2243] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.117332 12725 raft_consensus.cc:2272] T dcd5a97e61554ed698b00fdf13294a7b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.120198 12725 tablet_replica.cc:333] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.121150 12725 raft_consensus.cc:2243] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.122599 12725 raft_consensus.cc:2272] T 765348f2471a493d92e998132c0c746b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.125360 12725 tablet_replica.cc:333] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.126456 12725 raft_consensus.cc:2243] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.127974 12725 raft_consensus.cc:2272] T 8dfe02d97ee6481298ba92b29baa32b7 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.131057 12725 tablet_replica.cc:333] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.131829 12725 raft_consensus.cc:2243] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.133096 12725 raft_consensus.cc:2272] T 17624553356541c69c1644ca409df1e9 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.136140 12725 tablet_replica.cc:333] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.136922 12725 raft_consensus.cc:2243] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.138401 12725 raft_consensus.cc:2272] T 5d6ce914003a44f2b5a485ae7962383a P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.141027 12725 tablet_replica.cc:333] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.141846 12725 raft_consensus.cc:2243] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.142475 12725 raft_consensus.cc:2272] T 208a6eaac64647a59ee173ef6726ae8b P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.145512 12725 tablet_replica.cc:333] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.146370 12725 raft_consensus.cc:2243] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.147764 12725 raft_consensus.cc:2272] T be9dcc1a27af4a9a91a283a728d4132d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.150110 12725 tablet_replica.cc:333] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.150702 12725 raft_consensus.cc:2243] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.151616 12725 raft_consensus.cc:2272] T 567aed48a57a41e5820f0b937f2de9be P d1b4fd04200247d99ba9ad1fbeb42b42 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.154206 12725 tablet_replica.cc:333] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.154762 12725 raft_consensus.cc:2243] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.155309 12725 raft_consensus.cc:2272] T efeae9bc4fd84c4487b09362929c2fa6 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.157675 12725 tablet_replica.cc:333] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.158249 12725 raft_consensus.cc:2243] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.158918 12725 raft_consensus.cc:2272] T 4e20f6d8df1f4522b10fd2dc803e0e75 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.161036 12725 tablet_replica.cc:333] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.161561 12725 raft_consensus.cc:2243] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.162331 12725 raft_consensus.cc:2272] T 1ec9c4d3cc3a47bd838d18fe7da650fc P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.164870 12725 tablet_replica.cc:333] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.165802 12725 raft_consensus.cc:2243] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.167038 12725 raft_consensus.cc:2272] T a3d53d5ea0e8450e97d9e321677e63d5 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.169746 12725 tablet_replica.cc:333] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.170331 12725 raft_consensus.cc:2243] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.171626 12725 raft_consensus.cc:2272] T fdef734205364543b02ed4158216bf36 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.173851 12725 tablet_replica.cc:333] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.174475 12725 raft_consensus.cc:2243] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.175261 12725 raft_consensus.cc:2272] T 47fee46f08eb4928939d6ac41891db38 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.178026 12725 tablet_replica.cc:333] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.178839 12725 raft_consensus.cc:2243] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.179477 12725 raft_consensus.cc:2272] T 76071b78766a43e0b5097f615de4c569 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.181950 12725 tablet_replica.cc:333] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.182542 12725 raft_consensus.cc:2243] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.183386 12725 raft_consensus.cc:2272] T 481a7f1a133247ebb81e1be603cbdd57 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.185439 12725 tablet_replica.cc:333] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.185940 12725 raft_consensus.cc:2243] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.186491 12725 raft_consensus.cc:2272] T 6d6d5ae5feb04bda8f546e7e4ca80503 P d1b4fd04200247d99ba9ad1fbeb42b42 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.188611 12725 tablet_replica.cc:333] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.189363 12725 raft_consensus.cc:2243] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:58.192394 12725 raft_consensus.cc:2272] T 9f8599b3d11641199dc0043a592857fb P d1b4fd04200247d99ba9ad1fbeb42b42 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.196213 12725 tablet_replica.cc:333] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42: stopping tablet replica
I20260709 13:07:58.197147 12725 raft_consensus.cc:2243] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.198019 12725 raft_consensus.cc:2272] T 545fc4aeb71d491985e296a0721d905d P d1b4fd04200247d99ba9ad1fbeb42b42 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:58.219808 14522 consensus_peers.cc:597] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:58.236577 14524 consensus_peers.cc:597] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Couldn't send request to peer d1b4fd04200247d99ba9ad1fbeb42b42. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:58.444442 13821 auto_rebalancer.cc:629] Failed to schedule move for tablet 7f4c2ba739c04f16a8ed8c28f57fe695: Network error: Client connection negotiation failed: client connection to 127.12.109.67:46599: connect: Connection refused (error 111)
W20260709 13:07:58.449378 13821 auto_rebalancer.cc:629] Failed to schedule move for tablet 17624553356541c69c1644ca409df1e9: Network error: Client connection negotiation failed: client connection to 127.12.109.67:46599: connect: Connection refused (error 111)
W20260709 13:07:58.459271 13821 auto_rebalancer.cc:629] Failed to schedule move for tablet 617d39d9397c4724be821e5efacbde17: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
W20260709 13:07:58.464102 13821 auto_rebalancer.cc:629] Failed to schedule move for tablet 0768fb01770349dfb259ab0db91394ba: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
W20260709 13:07:58.502772 14524 consensus_peers.cc:597] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:07:58.522951 13759 catalog_manager.cc:4755] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 545fc4aeb71d491985e296a0721d905d: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:58.624096 14524 consensus_peers.cc:597] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:07:58.721572 14524 consensus_peers.cc:597] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:07:58.821578 22047 raft_consensus.cc:493] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:58.822336 22047 raft_consensus.cc:515] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:58.826160 22047 leader_election.cc:290] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:58.829208 14524 leader_election.cc:336] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:58.830543 14522 leader_election.cc:336] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:58.831024 14522 leader_election.cc:304] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
I20260709 13:07:58.831849 22047 raft_consensus.cc:2749] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:58.993512 14524 consensus_peers.cc:597] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:07:59.154315 22046 raft_consensus.cc:493] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:59.154953 22046 raft_consensus.cc:515] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:59.157251 22046 leader_election.cc:290] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:59.161993 14524 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:59.163832 14524 leader_election.cc:336] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Network error: Client connection negotiation failed: client connection to 127.12.109.67:46599: connect: Connection refused (error 111)
I20260709 13:07:59.164227 14524 leader_election.cc:304] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:59.164867 22046 raft_consensus.cc:2749] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:59.209990 22046 raft_consensus.cc:493] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:59.210635 22046 raft_consensus.cc:515] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:59.212659 22046 leader_election.cc:290] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:59.215776 22046 raft_consensus.cc:493] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:59.216291 22046 raft_consensus.cc:515] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
W20260709 13:07:59.218947 14522 leader_election.cc:336] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:59.218990 22046 leader_election.cc:290] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:59.221840 14522 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
W20260709 13:07:59.222862 14524 leader_election.cc:336] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Network error: Client connection negotiation failed: client connection to 127.12.109.67:46599: connect: Connection refused (error 111)
I20260709 13:07:59.223230 14524 leader_election.cc:304] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:59.223855 22046 raft_consensus.cc:2749] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:59.225513 14524 leader_election.cc:336] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Network error: Client connection negotiation failed: client connection to 127.12.109.67:46599: connect: Connection refused (error 111)
I20260709 13:07:59.225922 14524 leader_election.cc:304] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:59.226563 22046 raft_consensus.cc:2749] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:59.235028 14524 consensus_peers.cc:597] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:07:59.290490 22047 raft_consensus.cc:493] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:59.291157 22047 raft_consensus.cc:515] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:59.293478 22047 leader_election.cc:290] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
I20260709 13:07:59.296245 22047 raft_consensus.cc:493] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:59.296777 22047 raft_consensus.cc:515] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:59.298799 22047 leader_election.cc:290] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:59.303013 14524 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:07:59.309170 14524 leader_election.cc:336] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Network error: Client connection negotiation failed: client connection to 127.12.109.67:46599: connect: Connection refused (error 111)
I20260709 13:07:59.309590 14524 leader_election.cc:304] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:59.310348 22046 raft_consensus.cc:2749] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:07:59.310556 14524 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Network error: Client connection negotiation failed: client connection to 127.12.109.67:46599: connect: Connection refused (error 111)
W20260709 13:07:59.311337 14524 leader_election.cc:336] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:59.311694 14524 leader_election.cc:304] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:59.312335 22046 raft_consensus.cc:2749] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:07:59.330518 12725 tablet_server.cc:196] TabletServer@127.12.109.67:0 shutdown complete.
W20260709 13:07:59.362205 14522 consensus_peers.cc:597] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Couldn't send request to peer 3779649036364744823fc035dac892c7. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:59.365324 14524 consensus_peers.cc:597] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:07:59.383383 22046 raft_consensus.cc:493] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.384063 22046 raft_consensus.cc:515] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:59.386404 22046 leader_election.cc:290] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3779649036364744823fc035dac892c7 (127.12.109.66:43953), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:59.392200 14524 leader_election.cc:336] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Network error: Client connection negotiation failed: client connection to 127.12.109.67:46599: connect: Connection refused (error 111)
W20260709 13:07:59.392886 14522 leader_election.cc:336] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:59.393383 14522 leader_election.cc:304] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, d1b4fd04200247d99ba9ad1fbeb42b42
I20260709 13:07:59.394179 22046 raft_consensus.cc:2749] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:59.481808 12725 tablet_server.cc:179] TabletServer@127.12.109.68:0 shutting down...
W20260709 13:07:59.488615 14524 consensus_peers.cc:597] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 -> Peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Couldn't send request to peer e1649107c781428692458f10597c5742. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:07:59.488014 22046 raft_consensus.cc:493] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3779649036364744823fc035dac892c7)
I20260709 13:07:59.492996 22046 raft_consensus.cc:515] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "3779649036364744823fc035dac892c7" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 43953 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:59.496219 22046 leader_election.cc:290] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), 3779649036364744823fc035dac892c7 (127.12.109.66:43953)
W20260709 13:07:59.502534 14522 leader_election.cc:336] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3779649036364744823fc035dac892c7 (127.12.109.66:43953): Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:59.506844 22047 raft_consensus.cc:493] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d1b4fd04200247d99ba9ad1fbeb42b42)
I20260709 13:07:59.508333 22047 raft_consensus.cc:515] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e1649107c781428692458f10597c5742" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 43607 } } peers { permanent_uuid: "d1b4fd04200247d99ba9ad1fbeb42b42" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46599 } } peers { permanent_uuid: "4dd265d61fc141a48ca9d3d5c8f52921" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 42851 } attrs { promote: false } }
I20260709 13:07:59.520759 22047 leader_election.cc:290] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e1649107c781428692458f10597c5742 (127.12.109.65:43607), d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599)
W20260709 13:07:59.539304 14524 leader_election.cc:336] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:59.539994 14524 leader_election.cc:304] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: 3779649036364744823fc035dac892c7, e1649107c781428692458f10597c5742
W20260709 13:07:59.541268 14524 leader_election.cc:336] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e1649107c781428692458f10597c5742 (127.12.109.65:43607): Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:59.541134 22047 raft_consensus.cc:2749] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:07:59.546538 14524 leader_election.cc:336] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d1b4fd04200247d99ba9ad1fbeb42b42 (127.12.109.67:46599): Network error: Client connection negotiation failed: client connection to 127.12.109.67:46599: connect: Connection refused (error 111)
I20260709 13:07:59.546934 14524 leader_election.cc:304] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dd265d61fc141a48ca9d3d5c8f52921; no voters: d1b4fd04200247d99ba9ad1fbeb42b42, e1649107c781428692458f10597c5742
I20260709 13:07:59.547611 22046 raft_consensus.cc:2749] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:59.571218 13759 catalog_manager.cc:4755] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 545fc4aeb71d491985e296a0721d905d: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:07:59.591063 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:59.592164 12725 tablet_replica.cc:333] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.593356 12725 raft_consensus.cc:2243] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.594792 12725 raft_consensus.cc:2272] T 6b098cc2d51e471cb2c9c1b17eebcb57 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.598264 12725 tablet_replica.cc:333] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.599150 12725 raft_consensus.cc:2243] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:59.599911 12725 raft_consensus.cc:2272] T 02dcf286c6bd48e3a6e548def7d786cc P 4dd265d61fc141a48ca9d3d5c8f52921 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.603154 12725 tablet_replica.cc:333] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.603960 12725 raft_consensus.cc:2243] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:07:59.604897 13821 auto_rebalancer.cc:629] Failed to schedule move for tablet 693a6579cdaa4e2a977368c2c6d5267a: Network error: Client connection negotiation failed: client connection to 127.12.109.67:46599: connect: Connection refused (error 111)
I20260709 13:07:59.605387 12725 raft_consensus.cc:2272] T 14bba4e2cef04a418c6f660869c7e6e5 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:59.608637 13821 auto_rebalancer.cc:629] Failed to schedule move for tablet 99b9907790614d49bc3adf40424fc1b6: Network error: Client connection negotiation failed: client connection to 127.12.109.66:43953: connect: Connection refused (error 111)
I20260709 13:07:59.608842 12725 tablet_replica.cc:333] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.609799 12725 raft_consensus.cc:2243] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.611032 12725 raft_consensus.cc:2272] T 545fc4aeb71d491985e296a0721d905d P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:59.612071 13821 auto_rebalancer.cc:629] Failed to schedule move for tablet 92405e92560c4099974ff3e6a4564895: Network error: Client connection negotiation failed: client connection to 127.12.109.67:46599: connect: Connection refused (error 111)
I20260709 13:07:59.614096 12725 tablet_replica.cc:333] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.615051 12725 raft_consensus.cc:2243] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 28 LEADER]: Raft consensus shutting down.
W20260709 13:07:59.615235 13821 auto_rebalancer.cc:629] Failed to schedule move for tablet be9dcc1a27af4a9a91a283a728d4132d: Network error: Client connection negotiation failed: client connection to 127.12.109.67:46599: connect: Connection refused (error 111)
I20260709 13:07:59.616182 12725 raft_consensus.cc:2272] T 435cf49d11154354987bf559d46d3c3a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.619148 12725 tablet_replica.cc:333] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.620072 12725 raft_consensus.cc:2243] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:59.620743 12725 raft_consensus.cc:2272] T 567aed48a57a41e5820f0b937f2de9be P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.623487 12725 tablet_replica.cc:333] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.624255 12725 raft_consensus.cc:2243] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:59.624830 12725 raft_consensus.cc:2272] T efeae9bc4fd84c4487b09362929c2fa6 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.627714 12725 tablet_replica.cc:333] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.628412 12725 raft_consensus.cc:2243] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.629535 12725 raft_consensus.cc:2272] T 82051b9bd85845d6b73773b21ccc6d6c P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.632364 12725 tablet_replica.cc:333] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.633054 12725 raft_consensus.cc:2243] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:59.633823 12725 raft_consensus.cc:2272] T 9a2c65bca5ce45d5af2ace962fa0a37f P 4dd265d61fc141a48ca9d3d5c8f52921 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.636538 12725 tablet_replica.cc:333] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.637280 12725 raft_consensus.cc:2243] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.638473 12725 raft_consensus.cc:2272] T 72ee2fe56c0b4dd5bf2a9c07597c79ba P 4dd265d61fc141a48ca9d3d5c8f52921 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.641320 12725 tablet_replica.cc:333] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.642002 12725 raft_consensus.cc:2243] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.643256 12725 raft_consensus.cc:2272] T 6d6d5ae5feb04bda8f546e7e4ca80503 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.646001 12725 tablet_replica.cc:333] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.646714 12725 raft_consensus.cc:2243] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:59.647312 12725 raft_consensus.cc:2272] T c3f6783295084924801a104ff3faf10e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.650182 12725 tablet_replica.cc:333] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.650843 12725 raft_consensus.cc:2243] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:59.651397 12725 raft_consensus.cc:2272] T 9f8599b3d11641199dc0043a592857fb P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.654325 12725 tablet_replica.cc:333] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.655022 12725 raft_consensus.cc:2243] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:59.655593 12725 raft_consensus.cc:2272] T 17bd99d4be2c4073bd867c161fa52482 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.658030 12725 tablet_replica.cc:333] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.658838 12725 raft_consensus.cc:2243] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:59.659466 12725 raft_consensus.cc:2272] T 403f6d16105d4532ae1de441c32f632a P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.662360 12725 tablet_replica.cc:333] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.663072 12725 raft_consensus.cc:2243] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.664179 12725 raft_consensus.cc:2272] T f5c93fb008bd4a4f8db94cdc59c65d93 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.667177 12725 tablet_replica.cc:333] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.667984 12725 raft_consensus.cc:2243] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.669173 12725 raft_consensus.cc:2272] T 47fee46f08eb4928939d6ac41891db38 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.671880 12725 tablet_replica.cc:333] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.672647 12725 raft_consensus.cc:2243] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.673859 12725 raft_consensus.cc:2272] T 4eeee4c6df564edcb3c96e2887208cc4 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.676520 12725 tablet_replica.cc:333] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.677294 12725 raft_consensus.cc:2243] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.678658 12725 pending_rounds.cc:70] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Trying to abort 1 pending ops.
I20260709 13:07:59.678879 12725 pending_rounds.cc:77] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921: Aborting op as it isn't in flight: id { term: 18 index: 15 } timestamp: 7305635747353858048 op_type: NO_OP noop_request { }
I20260709 13:07:59.679270 12725 raft_consensus.cc:2889] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:59.679585 12725 raft_consensus.cc:2272] T 208a6eaac64647a59ee173ef6726ae8b P 4dd265d61fc141a48ca9d3d5c8f52921 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.681993 12725 tablet_replica.cc:333] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.682596 12725 raft_consensus.cc:2243] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.683526 12725 raft_consensus.cc:2272] T ef6c4cc997504124a8bb7fa9c7f10e34 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.685073 12725 tablet_replica.cc:333] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.685636 12725 raft_consensus.cc:2243] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.686915 12725 raft_consensus.cc:2272] T 45103ab867574475a587979f8ea5fa4e P 4dd265d61fc141a48ca9d3d5c8f52921 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.689975 12725 tablet_replica.cc:333] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.690732 12725 raft_consensus.cc:2243] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.691769 12725 raft_consensus.cc:2272] T 0e8d34f260fb43229460064c72d795e8 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.694414 12725 tablet_replica.cc:333] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.694954 12725 raft_consensus.cc:2243] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:59.695953 12725 raft_consensus.cc:2272] T 05adb1eba4fe476d8504a36537b738db P 4dd265d61fc141a48ca9d3d5c8f52921 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.698312 12725 tablet_replica.cc:333] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921: stopping tablet replica
I20260709 13:07:59.698882 12725 raft_consensus.cc:2243] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:59.699474 12725 raft_consensus.cc:2272] T 4daef29e1ece473f9845687b3c7eae60 P 4dd265d61fc141a48ca9d3d5c8f52921 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:00.315644 12725 tablet_server.cc:196] TabletServer@127.12.109.68:0 shutdown complete.
I20260709 13:08:00.366236 12725 master.cc:562] Master@127.12.109.126:43315 shutting down...
W20260709 13:08:00.580120 13759 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111) [suppressed 566 similar messages]
W20260709 13:08:00.583738 13759 catalog_manager.cc:4755] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6b098cc2d51e471cb2c9c1b17eebcb57: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
W20260709 13:08:01.660080 13759 catalog_manager.cc:4755] TS e1649107c781428692458f10597c5742 (127.12.109.65:43607): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 545fc4aeb71d491985e296a0721d905d: Network error: Client connection negotiation failed: client connection to 127.12.109.65:43607: connect: Connection refused (error 111)
I20260709 13:08:01.887427 12725 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:01.888711 12725 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:01.889093 12725 tablet_replica.cc:333] T 00000000000000000000000000000000 P 91c63734839347c4aee340a59c8e3221: stopping tablet replica
I20260709 13:08:02.041499 12725 master.cc:584] Master@127.12.109.126:43315 shutdown complete.
I20260709 13:08:02.103708 12725 test_util.cc:182] -----------------------------------------------
I20260709 13:08:02.103950 12725 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602188851028-12725-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (268419 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:08:02.107769 12725 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.109.126:37329
I20260709 13:08:02.108989 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:02.115084 22061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:02.115238 22060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:02.116061 22063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:02.117599 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:02.121732 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:02.121951 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:02.122185 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602482122168 us; error 0 us; skew 500 ppm
I20260709 13:08:02.123023 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:02.125905 12725 webserver.cc:533] Webserver started at http://127.12.109.126:44957/ using document root <none> and password file <none>
I20260709 13:08:02.126533 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:02.126730 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:02.127064 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:02.128618 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/master-0-root/instance:
uuid: "c0557a4af02b4c25b02595b71c60436b"
format_stamp: "Formatted at 2026-07-09 13:08:02 on dist-test-slave-zhdc"
I20260709 13:08:02.133476 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:08:02.137279 22068 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.138159 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:08:02.138477 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/master-0-root
uuid: "c0557a4af02b4c25b02595b71c60436b"
format_stamp: "Formatted at 2026-07-09 13:08:02 on dist-test-slave-zhdc"
I20260709 13:08:02.138751 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:02.156543 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:02.158011 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:02.203668 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.126:37329
I20260709 13:08:02.203773 22119 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.126:37329 every 8 connection(s)
I20260709 13:08:02.208570 22120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:02.221454 22120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b: Bootstrap starting.
I20260709 13:08:02.227408 22120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:02.232105 22120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b: No bootstrap required, opened a new log
I20260709 13:08:02.234663 22120 raft_consensus.cc:359] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0557a4af02b4c25b02595b71c60436b" member_type: VOTER }
I20260709 13:08:02.235157 22120 raft_consensus.cc:385] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:02.235344 22120 raft_consensus.cc:740] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0557a4af02b4c25b02595b71c60436b, State: Initialized, Role: FOLLOWER
I20260709 13:08:02.235868 22120 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0557a4af02b4c25b02595b71c60436b" member_type: VOTER }
I20260709 13:08:02.236322 22120 raft_consensus.cc:399] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:08:02.236533 22120 raft_consensus.cc:493] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:08:02.236753 22120 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:02.242007 22120 raft_consensus.cc:515] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0557a4af02b4c25b02595b71c60436b" member_type: VOTER }
I20260709 13:08:02.242578 22120 leader_election.cc:304] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0557a4af02b4c25b02595b71c60436b; no voters: 
I20260709 13:08:02.243794 22120 leader_election.cc:290] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:08:02.244180 22123 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:02.245739 22123 raft_consensus.cc:697] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [term 1 LEADER]: Becoming Leader. State: Replica: c0557a4af02b4c25b02595b71c60436b, State: Running, Role: LEADER
I20260709 13:08:02.246521 22123 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0557a4af02b4c25b02595b71c60436b" member_type: VOTER }
I20260709 13:08:02.247167 22120 sys_catalog.cc:565] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:08:02.253826 22124 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0557a4af02b4c25b02595b71c60436b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0557a4af02b4c25b02595b71c60436b" member_type: VOTER } }
I20260709 13:08:02.254690 22124 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:02.253839 22125 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0557a4af02b4c25b02595b71c60436b. Latest consensus state: current_term: 1 leader_uuid: "c0557a4af02b4c25b02595b71c60436b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0557a4af02b4c25b02595b71c60436b" member_type: VOTER } }
I20260709 13:08:02.256299 22125 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:02.261348 22131 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:08:02.266865 22131 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:08:02.269425 12725 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:08:02.276562 22131 catalog_manager.cc:1383] Generated new cluster ID: a394914a73df4fbc9587822176353707
I20260709 13:08:02.276883 22131 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:08:02.297403 22131 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:08:02.298933 22131 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:08:02.311604 22131 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b: Generated new TSK 0
I20260709 13:08:02.312264 22131 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:08:02.336524 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:02.343930 22141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:02.345129 22142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:02.346825 22144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:02.347640 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:02.350387 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:02.350595 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:02.350741 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602482350724 us; error 0 us; skew 500 ppm
I20260709 13:08:02.351265 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:02.353765 12725 webserver.cc:533] Webserver started at http://127.12.109.65:41687/ using document root <none> and password file <none>
I20260709 13:08:02.354287 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:02.354457 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:02.354681 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:02.355985 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/instance:
uuid: "652d70a0b765447792ab9b4383712607"
format_stamp: "Formatted at 2026-07-09 13:08:02 on dist-test-slave-zhdc"
I20260709 13:08:02.361171 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:08:02.365054 22149 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.365994 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:08:02.366451 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-0-root
uuid: "652d70a0b765447792ab9b4383712607"
format_stamp: "Formatted at 2026-07-09 13:08:02 on dist-test-slave-zhdc"
I20260709 13:08:02.366720 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:02.383067 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:02.384493 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:02.386300 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:02.388922 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:02.389163 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.389436 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:02.389635 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.439850 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.65:46399
I20260709 13:08:02.439958 22211 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.65:46399 every 8 connection(s)
I20260709 13:08:02.446173 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:02.455120 22216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:02.457417 22217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:02.458650 22219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:02.461215 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:02.462284 22212 heartbeater.cc:344] Connected to a master server at 127.12.109.126:37329
I20260709 13:08:02.462800 22212 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:02.464020 22212 heartbeater.cc:507] Master 127.12.109.126:37329 requested a full tablet report, sending...
I20260709 13:08:02.465131 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:02.465410 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:02.465629 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602482465604 us; error 0 us; skew 500 ppm
I20260709 13:08:02.466346 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:02.466771 22085 ts_manager.cc:194] Registered new tserver with Master: 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:02.469300 22085 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47330
I20260709 13:08:02.469452 12725 webserver.cc:533] Webserver started at http://127.12.109.66:34471/ using document root <none> and password file <none>
I20260709 13:08:02.470162 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:02.470391 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:02.470752 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:02.472245 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-1-root/instance:
uuid: "bb26db1bd5a64429848b5d63b157c6f0"
format_stamp: "Formatted at 2026-07-09 13:08:02 on dist-test-slave-zhdc"
I20260709 13:08:02.477276 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:08:02.481365 22224 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.482447 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 13:08:02.482769 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-1-root
uuid: "bb26db1bd5a64429848b5d63b157c6f0"
format_stamp: "Formatted at 2026-07-09 13:08:02 on dist-test-slave-zhdc"
I20260709 13:08:02.483068 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:02.514617 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:02.516043 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:02.517702 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:02.520226 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:02.520458 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.520709 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:02.520890 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.568034 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.66:37225
I20260709 13:08:02.568157 22286 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.66:37225 every 8 connection(s)
I20260709 13:08:02.574028 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:02.581588 22290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:02.583652 22291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:02.585457 22293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:02.586992 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:02.587955 22287 heartbeater.cc:344] Connected to a master server at 127.12.109.126:37329
I20260709 13:08:02.588104 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:08:02.588409 22287 heartbeater.cc:461] Registering TS with master...
W20260709 13:08:02.588446 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:02.588819 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602482588798 us; error 0 us; skew 500 ppm
I20260709 13:08:02.589473 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:02.589504 22287 heartbeater.cc:507] Master 127.12.109.126:37329 requested a full tablet report, sending...
I20260709 13:08:02.592046 22085 ts_manager.cc:194] Registered new tserver with Master: bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:02.592475 12725 webserver.cc:533] Webserver started at http://127.12.109.67:46093/ using document root <none> and password file <none>
I20260709 13:08:02.593042 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:02.593259 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:02.593549 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:02.593772 22085 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47332
I20260709 13:08:02.595263 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-2-root/instance:
uuid: "93815010a884438c86fc6479359b69e8"
format_stamp: "Formatted at 2026-07-09 13:08:02 on dist-test-slave-zhdc"
I20260709 13:08:02.600227 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:08:02.603868 22298 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.604696 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:08:02.605023 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-2-root
uuid: "93815010a884438c86fc6479359b69e8"
format_stamp: "Formatted at 2026-07-09 13:08:02 on dist-test-slave-zhdc"
I20260709 13:08:02.605441 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:02.619310 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:02.620672 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:02.622392 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:02.624931 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:02.625175 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.625449 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:02.625643 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.674770 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.67:34889
I20260709 13:08:02.674891 22360 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.67:34889 every 8 connection(s)
I20260709 13:08:02.680320 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:02.690026 22364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:02.692909 22365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:02.693958 22367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:02.694238 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:02.694615 22361 heartbeater.cc:344] Connected to a master server at 127.12.109.126:37329
I20260709 13:08:02.695196 22361 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:02.695881 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:02.696239 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:02.696389 22361 heartbeater.cc:507] Master 127.12.109.126:37329 requested a full tablet report, sending...
I20260709 13:08:02.696499 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602482696476 us; error 0 us; skew 500 ppm
I20260709 13:08:02.697513 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:02.699098 22085 ts_manager.cc:194] Registered new tserver with Master: 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:02.700821 22085 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47344
I20260709 13:08:02.700911 12725 webserver.cc:533] Webserver started at http://127.12.109.68:43739/ using document root <none> and password file <none>
I20260709 13:08:02.701500 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:02.701783 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:02.702206 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:02.703977 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-3-root/instance:
uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
format_stamp: "Formatted at 2026-07-09 13:08:02 on dist-test-slave-zhdc"
I20260709 13:08:02.708803 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.005s
I20260709 13:08:02.712352 22372 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.713215 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:08:02.713523 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-3-root
uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
format_stamp: "Formatted at 2026-07-09 13:08:02 on dist-test-slave-zhdc"
I20260709 13:08:02.713847 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:02.740790 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:02.742327 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:02.743723 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:02.746325 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:02.746564 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.746819 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:02.747000 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:02.806329 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.68:41261
I20260709 13:08:02.806470 22434 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.68:41261 every 8 connection(s)
I20260709 13:08:02.823215 22435 heartbeater.cc:344] Connected to a master server at 127.12.109.126:37329
I20260709 13:08:02.823740 22435 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:02.824625 22435 heartbeater.cc:507] Master 127.12.109.126:37329 requested a full tablet report, sending...
I20260709 13:08:02.826714 22085 ts_manager.cc:194] Registered new tserver with Master: 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:02.827664 12725 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.017374525s
I20260709 13:08:02.828495 22085 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47360
I20260709 13:08:02.856945 22085 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:08:03.123805 22400 tablet_service.cc:1511] Processing CreateTablet for tablet a39127728d4e48a9aeb4e71e4c7e1518 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:08:03.125602 22400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a39127728d4e48a9aeb4e71e4c7e1518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.130930 22252 tablet_service.cc:1511] Processing CreateTablet for tablet fcf4b5402e45470c81a3389d815a9286 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:08:03.132757 22252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcf4b5402e45470c81a3389d815a9286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.135516 22251 tablet_service.cc:1511] Processing CreateTablet for tablet a39127728d4e48a9aeb4e71e4c7e1518 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:08:03.137162 22251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a39127728d4e48a9aeb4e71e4c7e1518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.139411 22250 tablet_service.cc:1511] Processing CreateTablet for tablet c87912ee2e4f46f1b0eb4e574255f31e (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:08:03.141074 22250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c87912ee2e4f46f1b0eb4e574255f31e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.143568 22249 tablet_service.cc:1511] Processing CreateTablet for tablet d3a61ae21bab486cbfa407942fdeac35 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:08:03.145224 22395 tablet_service.cc:1511] Processing CreateTablet for tablet 5106df2a56c74793aad5538c83a79f61 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:08:03.146971 22395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5106df2a56c74793aad5538c83a79f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.146677 22399 tablet_service.cc:1511] Processing CreateTablet for tablet c995e1a8ce9c48aab8235f258f77d080 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:08:03.148186 22399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c995e1a8ce9c48aab8235f258f77d080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.151552 22394 tablet_service.cc:1511] Processing CreateTablet for tablet 0d86dab181da42f79842cbdbd4fa5b05 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:08:03.153115 22394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d86dab181da42f79842cbdbd4fa5b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.152998 22397 tablet_service.cc:1511] Processing CreateTablet for tablet 878ae1fea9154e7088fdba8930c5eb23 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:08:03.151579 22396 tablet_service.cc:1511] Processing CreateTablet for tablet 814c209a7aff4b11bdc62505cc29c286 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:08:03.149878 22398 tablet_service.cc:1511] Processing CreateTablet for tablet c87912ee2e4f46f1b0eb4e574255f31e (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:08:03.154845 22397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 878ae1fea9154e7088fdba8930c5eb23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.156888 22398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c87912ee2e4f46f1b0eb4e574255f31e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.158367 22393 tablet_service.cc:1511] Processing CreateTablet for tablet a5e8f1fb740446b8b42c60afd2e7c3eb (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:08:03.159931 22393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5e8f1fb740446b8b42c60afd2e7c3eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.160072 22248 tablet_service.cc:1511] Processing CreateTablet for tablet 878ae1fea9154e7088fdba8930c5eb23 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:08:03.145274 22249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3a61ae21bab486cbfa407942fdeac35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.161620 22248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 878ae1fea9154e7088fdba8930c5eb23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.160637 22396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 814c209a7aff4b11bdc62505cc29c286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.163386 22247 tablet_service.cc:1511] Processing CreateTablet for tablet 5106df2a56c74793aad5538c83a79f61 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:08:03.165905 22246 tablet_service.cc:1511] Processing CreateTablet for tablet 86f9732bc4d24ee3a8e261358eba3a21 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:08:03.168653 22245 tablet_service.cc:1511] Processing CreateTablet for tablet 0d86dab181da42f79842cbdbd4fa5b05 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:08:03.178573 22247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5106df2a56c74793aad5538c83a79f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.185985 22246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86f9732bc4d24ee3a8e261358eba3a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.190624 22245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d86dab181da42f79842cbdbd4fa5b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.198515 22244 tablet_service.cc:1511] Processing CreateTablet for tablet 2076c24c8d1d47e6b3e4ece63fae1597 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:08:03.200735 22244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2076c24c8d1d47e6b3e4ece63fae1597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.201083 22326 tablet_service.cc:1511] Processing CreateTablet for tablet fcf4b5402e45470c81a3389d815a9286 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:08:03.202756 22326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcf4b5402e45470c81a3389d815a9286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.214313 22325 tablet_service.cc:1511] Processing CreateTablet for tablet c995e1a8ce9c48aab8235f258f77d080 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:08:03.216060 22325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c995e1a8ce9c48aab8235f258f77d080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.220135 22324 tablet_service.cc:1511] Processing CreateTablet for tablet c87912ee2e4f46f1b0eb4e574255f31e (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:08:03.221772 22324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c87912ee2e4f46f1b0eb4e574255f31e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.223984 22323 tablet_service.cc:1511] Processing CreateTablet for tablet d3a61ae21bab486cbfa407942fdeac35 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:08:03.230490 22323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3a61ae21bab486cbfa407942fdeac35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.244194 22322 tablet_service.cc:1511] Processing CreateTablet for tablet 814c209a7aff4b11bdc62505cc29c286 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:08:03.247874 22321 tablet_service.cc:1511] Processing CreateTablet for tablet 5106df2a56c74793aad5538c83a79f61 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:08:03.248474 22320 tablet_service.cc:1511] Processing CreateTablet for tablet 86f9732bc4d24ee3a8e261358eba3a21 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:08:03.251848 22319 tablet_service.cc:1511] Processing CreateTablet for tablet a5e8f1fb740446b8b42c60afd2e7c3eb (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:08:03.245954 22322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 814c209a7aff4b11bdc62505cc29c286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.256707 22170 tablet_service.cc:1511] Processing CreateTablet for tablet 0d86dab181da42f79842cbdbd4fa5b05 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:08:03.257042 22177 tablet_service.cc:1511] Processing CreateTablet for tablet fcf4b5402e45470c81a3389d815a9286 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:08:03.258692 22320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86f9732bc4d24ee3a8e261358eba3a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.259078 22176 tablet_service.cc:1511] Processing CreateTablet for tablet a39127728d4e48a9aeb4e71e4c7e1518 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:08:03.259392 22169 tablet_service.cc:1511] Processing CreateTablet for tablet a5e8f1fb740446b8b42c60afd2e7c3eb (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:08:03.260520 22170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d86dab181da42f79842cbdbd4fa5b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.261359 22175 tablet_service.cc:1511] Processing CreateTablet for tablet c995e1a8ce9c48aab8235f258f77d080 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:08:03.262738 22319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5e8f1fb740446b8b42c60afd2e7c3eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.263604 22174 tablet_service.cc:1511] Processing CreateTablet for tablet d3a61ae21bab486cbfa407942fdeac35 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:08:03.265796 22173 tablet_service.cc:1511] Processing CreateTablet for tablet 878ae1fea9154e7088fdba8930c5eb23 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:08:03.268910 22171 tablet_service.cc:1511] Processing CreateTablet for tablet 86f9732bc4d24ee3a8e261358eba3a21 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:08:03.269155 22172 tablet_service.cc:1511] Processing CreateTablet for tablet 814c209a7aff4b11bdc62505cc29c286 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
W20260709 13:08:03.272903 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:03.273707 22321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5106df2a56c74793aad5538c83a79f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.275714 22318 tablet_service.cc:1511] Processing CreateTablet for tablet 2076c24c8d1d47e6b3e4ece63fae1597 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:08:03.277307 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2076c24c8d1d47e6b3e4ece63fae1597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.278733 22176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a39127728d4e48a9aeb4e71e4c7e1518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.292513 22455 tablet_bootstrap.cc:492] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:03.292853 22169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5e8f1fb740446b8b42c60afd2e7c3eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.306890 22173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 878ae1fea9154e7088fdba8930c5eb23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.310376 22172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 814c209a7aff4b11bdc62505cc29c286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.314440 22177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcf4b5402e45470c81a3389d815a9286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.316869 22175 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c995e1a8ce9c48aab8235f258f77d080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.318693 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3a61ae21bab486cbfa407942fdeac35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.333599 22171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86f9732bc4d24ee3a8e261358eba3a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.367762 22397 tablet_service.cc:1511] Processing CreateTablet for tablet 2076c24c8d1d47e6b3e4ece63fae1597 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:08:03.369350 22397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2076c24c8d1d47e6b3e4ece63fae1597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.384833 22455 tablet_bootstrap.cc:654] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:03.416694 22458 tablet_bootstrap.cc:492] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:03.416890 22170 tablet_service.cc:1511] Processing CreateTablet for tablet 3a4c4a9cd1164b7997f7188f9da9f70b (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:08:03.419293 22170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a4c4a9cd1164b7997f7188f9da9f70b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.427222 22456 tablet_bootstrap.cc:492] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:03.455399 22457 tablet_bootstrap.cc:492] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:03.456507 22458 tablet_bootstrap.cc:654] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:03.459811 22456 tablet_bootstrap.cc:654] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:03.466167 22249 tablet_service.cc:1511] Processing CreateTablet for tablet 3a4c4a9cd1164b7997f7188f9da9f70b (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:08:03.467036 22457 tablet_bootstrap.cc:654] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:03.472433 22212 heartbeater.cc:499] Master 127.12.109.126:37329 was elected leader, sending a full tablet report...
I20260709 13:08:03.476339 22249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a4c4a9cd1164b7997f7188f9da9f70b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.483908 22323 tablet_service.cc:1511] Processing CreateTablet for tablet 3a4c4a9cd1164b7997f7188f9da9f70b (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:08:03.499243 22455 tablet_bootstrap.cc:492] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:03.499909 22455 ts_tablet_manager.cc:1403] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.208s	user 0.035s	sys 0.027s
I20260709 13:08:03.512375 22399 tablet_service.cc:1511] Processing CreateTablet for tablet ade9dda3178e42c783bc93e5deec21b7 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:08:03.514017 22399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade9dda3178e42c783bc93e5deec21b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.516813 22458 tablet_bootstrap.cc:492] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:03.517498 22458 ts_tablet_manager.cc:1403] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.101s	user 0.022s	sys 0.019s
I20260709 13:08:03.523272 22458 raft_consensus.cc:359] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:03.524293 22458 raft_consensus.cc:385] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:03.524690 22458 raft_consensus.cc:740] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:03.531924 22323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a4c4a9cd1164b7997f7188f9da9f70b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.539852 22456 tablet_bootstrap.cc:492] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:03.540586 22456 ts_tablet_manager.cc:1403] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.114s	user 0.018s	sys 0.016s
I20260709 13:08:03.540167 22458 consensus_queue.cc:260] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:03.542527 22455 raft_consensus.cc:359] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:03.543331 22455 raft_consensus.cc:385] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:03.543622 22455 raft_consensus.cc:740] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:03.546309 22456 raft_consensus.cc:359] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:03.549738 22456 raft_consensus.cc:385] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:03.550117 22456 raft_consensus.cc:740] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:03.551151 22456 consensus_queue.cc:260] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:03.551393 22455 consensus_queue.cc:260] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:03.556694 22248 tablet_service.cc:1511] Processing CreateTablet for tablet ade9dda3178e42c783bc93e5deec21b7 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:08:03.558207 22248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade9dda3178e42c783bc93e5deec21b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.559475 22457 tablet_bootstrap.cc:492] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:03.560056 22457 ts_tablet_manager.cc:1403] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.105s	user 0.014s	sys 0.018s
I20260709 13:08:03.562286 22171 tablet_service.cc:1511] Processing CreateTablet for tablet ade9dda3178e42c783bc93e5deec21b7 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:08:03.564200 22457 raft_consensus.cc:359] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:03.567700 22457 raft_consensus.cc:385] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:03.568300 22457 raft_consensus.cc:740] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:03.566663 22321 tablet_service.cc:1511] Processing CreateTablet for tablet cf086b080a8c46ff809e2b9a14bdacae (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:08:03.570487 22321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf086b080a8c46ff809e2b9a14bdacae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.572149 22171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade9dda3178e42c783bc93e5deec21b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.578586 22396 tablet_service.cc:1511] Processing CreateTablet for tablet cf086b080a8c46ff809e2b9a14bdacae (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:08:03.580525 22396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf086b080a8c46ff809e2b9a14bdacae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.595196 22173 tablet_service.cc:1511] Processing CreateTablet for tablet cf086b080a8c46ff809e2b9a14bdacae (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:08:03.597150 22287 heartbeater.cc:499] Master 127.12.109.126:37329 was elected leader, sending a full tablet report...
I20260709 13:08:03.597332 22251 tablet_service.cc:1511] Processing CreateTablet for tablet 6cadc140203246b088e6e2883bbe1977 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:08:03.598265 22173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf086b080a8c46ff809e2b9a14bdacae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.598841 22251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cadc140203246b088e6e2883bbe1977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.606526 22463 raft_consensus.cc:493] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.607144 22322 tablet_service.cc:1511] Processing CreateTablet for tablet 6cadc140203246b088e6e2883bbe1977 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:08:03.607105 22463 raft_consensus.cc:515] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:03.609700 22456 ts_tablet_manager.cc:1434] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.069s	user 0.010s	sys 0.000s
I20260709 13:08:03.609845 22322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cadc140203246b088e6e2883bbe1977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.610270 22463 leader_election.cc:290] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:03.610724 22361 heartbeater.cc:499] Master 127.12.109.126:37329 was elected leader, sending a full tablet report...
I20260709 13:08:03.611202 22456 tablet_bootstrap.cc:492] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:03.578500 22457 consensus_queue.cc:260] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:03.626168 22393 tablet_service.cc:1511] Processing CreateTablet for tablet 6cadc140203246b088e6e2883bbe1977 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:08:03.634428 22393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cadc140203246b088e6e2883bbe1977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.635416 22177 tablet_service.cc:1511] Processing CreateTablet for tablet 2144e27a177e41b0b22e474d9f209102 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:08:03.637267 22177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2144e27a177e41b0b22e474d9f209102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.650476 22458 ts_tablet_manager.cc:1434] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.133s	user 0.027s	sys 0.019s
I20260709 13:08:03.660419 22249 tablet_service.cc:1511] Processing CreateTablet for tablet 2144e27a177e41b0b22e474d9f209102 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:08:03.661585 22435 heartbeater.cc:499] Master 127.12.109.126:37329 was elected leader, sending a full tablet report...
I20260709 13:08:03.662045 22249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2144e27a177e41b0b22e474d9f209102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.674876 22455 ts_tablet_manager.cc:1434] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.175s	user 0.013s	sys 0.005s
W20260709 13:08:03.684224 22362 tablet.cc:2406] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:08:03.684914 22458 tablet_bootstrap.cc:492] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:03.685979 22456 tablet_bootstrap.cc:654] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:03.693540 22399 tablet_service.cc:1511] Processing CreateTablet for tablet c0b51a3b1dc444319751dd92054d5dc3 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:08:03.695220 22399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b51a3b1dc444319751dd92054d5dc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.695068 22320 tablet_service.cc:1511] Processing CreateTablet for tablet 2144e27a177e41b0b22e474d9f209102 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:08:03.696480 22320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2144e27a177e41b0b22e474d9f209102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.701283 22170 tablet_service.cc:1511] Processing CreateTablet for tablet c0b51a3b1dc444319751dd92054d5dc3 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:08:03.702253 22455 tablet_bootstrap.cc:492] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:03.702773 22170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b51a3b1dc444319751dd92054d5dc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.705978 22251 tablet_service.cc:1511] Processing CreateTablet for tablet c0b51a3b1dc444319751dd92054d5dc3 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:08:03.707466 22251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b51a3b1dc444319751dd92054d5dc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.713310 22318 tablet_service.cc:1511] Processing CreateTablet for tablet 691ad2369e00484995e87d12f574dfdc (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:08:03.714758 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 691ad2369e00484995e87d12f574dfdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.715265 22397 tablet_service.cc:1511] Processing CreateTablet for tablet 691ad2369e00484995e87d12f574dfdc (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:08:03.716657 22397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 691ad2369e00484995e87d12f574dfdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.726994 22464 raft_consensus.cc:493] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.727632 22464 raft_consensus.cc:515] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:03.739542 22457 ts_tablet_manager.cc:1434] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.179s	user 0.008s	sys 0.032s
I20260709 13:08:03.758469 22457 tablet_bootstrap.cc:492] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:03.753141 22455 tablet_bootstrap.cc:654] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:03.756567 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:03.732810 22464 leader_election.cc:290] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
W20260709 13:08:03.764824 22151 leader_election.cc:343] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.767746 22469 raft_consensus.cc:493] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.768688 22469 raft_consensus.cc:515] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:03.772338 22469 leader_election.cc:290] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:03.787762 22458 tablet_bootstrap.cc:654] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:03.795941 22173 tablet_service.cc:1511] Processing CreateTablet for tablet 691ad2369e00484995e87d12f574dfdc (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:08:03.802876 22173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 691ad2369e00484995e87d12f574dfdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.804836 22455 tablet_bootstrap.cc:492] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:03.805610 22455 ts_tablet_manager.cc:1403] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.104s	user 0.004s	sys 0.009s
I20260709 13:08:03.813269 22248 tablet_service.cc:1511] Processing CreateTablet for tablet 01b83067f9fa498487aeed4f60e8bfde (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:08:03.814728 22248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01b83067f9fa498487aeed4f60e8bfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.818943 22396 tablet_service.cc:1511] Processing CreateTablet for tablet 01b83067f9fa498487aeed4f60e8bfde (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:08:03.820426 22396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01b83067f9fa498487aeed4f60e8bfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.824031 22457 tablet_bootstrap.cc:654] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:03.823629 22322 tablet_service.cc:1511] Processing CreateTablet for tablet 01b83067f9fa498487aeed4f60e8bfde (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:08:03.825099 22322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01b83067f9fa498487aeed4f60e8bfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.825670 22323 tablet_service.cc:1511] Processing CreateTablet for tablet f7f82cbc4e8544bbac74ba96797d687c (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:08:03.827178 22323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7f82cbc4e8544bbac74ba96797d687c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.825750 22244 tablet_service.cc:1511] Processing CreateTablet for tablet f7f82cbc4e8544bbac74ba96797d687c (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:08:03.829674 22244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7f82cbc4e8544bbac74ba96797d687c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.830127 22177 tablet_service.cc:1511] Processing CreateTablet for tablet f7f82cbc4e8544bbac74ba96797d687c (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:08:03.847586 22456 tablet_bootstrap.cc:492] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:03.847296 22455 raft_consensus.cc:359] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:03.848385 22456 ts_tablet_manager.cc:1403] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.237s	user 0.015s	sys 0.017s
I20260709 13:08:03.848644 22455 raft_consensus.cc:385] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:03.849058 22455 raft_consensus.cc:740] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:03.849984 22455 consensus_queue.cc:260] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:03.859150 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcf4b5402e45470c81a3389d815a9286" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:03.860164 22262 raft_consensus.cc:2468] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 0.
I20260709 13:08:03.861644 22397 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad7895d69404083ba56a16062a8ed20 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:08:03.863152 22397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad7895d69404083ba56a16062a8ed20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.865240 22249 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad7895d69404083ba56a16062a8ed20 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:08:03.866154 22171 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad7895d69404083ba56a16062a8ed20 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:08:03.866777 22249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad7895d69404083ba56a16062a8ed20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.831465 22177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7f82cbc4e8544bbac74ba96797d687c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.871075 22320 tablet_service.cc:1511] Processing CreateTablet for tablet ec8f7472c97049ca98e597c7d4b4eb1c (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:08:03.872366 22320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec8f7472c97049ca98e597c7d4b4eb1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.861672 22300 leader_election.cc:304] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:03.873518 22456 raft_consensus.cc:359] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:03.883083 22456 raft_consensus.cc:385] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:03.883322 22464 raft_consensus.cc:2804] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:03.884698 22456 raft_consensus.cc:740] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:03.890980 22456 consensus_queue.cc:260] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:03.885874 22464 raft_consensus.cc:493] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.893666 22171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad7895d69404083ba56a16062a8ed20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.893970 22464 raft_consensus.cc:3060] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:03.898262 22455 ts_tablet_manager.cc:1434] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.092s	user 0.011s	sys 0.001s
I20260709 13:08:03.899605 22455 tablet_bootstrap.cc:492] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:03.901854 22393 tablet_service.cc:1511] Processing CreateTablet for tablet ec8f7472c97049ca98e597c7d4b4eb1c (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:08:03.903401 22393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec8f7472c97049ca98e597c7d4b4eb1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.904984 22170 tablet_service.cc:1511] Processing CreateTablet for tablet ec8f7472c97049ca98e597c7d4b4eb1c (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:08:03.911356 22245 tablet_service.cc:1511] Processing CreateTablet for tablet c9cc1b69991141439a4ee9459859b4a6 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:08:03.912777 22245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9cc1b69991141439a4ee9459859b4a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.914558 22169 tablet_service.cc:1511] Processing CreateTablet for tablet 01099e972e9b4d08a64552be3fd52276 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:08:03.920030 22395 tablet_service.cc:1511] Processing CreateTablet for tablet c9cc1b69991141439a4ee9459859b4a6 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:08:03.921418 22395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9cc1b69991141439a4ee9459859b4a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.921751 22170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec8f7472c97049ca98e597c7d4b4eb1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.922468 22169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01099e972e9b4d08a64552be3fd52276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.947197 22322 tablet_service.cc:1511] Processing CreateTablet for tablet c9cc1b69991141439a4ee9459859b4a6 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:08:03.949018 22322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9cc1b69991141439a4ee9459859b4a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.951138 22455 tablet_bootstrap.cc:654] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:03.967311 22187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcf4b5402e45470c81a3389d815a9286" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:03.969754 22252 tablet_service.cc:1511] Processing CreateTablet for tablet 01099e972e9b4d08a64552be3fd52276 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:08:03.971331 22252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01099e972e9b4d08a64552be3fd52276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.974790 22464 raft_consensus.cc:515] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:03.977173 22464 leader_election.cc:290] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:03.978796 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcf4b5402e45470c81a3389d815a9286" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:03.979794 22262 raft_consensus.cc:3060] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:03.982540 22400 tablet_service.cc:1511] Processing CreateTablet for tablet 4f204603e14c438bb549d1116eaac788 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:08:03.984064 22400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f204603e14c438bb549d1116eaac788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:03.991192 22465 raft_consensus.cc:493] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.992077 22465 raft_consensus.cc:515] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:03.994656 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
W20260709 13:08:03.999274 22151 leader_election.cc:343] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.999887 22465 leader_election.cc:290] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:03.999819 22151 leader_election.cc:304] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:04.015472 22463 raft_consensus.cc:2749] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:04.007212 22187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcf4b5402e45470c81a3389d815a9286" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:04.026773 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcf4b5402e45470c81a3389d815a9286" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
W20260709 13:08:04.032639 22227 leader_election.cc:343] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.034552 22248 tablet_service.cc:1511] Processing CreateTablet for tablet 4f204603e14c438bb549d1116eaac788 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:08:04.035400 22173 tablet_service.cc:1511] Processing CreateTablet for tablet 4f204603e14c438bb549d1116eaac788 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:08:04.036139 22248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f204603e14c438bb549d1116eaac788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.036798 22173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f204603e14c438bb549d1116eaac788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.070009 22318 tablet_service.cc:1511] Processing CreateTablet for tablet 01099e972e9b4d08a64552be3fd52276 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:08:04.072162 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01099e972e9b4d08a64552be3fd52276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.075981 22457 tablet_bootstrap.cc:492] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:04.076325 22458 tablet_bootstrap.cc:492] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:04.076633 22457 ts_tablet_manager.cc:1403] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.318s	user 0.009s	sys 0.008s
I20260709 13:08:04.077133 22458 ts_tablet_manager.cc:1403] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.392s	user 0.032s	sys 0.044s
I20260709 13:08:04.078156 22262 raft_consensus.cc:2468] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:04.081818 22300 leader_election.cc:304] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:04.083563 22464 raft_consensus.cc:2804] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:04.084234 22464 raft_consensus.cc:697] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
I20260709 13:08:04.085343 22464 consensus_queue.cc:237] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:04.086344 22301 leader_election.cc:343] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:04.087666 22301 leader_election.cc:343] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.081390 22458 raft_consensus.cc:359] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.091097 22458 raft_consensus.cc:385] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.097014 22458 raft_consensus.cc:740] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.098073 22458 consensus_queue.cc:260] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.102559 22455 tablet_bootstrap.cc:492] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:04.103247 22455 ts_tablet_manager.cc:1403] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.204s	user 0.012s	sys 0.015s
I20260709 13:08:04.105286 22481 raft_consensus.cc:493] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.105840 22481 raft_consensus.cc:515] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:04.108112 22455 raft_consensus.cc:359] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.109432 22455 raft_consensus.cc:385] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.109846 22455 raft_consensus.cc:740] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.110807 22455 consensus_queue.cc:260] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.108489 22481 leader_election.cc:290] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:04.118405 22455 ts_tablet_manager.cc:1434] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:08:04.119022 22465 raft_consensus.cc:493] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.121301 22465 raft_consensus.cc:515] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.122354 22251 tablet_service.cc:1511] Processing CreateTablet for tablet 1cd9f843a01847a4b475b6dbcb704a6f (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:08:04.123769 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e8f1fb740446b8b42c60afd2e7c3eb" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:04.123972 22465 leader_election.cc:290] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:04.112792 22249 tablet_service.cc:1511] Processing CreateTablet for tablet 7f757dd6e4b94a8a84bfaddd602685e4 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
W20260709 13:08:04.125494 22300 leader_election.cc:343] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.126861 22455 tablet_bootstrap.cc:492] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:04.127099 22171 tablet_service.cc:1511] Processing CreateTablet for tablet 5e95f83f20904ff692e70a1872c3e77a (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:08:04.128894 22171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e95f83f20904ff692e70a1872c3e77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.129200 22177 tablet_service.cc:1511] Processing CreateTablet for tablet 1cd9f843a01847a4b475b6dbcb704a6f (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:08:04.130869 22463 raft_consensus.cc:493] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.131448 22463 raft_consensus.cc:515] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.132532 22174 tablet_service.cc:1511] Processing CreateTablet for tablet fd464ed1f28f42ccad10b0df649c7ce7 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:08:04.133713 22463 leader_election.cc:290] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:04.135702 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
W20260709 13:08:04.137935 22151 leader_election.cc:343] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.138350 22458 ts_tablet_manager.cc:1434] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.061s	user 0.010s	sys 0.002s
I20260709 13:08:04.139153 22177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cd9f843a01847a4b475b6dbcb704a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.136155 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:04.139859 22410 raft_consensus.cc:2468] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:04.141608 22151 leader_election.cc:304] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:04.143045 22463 raft_consensus.cc:2804] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:04.143467 22463 raft_consensus.cc:493] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.143908 22463 raft_consensus.cc:3060] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.149107 22244 tablet_service.cc:1511] Processing CreateTablet for tablet fd464ed1f28f42ccad10b0df649c7ce7 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:08:04.152993 22455 tablet_bootstrap.cc:654] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.152742 22463 raft_consensus.cc:515] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.172763 22463 leader_election.cc:290] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:04.154598 22457 raft_consensus.cc:359] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.156323 22456 ts_tablet_manager.cc:1434] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.307s	user 0.029s	sys 0.007s
I20260709 13:08:04.154641 22251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cd9f843a01847a4b475b6dbcb704a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.157140 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd464ed1f28f42ccad10b0df649c7ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.186009 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:04.187427 22456 tablet_bootstrap.cc:492] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:04.186120 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87912ee2e4f46f1b0eb4e574255f31e" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
W20260709 13:08:04.189186 22374 leader_election.cc:343] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:04.190003 22374 leader_election.cc:343] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.186496 22244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd464ed1f28f42ccad10b0df649c7ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.174506 22457 raft_consensus.cc:385] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.194540 22249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f757dd6e4b94a8a84bfaddd602685e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.194626 22457 raft_consensus.cc:740] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.184092 22455 tablet_bootstrap.cc:492] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:04.203179 22455 ts_tablet_manager.cc:1403] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.077s	user 0.015s	sys 0.006s
I20260709 13:08:04.202394 22457 consensus_queue.cc:260] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.208719 22457 ts_tablet_manager.cc:1434] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.132s	user 0.012s	sys 0.000s
I20260709 13:08:04.209888 22457 tablet_bootstrap.cc:492] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:04.215497 22458 tablet_bootstrap.cc:492] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:04.217054 22455 raft_consensus.cc:359] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.217870 22455 raft_consensus.cc:385] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.218269 22455 raft_consensus.cc:740] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.222935 22456 tablet_bootstrap.cc:654] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.227173 22395 tablet_service.cc:1511] Processing CreateTablet for tablet 5e95f83f20904ff692e70a1872c3e77a (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:08:04.228655 22395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e95f83f20904ff692e70a1872c3e77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.252462 22455 consensus_queue.cc:260] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.266917 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:04.271143 22410 raft_consensus.cc:3060] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.275739 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:04.276916 22323 tablet_service.cc:1511] Processing CreateTablet for tablet 7f757dd6e4b94a8a84bfaddd602685e4 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
W20260709 13:08:04.277479 22151 leader_election.cc:343] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.275838 22245 tablet_service.cc:1511] Processing CreateTablet for tablet 0418b53b8c2e4786b445d55f0ea5a723 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:08:04.278501 22323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f757dd6e4b94a8a84bfaddd602685e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.279152 22245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0418b53b8c2e4786b445d55f0ea5a723. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:04.284235 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:04.284075 22410 raft_consensus.cc:2468] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:04.284202 22455 ts_tablet_manager.cc:1434] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.081s	user 0.000s	sys 0.014s
I20260709 13:08:04.260783 22320 tablet_service.cc:1511] Processing CreateTablet for tablet 5e95f83f20904ff692e70a1872c3e77a (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:08:04.285805 22151 leader_election.cc:304] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:04.286545 22320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e95f83f20904ff692e70a1872c3e77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.287127 22463 raft_consensus.cc:2804] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:04.290421 22455 tablet_bootstrap.cc:492] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:04.300282 22456 tablet_bootstrap.cc:492] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:04.305212 22396 tablet_service.cc:1511] Processing CreateTablet for tablet fd464ed1f28f42ccad10b0df649c7ce7 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:08:04.306952 22396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd464ed1f28f42ccad10b0df649c7ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.307066 22322 tablet_service.cc:1511] Processing CreateTablet for tablet 1cd9f843a01847a4b475b6dbcb704a6f (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:08:04.307667 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:04.307883 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87912ee2e4f46f1b0eb4e574255f31e" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:04.308429 22322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cd9f843a01847a4b475b6dbcb704a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.305406 22397 tablet_service.cc:1511] Processing CreateTablet for tablet 40cf2d6d73fe4593b96ead779d12207e (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:08:04.307657 22456 ts_tablet_manager.cc:1403] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.120s	user 0.009s	sys 0.026s
W20260709 13:08:04.318604 22375 leader_election.cc:343] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.319474 22457 tablet_bootstrap.cc:654] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.319281 22375 leader_election.cc:304] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:04.320371 22465 raft_consensus.cc:2749] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:04.321947 22397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40cf2d6d73fe4593b96ead779d12207e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.317834 22393 tablet_service.cc:1511] Processing CreateTablet for tablet 7f757dd6e4b94a8a84bfaddd602685e4 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:08:04.326069 22174 tablet_service.cc:1511] Processing CreateTablet for tablet 40cf2d6d73fe4593b96ead779d12207e (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:08:04.331771 22321 tablet_service.cc:1511] Processing CreateTablet for tablet 40cf2d6d73fe4593b96ead779d12207e (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:08:04.333145 22463 raft_consensus.cc:697] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:04.334610 22321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40cf2d6d73fe4593b96ead779d12207e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.334764 22463 consensus_queue.cc:237] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.338838 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40cf2d6d73fe4593b96ead779d12207e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.341771 22187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e8f1fb740446b8b42c60afd2e7c3eb" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:04.338845 22393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f757dd6e4b94a8a84bfaddd602685e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.345129 22325 tablet_service.cc:1511] Processing CreateTablet for tablet 0418b53b8c2e4786b445d55f0ea5a723 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:08:04.331542 22455 tablet_bootstrap.cc:654] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.355512 22458 tablet_bootstrap.cc:654] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.355166 22456 raft_consensus.cc:359] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.356403 22456 raft_consensus.cc:385] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.356807 22456 raft_consensus.cc:740] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.358031 22400 tablet_service.cc:1511] Processing CreateTablet for tablet 0418b53b8c2e4786b445d55f0ea5a723 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:08:04.359584 22400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0418b53b8c2e4786b445d55f0ea5a723. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.364670 22171 tablet_service.cc:1511] Processing CreateTablet for tablet 214d3c5d47d848af997da7647c556b85 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:08:04.366092 22248 tablet_service.cc:1511] Processing CreateTablet for tablet 214d3c5d47d848af997da7647c556b85 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:08:04.367493 22248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214d3c5d47d848af997da7647c556b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.367537 22171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214d3c5d47d848af997da7647c556b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.357971 22456 consensus_queue.cc:260] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.371441 22456 ts_tablet_manager.cc:1434] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.001s
I20260709 13:08:04.372692 22456 tablet_bootstrap.cc:492] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:04.355417 22325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0418b53b8c2e4786b445d55f0ea5a723. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.378822 22465 raft_consensus.cc:493] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.379462 22465 raft_consensus.cc:515] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.382912 22187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "878ae1fea9154e7088fdba8930c5eb23" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:04.383299 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "878ae1fea9154e7088fdba8930c5eb23" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
W20260709 13:08:04.384755 22375 leader_election.cc:343] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:04.385459 22374 leader_election.cc:343] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.385874 22374 leader_election.cc:304] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:04.386977 22465 leader_election.cc:290] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:04.387830 22465 raft_consensus.cc:2749] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:04.400177 22375 leader_election.cc:304] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:04.401379 22465 raft_consensus.cc:2749] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:04.402036 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcf4b5402e45470c81a3389d815a9286" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:04.427975 22082 catalog_manager.cc:5697] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:04.436136 22456 tablet_bootstrap.cc:654] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.437316 22457 tablet_bootstrap.cc:492] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:04.438112 22457 ts_tablet_manager.cc:1403] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.228s	user 0.017s	sys 0.010s
W20260709 13:08:04.425160 22301 leader_election.cc:343] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.441649 22301 leader_election.cc:304] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:04.443681 22486 raft_consensus.cc:2749] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:04.444965 22227 leader_election.cc:304] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:04.446508 22475 raft_consensus.cc:2749] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:04.447165 22320 tablet_service.cc:1511] Processing CreateTablet for tablet 214d3c5d47d848af997da7647c556b85 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:08:04.448861 22320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214d3c5d47d848af997da7647c556b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.449803 22174 tablet_service.cc:1511] Processing CreateTablet for tablet fa44244e76604221bcbd08e0f3bbdb82 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:08:04.452610 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa44244e76604221bcbd08e0f3bbdb82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.453810 22318 tablet_service.cc:1511] Processing CreateTablet for tablet 96821e05d1924572a73e450bc7dcb265 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:08:04.454772 22457 raft_consensus.cc:359] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:04.455605 22457 raft_consensus.cc:385] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.455917 22457 raft_consensus.cc:740] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.452369 22252 tablet_service.cc:1511] Processing CreateTablet for tablet fa44244e76604221bcbd08e0f3bbdb82 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:08:04.457509 22252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa44244e76604221bcbd08e0f3bbdb82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.457759 22457 consensus_queue.cc:260] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:04.460259 22457 ts_tablet_manager.cc:1434] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:08:04.455267 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96821e05d1924572a73e450bc7dcb265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.433468 22396 tablet_service.cc:1511] Processing CreateTablet for tablet fa44244e76604221bcbd08e0f3bbdb82 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:08:04.469496 22396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa44244e76604221bcbd08e0f3bbdb82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.471321 22399 tablet_service.cc:1511] Processing CreateTablet for tablet 96821e05d1924572a73e450bc7dcb265 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:08:04.472712 22399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96821e05d1924572a73e450bc7dcb265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.475054 22170 tablet_service.cc:1511] Processing CreateTablet for tablet 96821e05d1924572a73e450bc7dcb265 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:08:04.476438 22170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96821e05d1924572a73e450bc7dcb265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.449193 22083 catalog_manager.cc:5697] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:04.481353 22245 tablet_service.cc:1511] Processing CreateTablet for tablet 3e49aee5d99347029a0f5e51fe01b896 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:08:04.472699 22457 tablet_bootstrap.cc:492] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:04.485069 22245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e49aee5d99347029a0f5e51fe01b896. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:04.466811 22374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 81377.2us
I20260709 13:08:04.496469 22486 raft_consensus.cc:493] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.497828 22486 raft_consensus.cc:515] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.500063 22324 tablet_service.cc:1511] Processing CreateTablet for tablet 3e49aee5d99347029a0f5e51fe01b896 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:08:04.501191 22486 leader_election.cc:290] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:04.501609 22324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e49aee5d99347029a0f5e51fe01b896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.503441 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87912ee2e4f46f1b0eb4e574255f31e" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:04.504827 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87912ee2e4f46f1b0eb4e574255f31e" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:04.505576 22410 raft_consensus.cc:2468] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 0.
W20260709 13:08:04.508229 22300 leader_election.cc:343] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.511847 22300 leader_election.cc:304] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:04.518517 22177 tablet_service.cc:1511] Processing CreateTablet for tablet ffb774f42e99437f8180b3d9e89a9cf5 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:08:04.520020 22177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffb774f42e99437f8180b3d9e89a9cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.521152 22326 tablet_service.cc:1511] Processing CreateTablet for tablet ffb774f42e99437f8180b3d9e89a9cf5 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:08:04.516908 22486 raft_consensus.cc:2804] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:04.523423 22486 raft_consensus.cc:493] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.523504 22326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffb774f42e99437f8180b3d9e89a9cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.523835 22486 raft_consensus.cc:3060] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.526872 22457 tablet_bootstrap.cc:654] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.527752 22458 tablet_bootstrap.cc:492] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:04.543877 22394 tablet_service.cc:1511] Processing CreateTablet for tablet ffdd1d227bbe48c8a46c4fefd02eab0b (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:08:04.547315 22174 tablet_service.cc:1511] Processing CreateTablet for tablet ffdd1d227bbe48c8a46c4fefd02eab0b (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:08:04.547437 22248 tablet_service.cc:1511] Processing CreateTablet for tablet ffdd1d227bbe48c8a46c4fefd02eab0b (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:08:04.548689 22252 tablet_service.cc:1511] Processing CreateTablet for tablet ffb774f42e99437f8180b3d9e89a9cf5 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:08:04.551340 22456 tablet_bootstrap.cc:492] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:04.548868 22322 tablet_service.cc:1511] Processing CreateTablet for tablet 30e6db323fdb4762a2e92b729b051111 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:08:04.557246 22458 ts_tablet_manager.cc:1403] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.342s	user 0.023s	sys 0.010s
I20260709 13:08:04.559142 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffdd1d227bbe48c8a46c4fefd02eab0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.559628 22486 raft_consensus.cc:515] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.560796 22458 raft_consensus.cc:359] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:04.561610 22458 raft_consensus.cc:385] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.562361 22394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffdd1d227bbe48c8a46c4fefd02eab0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.562450 22393 tablet_service.cc:1511] Processing CreateTablet for tablet 3e49aee5d99347029a0f5e51fe01b896 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:08:04.567654 22245 tablet_service.cc:1511] Processing CreateTablet for tablet 808e97b948be44ee895c060c4329964e (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:08:04.567966 22398 tablet_service.cc:1511] Processing CreateTablet for tablet 30e6db323fdb4762a2e92b729b051111 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:08:04.569221 22322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30e6db323fdb4762a2e92b729b051111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.569530 22398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30e6db323fdb4762a2e92b729b051111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.573177 22456 ts_tablet_manager.cc:1403] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.201s	user 0.005s	sys 0.012s
I20260709 13:08:04.561901 22458 raft_consensus.cc:740] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.582896 22248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffdd1d227bbe48c8a46c4fefd02eab0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.585286 22171 tablet_service.cc:1511] Processing CreateTablet for tablet 30e6db323fdb4762a2e92b729b051111 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:08:04.588542 22456 raft_consensus.cc:359] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:04.589376 22456 raft_consensus.cc:385] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.589687 22456 raft_consensus.cc:740] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.590438 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87912ee2e4f46f1b0eb4e574255f31e" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
W20260709 13:08:04.593881 22300 leader_election.cc:343] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:04.595801 22301 consensus_peers.cc:597] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:04.596839 22171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30e6db323fdb4762a2e92b729b051111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.606650 22494 raft_consensus.cc:493] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.607441 22494 raft_consensus.cc:515] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.612118 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:04.612793 22393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e49aee5d99347029a0f5e51fe01b896. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:04.613857 22374 leader_election.cc:343] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.614311 22475 raft_consensus.cc:493] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.614701 22494 leader_election.cc:290] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:04.614838 22475 raft_consensus.cc:515] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:04.617112 22475 leader_election.cc:290] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:04.617533 22252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffb774f42e99437f8180b3d9e89a9cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.619340 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3a61ae21bab486cbfa407942fdeac35" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
W20260709 13:08:04.621066 22227 leader_election.cc:343] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.622282 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:04.622624 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3a61ae21bab486cbfa407942fdeac35" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:04.623144 22186 raft_consensus.cc:2468] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:04.624672 22375 leader_election.cc:304] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:04.626415 22494 raft_consensus.cc:2804] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:04.630790 22494 raft_consensus.cc:493] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.641849 22475 raft_consensus.cc:493] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.638962 22458 consensus_queue.cc:260] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:04.644114 22475 raft_consensus.cc:515] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.645659 22245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 808e97b948be44ee895c060c4329964e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.638738 22494 raft_consensus.cc:3060] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.646430 22458 ts_tablet_manager.cc:1434] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.089s	user 0.005s	sys 0.004s
I20260709 13:08:04.647591 22458 tablet_bootstrap.cc:492] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:04.647740 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87912ee2e4f46f1b0eb4e574255f31e" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:04.649573 22410 raft_consensus.cc:3060] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.663563 22475 leader_election.cc:290] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:04.664448 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5106df2a56c74793aad5538c83a79f61" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:04.666391 22227 leader_election.cc:343] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.643211 22456 consensus_queue.cc:260] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:04.668665 22336 raft_consensus.cc:2468] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 0.
I20260709 13:08:04.669171 22455 tablet_bootstrap.cc:492] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:04.669828 22455 ts_tablet_manager.cc:1403] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.381s	user 0.014s	sys 0.034s
I20260709 13:08:04.671458 22456 ts_tablet_manager.cc:1434] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.090s	user 0.007s	sys 0.000s
I20260709 13:08:04.672222 22458 tablet_bootstrap.cc:654] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.672724 22456 tablet_bootstrap.cc:492] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:04.686807 22455 raft_consensus.cc:359] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.687708 22455 raft_consensus.cc:385] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.688033 22455 raft_consensus.cc:740] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.688809 22455 consensus_queue.cc:260] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.680152 22494 raft_consensus.cc:515] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.696694 22494 leader_election.cc:290] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:04.697528 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:04.698225 22186 raft_consensus.cc:3060] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.708396 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:04.708808 22322 tablet_service.cc:1511] Processing CreateTablet for tablet 808e97b948be44ee895c060c4329964e (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:08:04.715950 22486 leader_election.cc:290] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:04.736835 22491 raft_consensus.cc:493] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.760104 22227 leader_election.cc:304] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607
W20260709 13:08:04.764897 22374 leader_election.cc:343] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.765810 22170 tablet_service.cc:1511] Processing CreateTablet for tablet cd676d7998314b3b9964705206bd4bcb (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:08:04.766659 22491 raft_consensus.cc:515] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:04.767259 22170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd676d7998314b3b9964705206bd4bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.769550 22394 tablet_service.cc:1511] Processing CreateTablet for tablet 808e97b948be44ee895c060c4329964e (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:08:04.770687 22186 raft_consensus.cc:2468] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:04.771045 22394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 808e97b948be44ee895c060c4329964e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.772855 22318 tablet_service.cc:1511] Processing CreateTablet for tablet cd676d7998314b3b9964705206bd4bcb (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:08:04.773500 22458 tablet_bootstrap.cc:492] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:04.774117 22458 ts_tablet_manager.cc:1403] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.127s	user 0.013s	sys 0.002s
I20260709 13:08:04.775736 22375 leader_election.cc:304] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:04.777254 22494 raft_consensus.cc:2804] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:04.777818 22494 raft_consensus.cc:697] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:04.779121 22491 leader_election.cc:290] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:04.783177 22409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814c209a7aff4b11bdc62505cc29c286" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:04.783736 22475 raft_consensus.cc:2804] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:04.784190 22475 raft_consensus.cc:493] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.784024 22248 tablet_service.cc:1511] Processing CreateTablet for tablet cd676d7998314b3b9964705206bd4bcb (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:08:04.784585 22475 raft_consensus.cc:3060] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:04.784799 22151 leader_election.cc:343] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.778739 22494 consensus_queue.cc:237] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:04.785553 22248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd676d7998314b3b9964705206bd4bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.763381 22410 raft_consensus.cc:2468] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:04.786878 22491 consensus_queue.cc:1048] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:04.789307 22457 tablet_bootstrap.cc:492] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:04.777419 22458 raft_consensus.cc:359] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:04.789644 22300 leader_election.cc:304] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:04.790081 22458 raft_consensus.cc:385] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.790504 22458 raft_consensus.cc:740] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.790954 22486 raft_consensus.cc:2804] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:04.791703 22486 raft_consensus.cc:697] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
I20260709 13:08:04.792435 22398 tablet_service.cc:1511] Processing CreateTablet for tablet 59f7f96ef51941e39bc97c0b769c7770 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:08:04.792611 22486 consensus_queue.cc:237] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.789894 22457 ts_tablet_manager.cc:1403] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.317s	user 0.023s	sys 0.004s
I20260709 13:08:04.800838 22464 consensus_queue.cc:1048] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:04.804369 22458 consensus_queue.cc:260] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:04.796273 22322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 808e97b948be44ee895c060c4329964e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:04.814253 22151 consensus_peers.cc:597] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:04.817117 22455 ts_tablet_manager.cc:1434] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.145s	user 0.010s	sys 0.000s
I20260709 13:08:04.824790 22455 tablet_bootstrap.cc:492] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:04.813676 22457 raft_consensus.cc:359] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.825286 22457 raft_consensus.cc:385] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.825637 22457 raft_consensus.cc:740] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.826351 22252 tablet_service.cc:1511] Processing CreateTablet for tablet 59f7f96ef51941e39bc97c0b769c7770 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:08:04.826622 22457 consensus_queue.cc:260] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.827773 22252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f7f96ef51941e39bc97c0b769c7770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.828794 22458 ts_tablet_manager.cc:1434] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.054s	user 0.011s	sys 0.000s
I20260709 13:08:04.830257 22458 tablet_bootstrap.cc:492] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:04.819113 22398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f7f96ef51941e39bc97c0b769c7770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.834463 22177 tablet_service.cc:1511] Processing CreateTablet for tablet 59f7f96ef51941e39bc97c0b769c7770 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:08:04.835999 22177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f7f96ef51941e39bc97c0b769c7770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.851109 22475 raft_consensus.cc:515] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:04.854255 22475 leader_election.cc:290] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:04.869143 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3a61ae21bab486cbfa407942fdeac35" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:04.863816 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd676d7998314b3b9964705206bd4bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.871949 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3a61ae21bab486cbfa407942fdeac35" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:04.872648 22336 raft_consensus.cc:3060] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.864346 22458 tablet_bootstrap.cc:654] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.873220 22324 tablet_service.cc:1511] Processing CreateTablet for tablet 17e65d47217943239d60b88b4b70ba68 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
W20260709 13:08:04.874816 22227 leader_election.cc:343] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.875748 22324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17e65d47217943239d60b88b4b70ba68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.887615 22083 catalog_manager.cc:5697] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:04.894320 22085 catalog_manager.cc:5697] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:04.924768 22394 tablet_service.cc:1511] Processing CreateTablet for tablet 17e65d47217943239d60b88b4b70ba68 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:08:04.925776 22336 raft_consensus.cc:2468] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:04.926555 22394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17e65d47217943239d60b88b4b70ba68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.927737 22227 leader_election.cc:304] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607
I20260709 13:08:04.931236 22510 raft_consensus.cc:2804] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:04.932296 22510 raft_consensus.cc:697] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:04.933553 22510 consensus_queue.cc:237] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:04.903434 22455 tablet_bootstrap.cc:654] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.942170 22409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5106df2a56c74793aad5538c83a79f61" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:04.950539 22409 raft_consensus.cc:2468] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 0.
I20260709 13:08:04.951216 22170 tablet_service.cc:1511] Processing CreateTablet for tablet 17e65d47217943239d60b88b4b70ba68 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:08:04.952243 22226 leader_election.cc:304] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:04.943004 22456 tablet_bootstrap.cc:654] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.945478 22245 tablet_service.cc:1511] Processing CreateTablet for tablet b5a06b42cc294ea4aa15d4ecb7ce088b (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:08:04.954123 22512 raft_consensus.cc:2804] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:04.954550 22475 raft_consensus.cc:493] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.955385 22457 ts_tablet_manager.cc:1434] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.149s	user 0.006s	sys 0.005s
I20260709 13:08:04.956297 22318 tablet_service.cc:1511] Processing CreateTablet for tablet b5a06b42cc294ea4aa15d4ecb7ce088b (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:08:04.957175 22393 tablet_service.cc:1511] Processing CreateTablet for tablet b5a06b42cc294ea4aa15d4ecb7ce088b (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:08:04.952860 22170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17e65d47217943239d60b88b4b70ba68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.958585 22512 raft_consensus.cc:493] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.960095 22475 raft_consensus.cc:515] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:04.961786 22512 raft_consensus.cc:3060] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.966567 22249 tablet_service.cc:1511] Processing CreateTablet for tablet 3808206385a0458abc04050aeab1cea7 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:08:04.967340 22393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5a06b42cc294ea4aa15d4ecb7ce088b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.970818 22245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5a06b42cc294ea4aa15d4ecb7ce088b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.971055 22409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87912ee2e4f46f1b0eb4e574255f31e" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:04.971897 22457 tablet_bootstrap.cc:492] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:04.966635 22475 leader_election.cc:290] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:04.974778 22249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3808206385a0458abc04050aeab1cea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.975073 22458 tablet_bootstrap.cc:492] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:04.975611 22458 ts_tablet_manager.cc:1403] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.146s	user 0.031s	sys 0.016s
I20260709 13:08:04.972023 22409 raft_consensus.cc:2393] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb26db1bd5a64429848b5d63b157c6f0 in current term 1: Already voted for candidate 93815010a884438c86fc6479359b69e8 in this term.
I20260709 13:08:04.978677 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5a06b42cc294ea4aa15d4ecb7ce088b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.983929 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87912ee2e4f46f1b0eb4e574255f31e" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:04.986280 22227 leader_election.cc:304] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:04.989025 22475 raft_consensus.cc:3060] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.995831 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814c209a7aff4b11bdc62505cc29c286" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:04.997917 22152 leader_election.cc:343] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.999677 22152 leader_election.cc:304] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:05.001940 22326 tablet_service.cc:1511] Processing CreateTablet for tablet 3808206385a0458abc04050aeab1cea7 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:08:05.002430 22171 tablet_service.cc:1511] Processing CreateTablet for tablet 3808206385a0458abc04050aeab1cea7 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:08:05.003701 22326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3808206385a0458abc04050aeab1cea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.003818 22171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3808206385a0458abc04050aeab1cea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.984822 22458 raft_consensus.cc:359] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.006956 22458 raft_consensus.cc:385] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.007365 22458 raft_consensus.cc:740] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.012154 22491 raft_consensus.cc:2749] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:05.020569 22400 tablet_service.cc:1511] Processing CreateTablet for tablet f9b737a6d0d24895b749f55ac078ac5f (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:08:05.023217 22457 tablet_bootstrap.cc:654] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.027215 22400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9b737a6d0d24895b749f55ac078ac5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.034521 22512 raft_consensus.cc:515] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.030519 22475 raft_consensus.cc:2749] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:05.017623 22458 consensus_queue.cc:260] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.055292 22455 tablet_bootstrap.cc:492] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:05.063309 22455 ts_tablet_manager.cc:1403] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.239s	user 0.010s	sys 0.010s
I20260709 13:08:05.059828 22456 tablet_bootstrap.cc:492] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:05.060866 22458 ts_tablet_manager.cc:1434] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.085s	user 0.009s	sys 0.002s
I20260709 13:08:05.074797 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5106df2a56c74793aad5538c83a79f61" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:05.076948 22174 tablet_service.cc:1511] Processing CreateTablet for tablet f9b737a6d0d24895b749f55ac078ac5f (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
W20260709 13:08:05.079295 22227 leader_election.cc:343] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.096206 22512 leader_election.cc:290] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:05.100390 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5106df2a56c74793aad5538c83a79f61" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:05.101184 22410 raft_consensus.cc:3060] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.102919 22252 tablet_service.cc:1511] Processing CreateTablet for tablet f9b737a6d0d24895b749f55ac078ac5f (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:08:05.104507 22252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9b737a6d0d24895b749f55ac078ac5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.106433 22491 raft_consensus.cc:493] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.106994 22491 raft_consensus.cc:515] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:05.108497 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9b737a6d0d24895b749f55ac078ac5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.109475 22491 leader_election.cc:290] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:05.110162 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f9732bc4d24ee3a8e261358eba3a21" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
W20260709 13:08:05.111579 22151 leader_election.cc:343] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.114466 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f9732bc4d24ee3a8e261358eba3a21" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:05.116246 22152 leader_election.cc:343] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.116745 22152 leader_election.cc:304] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:05.098939 22456 ts_tablet_manager.cc:1403] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.426s	user 0.013s	sys 0.009s
I20260709 13:08:05.118901 22458 tablet_bootstrap.cc:492] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:05.118511 22491 raft_consensus.cc:2749] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:05.121788 22456 raft_consensus.cc:359] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.122534 22456 raft_consensus.cc:385] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.122836 22456 raft_consensus.cc:740] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.123624 22456 consensus_queue.cc:260] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.125838 22456 ts_tablet_manager.cc:1434] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.003s
I20260709 13:08:05.133673 22458 tablet_bootstrap.cc:654] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.136647 22457 tablet_bootstrap.cc:492] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:05.137264 22457 ts_tablet_manager.cc:1403] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.166s	user 0.017s	sys 0.012s
I20260709 13:08:05.141167 22457 raft_consensus.cc:359] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:05.141961 22457 raft_consensus.cc:385] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.142450 22457 raft_consensus.cc:740] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.142683 22455 raft_consensus.cc:359] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.143558 22455 raft_consensus.cc:385] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.143352 22457 consensus_queue.cc:260] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:05.145895 22457 ts_tablet_manager.cc:1434] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:08:05.147089 22457 tablet_bootstrap.cc:492] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:05.143898 22455 raft_consensus.cc:740] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.153791 22455 consensus_queue.cc:260] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.130671 22410 raft_consensus.cc:2468] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:05.162842 22318 tablet_service.cc:1511] Processing CreateTablet for tablet f0de2f3edc854918bb42aa68cdc2c9fa (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:08:05.164343 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0de2f3edc854918bb42aa68cdc2c9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.164561 22226 leader_election.cc:304] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:05.178754 22455 ts_tablet_manager.cc:1434] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.115s	user 0.009s	sys 0.003s
I20260709 13:08:05.179986 22455 tablet_bootstrap.cc:492] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:05.171931 22512 raft_consensus.cc:2804] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:05.183379 22512 raft_consensus.cc:697] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:05.184373 22512 consensus_queue.cc:237] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.191020 22398 tablet_service.cc:1511] Processing CreateTablet for tablet f0de2f3edc854918bb42aa68cdc2c9fa (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:08:05.192665 22398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0de2f3edc854918bb42aa68cdc2c9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.195851 22171 tablet_service.cc:1511] Processing CreateTablet for tablet f0de2f3edc854918bb42aa68cdc2c9fa (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:08:05.196377 22245 tablet_service.cc:1511] Processing CreateTablet for tablet fdbf4608aa9f48d68cb77a4907dff6a6 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:08:05.197389 22171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0de2f3edc854918bb42aa68cdc2c9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.197834 22245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdbf4608aa9f48d68cb77a4907dff6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.198236 22456 tablet_bootstrap.cc:492] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:05.203482 22494 consensus_queue.cc:1048] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:05.204231 22394 tablet_service.cc:1511] Processing CreateTablet for tablet fdbf4608aa9f48d68cb77a4907dff6a6 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:08:05.205626 22394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdbf4608aa9f48d68cb77a4907dff6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.210806 22457 tablet_bootstrap.cc:654] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.220292 22456 tablet_bootstrap.cc:654] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.222831 22318 tablet_service.cc:1511] Processing CreateTablet for tablet fdbf4608aa9f48d68cb77a4907dff6a6 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:08:05.224373 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdbf4608aa9f48d68cb77a4907dff6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.226998 22486 consensus_queue.cc:1048] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:05.227111 22083 catalog_manager.cc:5697] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:05.239351 22174 tablet_service.cc:1511] Processing CreateTablet for tablet 7d38314054d8474eb00ca91785ea945d (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:08:05.240888 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d38314054d8474eb00ca91785ea945d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.241746 22510 raft_consensus.cc:493] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.242316 22510 raft_consensus.cc:515] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:05.240635 22374 consensus_peers.cc:597] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:05.244621 22510 leader_election.cc:290] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:05.247330 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f9732bc4d24ee3a8e261358eba3a21" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:05.248587 22252 tablet_service.cc:1511] Processing CreateTablet for tablet 7d38314054d8474eb00ca91785ea945d (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
W20260709 13:08:05.249055 22227 leader_election.cc:343] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.247370 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f9732bc4d24ee3a8e261358eba3a21" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:05.250259 22252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d38314054d8474eb00ca91785ea945d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.250401 22186 raft_consensus.cc:2468] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 0.
I20260709 13:08:05.251832 22227 leader_election.cc:304] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:05.253106 22510 raft_consensus.cc:2804] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:05.253533 22510 raft_consensus.cc:493] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.253966 22510 raft_consensus.cc:3060] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.270619 22318 tablet_service.cc:1511] Processing CreateTablet for tablet 7d38314054d8474eb00ca91785ea945d (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:08:05.272557 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d38314054d8474eb00ca91785ea945d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.227226 22455 tablet_bootstrap.cc:654] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.277403 22400 tablet_service.cc:1511] Processing CreateTablet for tablet 16a08df7f7be4e808ba69787809be0d3 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:08:05.280833 22171 tablet_service.cc:1511] Processing CreateTablet for tablet 16a08df7f7be4e808ba69787809be0d3 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:08:05.281246 22486 consensus_queue.cc:1048] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:05.282354 22171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16a08df7f7be4e808ba69787809be0d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.294829 22400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16a08df7f7be4e808ba69787809be0d3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:05.302335 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:05.307986 22457 tablet_bootstrap.cc:492] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:05.308631 22456 tablet_bootstrap.cc:492] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:05.309398 22456 ts_tablet_manager.cc:1403] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.137s	user 0.014s	sys 0.005s
I20260709 13:08:05.311895 22458 tablet_bootstrap.cc:492] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:05.312474 22458 ts_tablet_manager.cc:1403] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.194s	user 0.010s	sys 0.011s
I20260709 13:08:05.285506 22510 raft_consensus.cc:515] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:05.315470 22458 raft_consensus.cc:359] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.316424 22458 raft_consensus.cc:385] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.316746 22458 raft_consensus.cc:740] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.317576 22510 leader_election.cc:290] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:05.308732 22457 ts_tablet_manager.cc:1403] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.162s	user 0.009s	sys 0.009s
I20260709 13:08:05.320888 22249 tablet_service.cc:1511] Processing CreateTablet for tablet 16a08df7f7be4e808ba69787809be0d3 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:08:05.322515 22249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16a08df7f7be4e808ba69787809be0d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.324143 22457 raft_consensus.cc:359] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.325261 22457 raft_consensus.cc:385] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.325664 22457 raft_consensus.cc:740] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.327471 22324 tablet_service.cc:1511] Processing CreateTablet for tablet fb4bb4b406fa4f4f9a55ba3b7c005cf0 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:08:05.327813 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f9732bc4d24ee3a8e261358eba3a21" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:05.328658 22186 raft_consensus.cc:3060] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.335021 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f9732bc4d24ee3a8e261358eba3a21" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:05.337246 22324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb4bb4b406fa4f4f9a55ba3b7c005cf0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:05.340086 22227 leader_election.cc:343] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.341473 22186 raft_consensus.cc:2468] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:05.342375 22457 consensus_queue.cc:260] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.346226 22083 catalog_manager.cc:5697] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:05.313396 22456 raft_consensus.cc:359] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.366963 22456 raft_consensus.cc:385] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.317678 22458 consensus_queue.cc:260] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.367893 22456 raft_consensus.cc:740] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.368791 22456 consensus_queue.cc:260] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.372641 22456 ts_tablet_manager.cc:1434] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.004s
I20260709 13:08:05.372879 22393 tablet_service.cc:1511] Processing CreateTablet for tablet fb4bb4b406fa4f4f9a55ba3b7c005cf0 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:08:05.373857 22456 tablet_bootstrap.cc:492] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:05.375840 22393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb4bb4b406fa4f4f9a55ba3b7c005cf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.380623 22458 ts_tablet_manager.cc:1434] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.068s	user 0.009s	sys 0.000s
I20260709 13:08:05.381873 22458 tablet_bootstrap.cc:492] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:05.382804 22227 leader_election.cc:304] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:05.383992 22465 raft_consensus.cc:493] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.384096 22510 raft_consensus.cc:2804] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:05.374119 22457 ts_tablet_manager.cc:1434] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.000s
I20260709 13:08:05.384063 22174 tablet_service.cc:1511] Processing CreateTablet for tablet fb4bb4b406fa4f4f9a55ba3b7c005cf0 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:08:05.391109 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb4bb4b406fa4f4f9a55ba3b7c005cf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.384656 22465 raft_consensus.cc:515] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.384877 22510 raft_consensus.cc:697] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:05.393391 22491 raft_consensus.cc:493] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.395519 22248 tablet_service.cc:1511] Processing CreateTablet for tablet 96356b5b69fc405cb1678dbfaec2df1f (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:08:05.396261 22491 raft_consensus.cc:515] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.396716 22457 tablet_bootstrap.cc:492] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:05.397058 22248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96356b5b69fc405cb1678dbfaec2df1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.399621 22491 leader_election.cc:290] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:05.400693 22510 consensus_queue.cc:237] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:05.401243 22318 tablet_service.cc:1511] Processing CreateTablet for tablet 96356b5b69fc405cb1678dbfaec2df1f (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:08:05.402603 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96356b5b69fc405cb1678dbfaec2df1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.403828 22465 leader_election.cc:290] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:05.406292 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2076c24c8d1d47e6b3e4ece63fae1597" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:05.407169 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c995e1a8ce9c48aab8235f258f77d080" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:05.407611 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2076c24c8d1d47e6b3e4ece63fae1597" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:05.407874 22336 raft_consensus.cc:2468] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:05.408263 22335 raft_consensus.cc:2468] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
W20260709 13:08:05.409638 22374 leader_election.cc:343] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.408275 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c995e1a8ce9c48aab8235f258f77d080" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:05.411504 22375 leader_election.cc:304] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:05.411702 22152 leader_election.cc:304] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8; no voters: 
W20260709 13:08:05.412578 22151 leader_election.cc:343] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.412787 22494 raft_consensus.cc:2804] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:05.413260 22494 raft_consensus.cc:493] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.413619 22494 raft_consensus.cc:3060] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.418372 22491 raft_consensus.cc:2804] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:05.418828 22491 raft_consensus.cc:493] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.419183 22491 raft_consensus.cc:3060] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.423029 22457 tablet_bootstrap.cc:654] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.426141 22456 tablet_bootstrap.cc:654] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.458469 22458 tablet_bootstrap.cc:654] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.471658 22512 raft_consensus.cc:493] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.443423 22494 raft_consensus.cc:515] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.484404 22174 tablet_service.cc:1511] Processing CreateTablet for tablet dc4bb1a452944c59aa3dc6717c52be6a (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:08:05.485800 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc4bb1a452944c59aa3dc6717c52be6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.475095 22398 tablet_service.cc:1511] Processing CreateTablet for tablet 96356b5b69fc405cb1678dbfaec2df1f (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:08:05.487125 22398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96356b5b69fc405cb1678dbfaec2df1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.476539 22512 raft_consensus.cc:515] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.491060 22512 leader_election.cc:290] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:05.491379 22400 tablet_service.cc:1511] Processing CreateTablet for tablet a51f4b4253994202856577a87dd8d61c (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:08:05.492744 22400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51f4b4253994202856577a87dd8d61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.494463 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:05.498502 22249 tablet_service.cc:1511] Processing CreateTablet for tablet dc4bb1a452944c59aa3dc6717c52be6a (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:08:05.499940 22249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc4bb1a452944c59aa3dc6717c52be6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.501704 22455 tablet_bootstrap.cc:492] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:05.502287 22455 ts_tablet_manager.cc:1403] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.323s	user 0.017s	sys 0.006s
I20260709 13:08:05.502883 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:05.504124 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2076c24c8d1d47e6b3e4ece63fae1597" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
W20260709 13:08:05.505667 22374 leader_election.cc:343] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.505856 22226 leader_election.cc:304] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:05.507251 22512 raft_consensus.cc:2749] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:05.510257 22494 leader_election.cc:290] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:05.510926 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2076c24c8d1d47e6b3e4ece63fae1597" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:05.511649 22335 raft_consensus.cc:3060] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.514389 22456 tablet_bootstrap.cc:492] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:05.514921 22456 ts_tablet_manager.cc:1403] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.141s	user 0.017s	sys 0.001s
I20260709 13:08:05.513744 22083 catalog_manager.cc:5697] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:05.505863 22455 raft_consensus.cc:359] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.516678 22455 raft_consensus.cc:385] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.516757 22458 tablet_bootstrap.cc:492] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:05.517372 22458 ts_tablet_manager.cc:1403] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.136s	user 0.015s	sys 0.016s
I20260709 13:08:05.518543 22324 tablet_service.cc:1511] Processing CreateTablet for tablet dc4bb1a452944c59aa3dc6717c52be6a (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:08:05.518590 22456 raft_consensus.cc:359] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:05.536937 22456 raft_consensus.cc:385] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.537359 22456 raft_consensus.cc:740] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.537545 22324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc4bb1a452944c59aa3dc6717c52be6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.538259 22456 consensus_queue.cc:260] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:05.521179 22227 consensus_peers.cc:597] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:05.520865 22458 raft_consensus.cc:359] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:05.540304 22318 tablet_service.cc:1511] Processing CreateTablet for tablet 42127cae8f494cb081ec04febe577130 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:08:05.541011 22458 raft_consensus.cc:385] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.542032 22458 raft_consensus.cc:740] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.542764 22456 ts_tablet_manager.cc:1434] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:08:05.527122 22491 raft_consensus.cc:515] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.528716 22174 tablet_service.cc:1511] Processing CreateTablet for tablet a51f4b4253994202856577a87dd8d61c (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:08:05.529439 22245 tablet_service.cc:1511] Processing CreateTablet for tablet a51f4b4253994202856577a87dd8d61c (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:08:05.543943 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51f4b4253994202856577a87dd8d61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.543947 22245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51f4b4253994202856577a87dd8d61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.544373 22398 tablet_service.cc:1511] Processing CreateTablet for tablet 42127cae8f494cb081ec04febe577130 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:08:05.544920 22491 leader_election.cc:290] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:05.545693 22398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42127cae8f494cb081ec04febe577130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.547922 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c995e1a8ce9c48aab8235f258f77d080" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
W20260709 13:08:05.549528 22151 leader_election.cc:343] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.550109 22171 tablet_service.cc:1511] Processing CreateTablet for tablet 42127cae8f494cb081ec04febe577130 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:08:05.523394 22455 raft_consensus.cc:740] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.551337 22171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42127cae8f494cb081ec04febe577130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.551431 22456 tablet_bootstrap.cc:492] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:05.525301 22457 tablet_bootstrap.cc:492] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:05.552204 22457 ts_tablet_manager.cc:1403] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.156s	user 0.015s	sys 0.001s
I20260709 13:08:05.527199 22335 raft_consensus.cc:2468] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:05.550297 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c995e1a8ce9c48aab8235f258f77d080" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:05.552860 22336 raft_consensus.cc:3060] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.555365 22457 raft_consensus.cc:359] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.556164 22457 raft_consensus.cc:385] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.556427 22457 raft_consensus.cc:740] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.558220 22375 leader_election.cc:304] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:05.541611 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42127cae8f494cb081ec04febe577130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.559085 22494 raft_consensus.cc:2804] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:05.559602 22494 raft_consensus.cc:697] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:05.560454 22494 consensus_queue.cc:237] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.565577 22457 consensus_queue.cc:260] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.551518 22455 consensus_queue.cc:260] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.567008 22491 raft_consensus.cc:493] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.570106 22491 raft_consensus.cc:515] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.570423 22458 consensus_queue.cc:260] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:05.573148 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "878ae1fea9154e7088fdba8930c5eb23" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:05.573820 22410 raft_consensus.cc:2468] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:05.575450 22151 leader_election.cc:304] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:05.577271 22491 leader_election.cc:290] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:05.578130 22491 raft_consensus.cc:2804] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:05.579250 22491 raft_consensus.cc:493] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.580036 22491 raft_consensus.cc:3060] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.585582 22512 consensus_queue.cc:1048] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:05.589004 22457 ts_tablet_manager.cc:1434] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.036s	user 0.003s	sys 0.004s
I20260709 13:08:05.590353 22457 tablet_bootstrap.cc:492] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:05.604894 22336 raft_consensus.cc:2468] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:05.617509 22152 leader_election.cc:304] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:05.606940 22455 ts_tablet_manager.cc:1434] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.104s	user 0.007s	sys 0.002s
I20260709 13:08:05.618322 22083 catalog_manager.cc:5697] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:05.620505 22455 tablet_bootstrap.cc:492] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:05.613960 22324 tablet_service.cc:1511] Processing CreateTablet for tablet 2d9ed71058164d51a8aa5b9980536bd8 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:08:05.636814 22537 raft_consensus.cc:2804] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:05.637377 22537 raft_consensus.cc:697] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:05.637759 22324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d9ed71058164d51a8aa5b9980536bd8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:05.638238 22151 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:41261, user_credentials={real_user=slave}} blocked reactor thread for 62887.3us
I20260709 13:08:05.639401 22245 tablet_service.cc:1511] Processing CreateTablet for tablet 57dfb3dc107a469c880393583ee0c218 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:08:05.640580 22245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57dfb3dc107a469c880393583ee0c218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.638324 22537 consensus_queue.cc:237] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.646831 22458 ts_tablet_manager.cc:1434] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.129s	user 0.012s	sys 0.017s
I20260709 13:08:05.652778 22464 raft_consensus.cc:493] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.653359 22464 raft_consensus.cc:515] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:05.655301 22491 raft_consensus.cc:515] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
W20260709 13:08:05.627594 22227 consensus_peers.cc:597] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:05.667548 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "878ae1fea9154e7088fdba8930c5eb23" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:05.631290 22249 tablet_service.cc:1511] Processing CreateTablet for tablet 2d9ed71058164d51a8aa5b9980536bd8 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:08:05.632110 22457 tablet_bootstrap.cc:654] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.668602 22318 tablet_service.cc:1511] Processing CreateTablet for tablet 57dfb3dc107a469c880393583ee0c218 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:08:05.669654 22249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d9ed71058164d51a8aa5b9980536bd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.670574 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57dfb3dc107a469c880393583ee0c218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.673604 22510 raft_consensus.cc:493] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.674193 22510 raft_consensus.cc:515] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.674712 22455 tablet_bootstrap.cc:654] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.676538 22510 leader_election.cc:290] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:05.635807 22171 tablet_service.cc:1511] Processing CreateTablet for tablet 57dfb3dc107a469c880393583ee0c218 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:08:05.635869 22393 tablet_service.cc:1511] Processing CreateTablet for tablet 2d9ed71058164d51a8aa5b9980536bd8 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:08:05.677879 22491 leader_election.cc:290] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:05.678373 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "878ae1fea9154e7088fdba8930c5eb23" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:05.679044 22410 raft_consensus.cc:3060] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.679282 22393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d9ed71058164d51a8aa5b9980536bd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.680968 22409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:05.657295 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f9732bc4d24ee3a8e261358eba3a21" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
W20260709 13:08:05.685449 22151 leader_election.cc:343] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.687247 22187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:05.622186 22456 tablet_bootstrap.cc:654] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.658871 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f9732bc4d24ee3a8e261358eba3a21" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:05.690505 22186 raft_consensus.cc:2393] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93815010a884438c86fc6479359b69e8 in current term 1: Already voted for candidate bb26db1bd5a64429848b5d63b157c6f0 in this term.
I20260709 13:08:05.658915 22464 leader_election.cc:290] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:05.648144 22458 tablet_bootstrap.cc:492] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:05.692451 22227 leader_election.cc:304] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:05.694537 22512 raft_consensus.cc:2749] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:05.695214 22301 leader_election.cc:304] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:05.696473 22464 raft_consensus.cc:3060] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.697036 22262 raft_consensus.cc:3060] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.699101 22510 consensus_queue.cc:1048] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:05.690951 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "878ae1fea9154e7088fdba8930c5eb23" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
W20260709 13:08:05.705082 22151 leader_election.cc:343] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.708086 22464 raft_consensus.cc:2749] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:05.689767 22457 tablet_bootstrap.cc:492] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:05.709183 22457 ts_tablet_manager.cc:1403] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.119s	user 0.011s	sys 0.011s
I20260709 13:08:05.710042 22458 tablet_bootstrap.cc:654] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.712354 22457 raft_consensus.cc:359] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.713860 22457 raft_consensus.cc:385] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.714176 22171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57dfb3dc107a469c880393583ee0c218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.714314 22457 raft_consensus.cc:740] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.718430 22457 consensus_queue.cc:260] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.732648 22457 ts_tablet_manager.cc:1434] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:08:05.734424 22457 tablet_bootstrap.cc:492] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:05.736577 22398 tablet_service.cc:1511] Processing CreateTablet for tablet 62cc2afc26f14239af465ce79a044163 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:08:05.710177 22410 raft_consensus.cc:2468] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:05.738032 22398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62cc2afc26f14239af465ce79a044163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.763729 22151 leader_election.cc:304] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:05.764058 22174 tablet_service.cc:1511] Processing CreateTablet for tablet 62cc2afc26f14239af465ce79a044163 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:08:05.765461 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62cc2afc26f14239af465ce79a044163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.766010 22326 tablet_service.cc:1511] Processing CreateTablet for tablet a91b5edf8c9e4a71a195e0f796d439f3 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:08:05.767383 22326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a91b5edf8c9e4a71a195e0f796d439f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.771229 22249 tablet_service.cc:1511] Processing CreateTablet for tablet 62cc2afc26f14239af465ce79a044163 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:08:05.772557 22249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62cc2afc26f14239af465ce79a044163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.771944 22082 catalog_manager.cc:5697] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:05.774477 22494 raft_consensus.cc:493] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.774991 22494 raft_consensus.cc:515] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.776321 22457 tablet_bootstrap.cc:654] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.781219 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e8f1fb740446b8b42c60afd2e7c3eb" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:05.782029 22334 raft_consensus.cc:2468] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:05.783417 22375 leader_election.cc:304] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:05.783769 22537 raft_consensus.cc:2804] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:05.784307 22465 raft_consensus.cc:2804] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:05.784614 22494 leader_election.cc:290] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:05.785969 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e8f1fb740446b8b42c60afd2e7c3eb" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:05.792474 22465 raft_consensus.cc:493] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.792943 22465 raft_consensus.cc:3060] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:05.794657 22375 leader_election.cc:343] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.795329 22458 tablet_bootstrap.cc:492] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:05.795867 22458 ts_tablet_manager.cc:1403] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.148s	user 0.014s	sys 0.010s
I20260709 13:08:05.799346 22458 raft_consensus.cc:359] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.800185 22458 raft_consensus.cc:385] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.800520 22458 raft_consensus.cc:740] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.801419 22458 consensus_queue.cc:260] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.802389 22537 raft_consensus.cc:697] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:05.808111 22455 tablet_bootstrap.cc:492] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:05.809168 22455 ts_tablet_manager.cc:1403] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.189s	user 0.013s	sys 0.005s
I20260709 13:08:05.808147 22465 raft_consensus.cc:515] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.810909 22400 tablet_service.cc:1511] Processing CreateTablet for tablet a91b5edf8c9e4a71a195e0f796d439f3 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:08:05.812290 22400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a91b5edf8c9e4a71a195e0f796d439f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.812085 22174 tablet_service.cc:1511] Processing CreateTablet for tablet a91b5edf8c9e4a71a195e0f796d439f3 (DEFAULT_TABLE table=test-workload [id=107e0e8d8e164514a8da26569371e71e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:08:05.812601 22465 leader_election.cc:290] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:05.815516 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a91b5edf8c9e4a71a195e0f796d439f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:05.805899 22458 ts_tablet_manager.cc:1434] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 13:08:05.818010 22458 tablet_bootstrap.cc:492] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:05.830037 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e8f1fb740446b8b42c60afd2e7c3eb" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:05.830806 22186 raft_consensus.cc:3060] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.836143 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e8f1fb740446b8b42c60afd2e7c3eb" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:05.836764 22334 raft_consensus.cc:3060] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.832962 22537 consensus_queue.cc:237] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.842085 22186 raft_consensus.cc:2468] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:05.845945 22261 raft_consensus.cc:3060] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.848100 22375 leader_election.cc:304] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:05.849596 22494 raft_consensus.cc:2804] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:05.860926 22535 raft_consensus.cc:493] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.861475 22457 tablet_bootstrap.cc:492] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:05.862223 22457 ts_tablet_manager.cc:1403] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.128s	user 0.012s	sys 0.010s
I20260709 13:08:05.861688 22535 raft_consensus.cc:515] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:05.867990 22541 consensus_queue.cc:1048] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:05.868934 22456 tablet_bootstrap.cc:492] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:05.878068 22456 ts_tablet_manager.cc:1403] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.327s	user 0.011s	sys 0.008s
I20260709 13:08:05.881470 22334 raft_consensus.cc:2468] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:05.891206 22456 raft_consensus.cc:359] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.892288 22456 raft_consensus.cc:385] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.894728 22456 raft_consensus.cc:740] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.872233 22457 raft_consensus.cc:359] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.873274 22535 leader_election.cc:290] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:05.875586 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3a61ae21bab486cbfa407942fdeac35" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:05.897562 22457 raft_consensus.cc:385] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.896739 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3a61ae21bab486cbfa407942fdeac35" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:05.898433 22457 raft_consensus.cc:740] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.899312 22457 consensus_queue.cc:260] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.900602 22456 consensus_queue.cc:260] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:05.904079 22457 ts_tablet_manager.cc:1434] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.002s
I20260709 13:08:05.905166 22457 tablet_bootstrap.cc:492] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:05.906458 22510 raft_consensus.cc:493] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.906988 22510 raft_consensus.cc:515] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:05.909243 22510 leader_election.cc:290] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:05.910125 22455 raft_consensus.cc:359] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.910844 22455 raft_consensus.cc:385] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.911120 22455 raft_consensus.cc:740] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.913048 22457 tablet_bootstrap.cc:654] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.912945 22455 consensus_queue.cc:260] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.910899 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "878ae1fea9154e7088fdba8930c5eb23" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:05.914584 22151 leader_election.cc:304] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:05.914876 22410 raft_consensus.cc:2393] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb26db1bd5a64429848b5d63b157c6f0 in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:05.916522 22537 raft_consensus.cc:2749] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:05.917558 22187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "878ae1fea9154e7088fdba8930c5eb23" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:05.901791 22494 raft_consensus.cc:697] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:05.918021 22455 ts_tablet_manager.cc:1434] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.109s	user 0.010s	sys 0.001s
I20260709 13:08:05.919497 22455 tablet_bootstrap.cc:492] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:05.920251 22227 leader_election.cc:304] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:05.921258 22510 raft_consensus.cc:3060] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.921470 22458 tablet_bootstrap.cc:654] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.925009 22456 ts_tablet_manager.cc:1434] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.047s	user 0.008s	sys 0.001s
I20260709 13:08:05.923430 22082 catalog_manager.cc:5697] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:05.930392 22494 consensus_queue.cc:237] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:05.935334 22510 raft_consensus.cc:2749] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:05.936591 22456 tablet_bootstrap.cc:492] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:05.946645 22455 tablet_bootstrap.cc:654] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.970458 22458 tablet_bootstrap.cc:492] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:05.971422 22458 ts_tablet_manager.cc:1403] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.154s	user 0.015s	sys 0.015s
I20260709 13:08:05.976227 22541 consensus_queue.cc:1048] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:05.976464 22458 raft_consensus.cc:359] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:05.977512 22458 raft_consensus.cc:385] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.977937 22458 raft_consensus.cc:740] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.979000 22458 consensus_queue.cc:260] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:05.981992 22458 ts_tablet_manager.cc:1434] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:08:05.984164 22458 tablet_bootstrap.cc:492] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:05.997611 22260 raft_consensus.cc:3060] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.999825 22457 tablet_bootstrap.cc:492] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:06.000404 22457 ts_tablet_manager.cc:1403] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.095s	user 0.031s	sys 0.026s
I20260709 13:08:06.001926 22458 tablet_bootstrap.cc:654] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.995519 22082 catalog_manager.cc:5697] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:06.007164 22187 raft_consensus.cc:3060] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.008993 22456 tablet_bootstrap.cc:654] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.023772 22458 tablet_bootstrap.cc:492] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:06.024417 22458 ts_tablet_manager.cc:1403] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.005s
I20260709 13:08:06.006903 22457 raft_consensus.cc:359] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.027264 22457 raft_consensus.cc:385] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.027592 22457 raft_consensus.cc:740] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.027633 22458 raft_consensus.cc:359] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.028355 22457 consensus_queue.cc:260] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.030833 22457 ts_tablet_manager.cc:1434] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.001s
I20260709 13:08:06.031829 22457 tablet_bootstrap.cc:492] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:06.028465 22458 raft_consensus.cc:385] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.034754 22458 raft_consensus.cc:740] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.035529 22458 consensus_queue.cc:260] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.037842 22458 ts_tablet_manager.cc:1434] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:08:06.038877 22458 tablet_bootstrap.cc:492] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:06.039714 22457 tablet_bootstrap.cc:654] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.051997 22457 tablet_bootstrap.cc:492] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:06.052577 22457 ts_tablet_manager.cc:1403] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 13:08:06.053078 22458 tablet_bootstrap.cc:654] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.055665 22457 raft_consensus.cc:359] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.056478 22457 raft_consensus.cc:385] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.056778 22457 raft_consensus.cc:740] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.058497 22457 consensus_queue.cc:260] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.060768 22455 tablet_bootstrap.cc:492] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:06.061096 22457 ts_tablet_manager.cc:1434] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:08:06.061257 22455 ts_tablet_manager.cc:1403] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.142s	user 0.018s	sys 0.028s
I20260709 13:08:06.062191 22457 tablet_bootstrap.cc:492] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:06.065128 22455 raft_consensus.cc:359] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.065902 22455 raft_consensus.cc:385] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.066206 22455 raft_consensus.cc:740] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.067411 22456 tablet_bootstrap.cc:492] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:06.067049 22455 consensus_queue.cc:260] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.068004 22456 ts_tablet_manager.cc:1403] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.132s	user 0.009s	sys 0.003s
I20260709 13:08:06.069979 22455 ts_tablet_manager.cc:1434] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:08:06.075433 22455 tablet_bootstrap.cc:492] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:06.075920 22457 tablet_bootstrap.cc:654] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.080611 22494 consensus_queue.cc:1048] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:06.086494 22549 consensus_queue.cc:1048] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:06.088263 22457 tablet_bootstrap.cc:492] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:06.090090 22458 tablet_bootstrap.cc:492] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:06.090713 22458 ts_tablet_manager.cc:1403] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.009s
I20260709 13:08:06.091213 22187 raft_consensus.cc:3060] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.094753 22458 raft_consensus.cc:359] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.095896 22458 raft_consensus.cc:385] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.096257 22458 raft_consensus.cc:740] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.097126 22458 consensus_queue.cc:260] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.099490 22457 ts_tablet_manager.cc:1403] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.010s
I20260709 13:08:06.103057 22457 raft_consensus.cc:359] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:06.104039 22457 raft_consensus.cc:385] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.104388 22457 raft_consensus.cc:740] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.071038 22456 raft_consensus.cc:359] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.107465 22456 raft_consensus.cc:385] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.110723 22458 ts_tablet_manager.cc:1434] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:08:06.112632 22458 tablet_bootstrap.cc:492] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:06.107852 22456 raft_consensus.cc:740] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.116921 22456 consensus_queue.cc:260] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.105314 22457 consensus_queue.cc:260] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:06.120036 22555 raft_consensus.cc:493] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.123955 22458 tablet_bootstrap.cc:654] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.123044 22457 ts_tablet_manager.cc:1434] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:08:06.120369 22456 ts_tablet_manager.cc:1434] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.052s	user 0.003s	sys 0.007s
I20260709 13:08:06.125712 22457 tablet_bootstrap.cc:492] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:06.125730 22456 tablet_bootstrap.cc:492] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:06.123811 22555 raft_consensus.cc:515] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.132498 22555 leader_election.cc:290] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:06.123984 22455 tablet_bootstrap.cc:654] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.133786 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4c4a9cd1164b7997f7188f9da9f70b" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:06.133981 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4c4a9cd1164b7997f7188f9da9f70b" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:06.134613 22186 raft_consensus.cc:2468] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 0.
I20260709 13:08:06.135571 22458 tablet_bootstrap.cc:492] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:06.136101 22458 ts_tablet_manager.cc:1403] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.003s
I20260709 13:08:06.136313 22227 leader_election.cc:304] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
W20260709 13:08:06.137403 22227 leader_election.cc:343] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.138859 22555 raft_consensus.cc:2804] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:06.139413 22555 raft_consensus.cc:493] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.139887 22555 raft_consensus.cc:3060] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.140329 22458 raft_consensus.cc:359] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.141100 22458 raft_consensus.cc:385] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.141363 22458 raft_consensus.cc:740] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.141723 22541 raft_consensus.cc:493] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.142143 22458 consensus_queue.cc:260] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.142280 22541 raft_consensus.cc:515] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:06.146121 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cadc140203246b088e6e2883bbe1977" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:06.147179 22541 leader_election.cc:290] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
W20260709 13:08:06.148442 22226 leader_election.cc:343] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.149196 22458 ts_tablet_manager.cc:1434] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:08:06.150403 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cadc140203246b088e6e2883bbe1977" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:06.150703 22458 tablet_bootstrap.cc:492] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
W20260709 13:08:06.152158 22227 leader_election.cc:343] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.152884 22227 leader_election.cc:304] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:06.154179 22457 tablet_bootstrap.cc:654] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.154289 22541 raft_consensus.cc:2749] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:06.150363 22555 raft_consensus.cc:515] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.159762 22458 tablet_bootstrap.cc:654] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.163743 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4c4a9cd1164b7997f7188f9da9f70b" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:06.164435 22186 raft_consensus.cc:3060] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.174649 22535 consensus_queue.cc:1048] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:06.177014 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4c4a9cd1164b7997f7188f9da9f70b" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
W20260709 13:08:06.178789 22227 leader_election.cc:343] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.180490 22186 raft_consensus.cc:2468] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:06.182895 22227 leader_election.cc:304] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:06.183871 22541 raft_consensus.cc:2804] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:06.184478 22541 raft_consensus.cc:697] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:06.185444 22541 consensus_queue.cc:237] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.188025 22458 tablet_bootstrap.cc:492] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:06.188740 22458 ts_tablet_manager.cc:1403] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.011s
I20260709 13:08:06.190410 22456 tablet_bootstrap.cc:654] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.181344 22455 tablet_bootstrap.cc:492] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:06.191382 22455 ts_tablet_manager.cc:1403] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.116s	user 0.012s	sys 0.008s
I20260709 13:08:06.192581 22458 raft_consensus.cc:359] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.181648 22555 leader_election.cc:290] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:06.193526 22458 raft_consensus.cc:385] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.193919 22458 raft_consensus.cc:740] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.196692 22458 consensus_queue.cc:260] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.198673 22455 raft_consensus.cc:359] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.201293 22455 raft_consensus.cc:385] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.201640 22455 raft_consensus.cc:740] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.202594 22455 consensus_queue.cc:260] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.204784 22455 ts_tablet_manager.cc:1434] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:08:06.205914 22455 tablet_bootstrap.cc:492] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:06.199374 22458 ts_tablet_manager.cc:1434] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:08:06.207125 22458 tablet_bootstrap.cc:492] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:06.218772 22334 raft_consensus.cc:3060] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.221362 22455 tablet_bootstrap.cc:654] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.221362 22458 tablet_bootstrap.cc:654] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.227326 22464 raft_consensus.cc:493] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.227984 22464 raft_consensus.cc:515] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.230577 22464 leader_election.cc:290] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:06.231410 22455 tablet_bootstrap.cc:492] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:06.231992 22455 ts_tablet_manager.cc:1403] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.000s
I20260709 13:08:06.233381 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814c209a7aff4b11bdc62505cc29c286" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:06.234167 22410 raft_consensus.cc:2468] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 0.
I20260709 13:08:06.235270 22455 raft_consensus.cc:359] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:06.236003 22455 raft_consensus.cc:385] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.236289 22455 raft_consensus.cc:740] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.236901 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814c209a7aff4b11bdc62505cc29c286" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:06.237298 22300 leader_election.cc:304] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:06.237732 22186 raft_consensus.cc:2468] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 0.
I20260709 13:08:06.239148 22486 raft_consensus.cc:2804] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:06.239687 22486 raft_consensus.cc:493] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.240099 22486 raft_consensus.cc:3060] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.237020 22455 consensus_queue.cc:260] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:06.242363 22455 ts_tablet_manager.cc:1434] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:08:06.243310 22458 tablet_bootstrap.cc:492] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:06.243955 22458 ts_tablet_manager.cc:1403] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.037s	user 0.003s	sys 0.017s
I20260709 13:08:06.245656 22457 tablet_bootstrap.cc:492] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:06.246202 22457 ts_tablet_manager.cc:1403] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.121s	user 0.037s	sys 0.025s
I20260709 13:08:06.247944 22410 raft_consensus.cc:3060] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.248100 22458 raft_consensus.cc:359] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.249001 22458 raft_consensus.cc:385] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.249362 22458 raft_consensus.cc:740] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.250291 22458 consensus_queue.cc:260] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.252933 22458 ts_tablet_manager.cc:1434] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:08:06.254014 22458 tablet_bootstrap.cc:492] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:06.257990 22491 consensus_queue.cc:1048] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:06.263972 22457 raft_consensus.cc:359] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.264846 22457 raft_consensus.cc:385] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.265122 22457 raft_consensus.cc:740] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.265879 22457 consensus_queue.cc:260] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.269356 22457 ts_tablet_manager.cc:1434] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:08:06.270496 22457 tablet_bootstrap.cc:492] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:06.270612 22486 raft_consensus.cc:515] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.273275 22458 tablet_bootstrap.cc:654] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.274715 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814c209a7aff4b11bdc62505cc29c286" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:06.275261 22455 tablet_bootstrap.cc:492] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:06.275499 22408 raft_consensus.cc:3060] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.277829 22457 tablet_bootstrap.cc:654] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.278486 22537 raft_consensus.cc:493] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 93815010a884438c86fc6479359b69e8)
I20260709 13:08:06.279095 22537 raft_consensus.cc:515] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.278855 22082 catalog_manager.cc:5697] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:06.289253 22537 leader_election.cc:290] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:06.290150 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcf4b5402e45470c81a3389d815a9286" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:06.291096 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcf4b5402e45470c81a3389d815a9286" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:06.296036 22456 tablet_bootstrap.cc:492] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:06.296824 22456 ts_tablet_manager.cc:1403] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.171s	user 0.015s	sys 0.000s
I20260709 13:08:06.298332 22152 leader_election.cc:304] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:06.302448 22491 raft_consensus.cc:493] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.303001 22491 raft_consensus.cc:515] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.305403 22491 leader_election.cc:290] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:06.305673 22408 raft_consensus.cc:2468] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:06.300135 22537 raft_consensus.cc:2749] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:06.310412 22486 leader_election.cc:290] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:06.315168 22456 raft_consensus.cc:359] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.316116 22456 raft_consensus.cc:385] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.316519 22456 raft_consensus.cc:740] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.317514 22456 consensus_queue.cc:260] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.318203 22458 tablet_bootstrap.cc:492] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:06.319002 22458 ts_tablet_manager.cc:1403] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.065s	user 0.008s	sys 0.009s
I20260709 13:08:06.321005 22300 leader_election.cc:304] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:06.322772 22457 tablet_bootstrap.cc:492] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:06.323378 22457 ts_tablet_manager.cc:1403] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.053s	user 0.018s	sys 0.029s
I20260709 13:08:06.324136 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b51a3b1dc444319751dd92054d5dc3" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:06.324582 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b51a3b1dc444319751dd92054d5dc3" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:06.326428 22151 leader_election.cc:343] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.326783 22457 raft_consensus.cc:359] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:06.328510 22457 raft_consensus.cc:385] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.327453 22486 raft_consensus.cc:2804] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:06.326004 22187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814c209a7aff4b11bdc62505cc29c286" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:06.328842 22457 raft_consensus.cc:740] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.330119 22187 raft_consensus.cc:3060] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:06.326390 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:06.329366 22486 raft_consensus.cc:697] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
I20260709 13:08:06.330780 22457 consensus_queue.cc:260] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
W20260709 13:08:06.331509 22151 leader_election.cc:343] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.331436 22486 consensus_queue.cc:237] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.332129 22151 leader_election.cc:304] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:06.333156 22457 ts_tablet_manager.cc:1434] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:08:06.334268 22457 tablet_bootstrap.cc:492] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:06.334868 22455 tablet_bootstrap.cc:654] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.335103 22465 consensus_queue.cc:1048] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:06.338112 22491 raft_consensus.cc:2749] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:06.340250 22456 ts_tablet_manager.cc:1434] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:08:06.341223 22457 tablet_bootstrap.cc:654] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.342373 22456 tablet_bootstrap.cc:492] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:06.352139 22187 raft_consensus.cc:2468] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:06.356931 22456 tablet_bootstrap.cc:654] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.356950 22455 tablet_bootstrap.cc:492] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:06.357766 22455 ts_tablet_manager.cc:1403] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.083s	user 0.010s	sys 0.005s
I20260709 13:08:06.365150 22457 tablet_bootstrap.cc:492] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:06.365805 22457 ts_tablet_manager.cc:1403] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.001s
I20260709 13:08:06.368104 22458 raft_consensus.cc:359] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.369172 22458 raft_consensus.cc:385] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.374625 22457 raft_consensus.cc:359] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.379276 22458 raft_consensus.cc:740] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.379671 22457 raft_consensus.cc:385] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.380000 22457 raft_consensus.cc:740] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.380304 22458 consensus_queue.cc:260] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.380836 22457 consensus_queue.cc:260] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.382763 22458 ts_tablet_manager.cc:1434] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.063s	user 0.009s	sys 0.000s
I20260709 13:08:06.383899 22458 tablet_bootstrap.cc:492] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:06.382862 22457 ts_tablet_manager.cc:1434] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.002s
I20260709 13:08:06.385236 22457 tablet_bootstrap.cc:492] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:06.386006 22549 raft_consensus.cc:493] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.377969 22465 consensus_queue.cc:1048] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:08:06.386673 22549 raft_consensus.cc:515] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:06.374619 22537 consensus_queue.cc:1048] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:08:06.391523 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cadc140203246b088e6e2883bbe1977" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:06.386673 22464 raft_consensus.cc:493] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bb26db1bd5a64429848b5d63b157c6f0)
I20260709 13:08:06.392313 22261 raft_consensus.cc:2468] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:06.392565 22464 raft_consensus.cc:515] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:06.389757 22549 leader_election.cc:290] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:06.396984 22456 tablet_bootstrap.cc:492] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:06.397703 22456 ts_tablet_manager.cc:1403] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.000s
I20260709 13:08:06.400758 22457 tablet_bootstrap.cc:654] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.400465 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cadc140203246b088e6e2883bbe1977" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:06.400990 22456 raft_consensus.cc:359] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:06.376812 22455 raft_consensus.cc:359] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.401739 22456 raft_consensus.cc:385] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.401933 22455 raft_consensus.cc:385] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.402038 22456 raft_consensus.cc:740] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.402349 22455 raft_consensus.cc:740] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.393935 22374 leader_election.cc:304] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:06.402930 22456 consensus_queue.cc:260] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:06.403616 22549 raft_consensus.cc:2804] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:06.403872 22336 raft_consensus.cc:2468] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:06.404240 22549 raft_consensus.cc:493] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.405126 22456 ts_tablet_manager.cc:1434] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:08:06.408679 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5106df2a56c74793aad5538c83a79f61" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:06.403142 22455 consensus_queue.cc:260] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.413110 22455 ts_tablet_manager.cc:1434] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.000s
I20260709 13:08:06.414310 22455 tablet_bootstrap.cc:492] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:06.416698 22409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5106df2a56c74793aad5538c83a79f61" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:06.418699 22456 tablet_bootstrap.cc:492] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:06.419776 22486 raft_consensus.cc:493] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.420387 22486 raft_consensus.cc:515] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.409852 22458 tablet_bootstrap.cc:654] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.425992 22456 tablet_bootstrap.cc:654] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.428550 22300 leader_election.cc:304] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:06.430881 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4c4a9cd1164b7997f7188f9da9f70b" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:06.434976 22186 raft_consensus.cc:2393] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93815010a884438c86fc6479359b69e8 in current term 1: Already voted for candidate bb26db1bd5a64429848b5d63b157c6f0 in this term.
I20260709 13:08:06.433568 22581 raft_consensus.cc:2749] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:06.437713 22537 consensus_queue.cc:1048] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:06.438179 22486 leader_election.cc:290] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:06.439189 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4c4a9cd1164b7997f7188f9da9f70b" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:06.441334 22512 raft_consensus.cc:493] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.441922 22512 raft_consensus.cc:515] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.405632 22549 raft_consensus.cc:3060] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.444478 22512 leader_election.cc:290] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:06.414695 22464 leader_election.cc:290] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:06.447273 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2144e27a177e41b0b22e474d9f209102" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:06.450510 22455 tablet_bootstrap.cc:654] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:06.451445 22227 leader_election.cc:343] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.451965 22300 leader_election.cc:304] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:06.453225 22555 raft_consensus.cc:493] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.453670 22486 raft_consensus.cc:3060] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.453840 22555 raft_consensus.cc:515] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.457209 22555 leader_election.cc:290] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:06.458467 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade9dda3178e42c783bc93e5deec21b7" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:06.459260 22408 raft_consensus.cc:2468] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 0.
I20260709 13:08:06.460695 22226 leader_election.cc:304] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:06.461675 22555 raft_consensus.cc:2804] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:06.462109 22555 raft_consensus.cc:493] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.462512 22555 raft_consensus.cc:3060] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.463150 22537 raft_consensus.cc:493] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.463728 22537 raft_consensus.cc:515] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.464491 22083 catalog_manager.cc:5697] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:06.465413 22455 tablet_bootstrap.cc:492] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:06.466135 22455 ts_tablet_manager.cc:1403] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.007s
I20260709 13:08:06.466552 22537 leader_election.cc:290] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:06.447688 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2144e27a177e41b0b22e474d9f209102" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:06.471693 22186 raft_consensus.cc:2468] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 0.
I20260709 13:08:06.470721 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:06.473425 22408 raft_consensus.cc:2468] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:06.475733 22151 leader_election.cc:304] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:06.476994 22537 raft_consensus.cc:2804] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:06.477399 22537 raft_consensus.cc:493] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.477764 22537 raft_consensus.cc:3060] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.483240 22227 leader_election.cc:304] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:06.484215 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade9dda3178e42c783bc93e5deec21b7" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:06.485124 22512 raft_consensus.cc:493] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.485940 22541 raft_consensus.cc:2804] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:06.486640 22541 raft_consensus.cc:493] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.486032 22512 raft_consensus.cc:515] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.487159 22541 raft_consensus.cc:3060] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.485085 22186 raft_consensus.cc:2468] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 0.
I20260709 13:08:06.494921 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:06.496989 22152 leader_election.cc:343] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.497697 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7f82cbc4e8544bbac74ba96797d687c" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:06.498980 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7f82cbc4e8544bbac74ba96797d687c" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
W20260709 13:08:06.500986 22227 leader_election.cc:343] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:06.506696 22227 leader_election.cc:343] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.524315 22227 leader_election.cc:304] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:06.534577 22596 raft_consensus.cc:2749] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:06.506836 22594 raft_consensus.cc:493] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.494756 22455 raft_consensus.cc:359] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.536278 22455 raft_consensus.cc:385] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.536638 22455 raft_consensus.cc:740] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.520030 22537 raft_consensus.cc:515] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.541326 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:06.521366 22549 raft_consensus.cc:515] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:06.489203 22512 leader_election.cc:290] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
W20260709 13:08:06.542773 22152 leader_election.cc:343] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.527190 22457 tablet_bootstrap.cc:492] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:06.544803 22457 ts_tablet_manager.cc:1403] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.160s	user 0.013s	sys 0.000s
I20260709 13:08:06.528555 22486 raft_consensus.cc:2749] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:06.550091 22457 raft_consensus.cc:359] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.551157 22457 raft_consensus.cc:385] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.551522 22457 raft_consensus.cc:740] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.552420 22457 consensus_queue.cc:260] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.553193 22537 leader_election.cc:290] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:06.553825 22409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:06.535274 22555 raft_consensus.cc:515] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.537637 22455 consensus_queue.cc:260] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.525347 22541 raft_consensus.cc:515] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.555716 22457 ts_tablet_manager.cc:1434] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:08:06.556799 22457 tablet_bootstrap.cc:492] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:06.545526 22594 raft_consensus.cc:515] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:06.553258 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cadc140203246b088e6e2883bbe1977" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:06.558499 22260 raft_consensus.cc:3060] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.559175 22455 ts_tablet_manager.cc:1434] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.093s	user 0.006s	sys 0.002s
I20260709 13:08:06.555076 22409 raft_consensus.cc:3060] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.560215 22455 tablet_bootstrap.cc:492] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:06.555233 22549 leader_election.cc:290] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:06.564417 22457 tablet_bootstrap.cc:654] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.565478 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade9dda3178e42c783bc93e5deec21b7" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:06.566285 22408 raft_consensus.cc:3060] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.569443 22594 leader_election.cc:290] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:06.571110 22541 leader_election.cc:290] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:06.575984 22555 leader_election.cc:290] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:06.576205 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01b83067f9fa498487aeed4f60e8bfde" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:06.577675 22226 leader_election.cc:343] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.560150 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cadc140203246b088e6e2883bbe1977" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:06.595711 22335 raft_consensus.cc:3060] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.597273 22456 tablet_bootstrap.cc:492] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:06.597970 22456 ts_tablet_manager.cc:1403] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.179s	user 0.019s	sys 0.048s
I20260709 13:08:06.606451 22187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2144e27a177e41b0b22e474d9f209102" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:06.607273 22187 raft_consensus.cc:3060] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.607976 22408 raft_consensus.cc:2468] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:06.609508 22226 leader_election.cc:304] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:06.610769 22597 raft_consensus.cc:2804] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:06.611366 22597 raft_consensus.cc:697] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:06.612344 22597 consensus_queue.cc:237] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.616212 22187 raft_consensus.cc:2468] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:06.619737 22458 tablet_bootstrap.cc:492] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:06.620350 22458 ts_tablet_manager.cc:1403] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.237s	user 0.027s	sys 0.029s
I20260709 13:08:06.625798 22458 raft_consensus.cc:359] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.626921 22458 raft_consensus.cc:385] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.627296 22458 raft_consensus.cc:740] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.628489 22458 consensus_queue.cc:260] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.643474 22409 raft_consensus.cc:2468] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:06.645323 22151 leader_election.cc:304] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:06.647181 22537 raft_consensus.cc:2804] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:06.647899 22537 raft_consensus.cc:697] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:06.649067 22537 consensus_queue.cc:237] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.655121 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01b83067f9fa498487aeed4f60e8bfde" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:06.655580 22458 ts_tablet_manager.cc:1434] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.035s	user 0.011s	sys 0.000s
I20260709 13:08:06.657956 22187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade9dda3178e42c783bc93e5deec21b7" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:06.658922 22187 raft_consensus.cc:3060] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.660799 22455 tablet_bootstrap.cc:654] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.648314 22456 raft_consensus.cc:359] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.663273 22458 tablet_bootstrap.cc:492] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:06.664016 22456 raft_consensus.cc:385] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.664561 22456 raft_consensus.cc:740] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.665532 22456 consensus_queue.cc:260] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.666182 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2144e27a177e41b0b22e474d9f209102" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:06.668011 22456 ts_tablet_manager.cc:1434] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.000s
W20260709 13:08:06.668458 22227 leader_election.cc:343] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.669353 22456 tablet_bootstrap.cc:492] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:06.682013 22457 tablet_bootstrap.cc:492] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:06.682700 22457 ts_tablet_manager.cc:1403] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.126s	user 0.029s	sys 0.029s
I20260709 13:08:06.686200 22457 raft_consensus.cc:359] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.687069 22457 raft_consensus.cc:385] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.687390 22457 raft_consensus.cc:740] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.688082 22457 consensus_queue.cc:260] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:06.669450 22227 leader_election.cc:304] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:06.691143 22458 tablet_bootstrap.cc:654] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.698712 22187 raft_consensus.cc:2468] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:06.700088 22457 ts_tablet_manager.cc:1434] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:08:06.706038 22457 tablet_bootstrap.cc:492] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:06.709555 22227 leader_election.cc:304] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:06.710794 22555 raft_consensus.cc:2804] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:06.711398 22555 raft_consensus.cc:697] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:06.712450 22555 consensus_queue.cc:237] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:06.717662 22456 tablet_bootstrap.cc:654] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.719368 22597 raft_consensus.cc:2749] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:06.723505 22549 raft_consensus.cc:493] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.724103 22549 raft_consensus.cc:515] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:06.728030 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ad2369e00484995e87d12f574dfdc" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:06.728941 22334 raft_consensus.cc:2468] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:06.746333 22457 tablet_bootstrap.cc:654] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.703624 22260 raft_consensus.cc:2468] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:06.773686 22375 leader_election.cc:304] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
W20260709 13:08:06.805172 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 52041.9us
I20260709 13:08:06.811904 22465 raft_consensus.cc:2804] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:06.812613 22465 raft_consensus.cc:493] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.813071 22465 raft_consensus.cc:3060] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.751011 22335 raft_consensus.cc:2468] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:06.823361 22549 leader_election.cc:290] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:06.928057 22541 raft_consensus.cc:493] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.926759 22374 leader_election.cc:304] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:06.947155 22541 raft_consensus.cc:515] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.036922 22541 leader_election.cc:290] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:07.044672 22409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b51a3b1dc444319751dd92054d5dc3" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:07.085692 22494 raft_consensus.cc:2804] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:08:07.041810 22227 leader_election.cc:343] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:07.113817 22494 raft_consensus.cc:697] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:07.162704 22494 consensus_queue.cc:237] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:07.185963 22457 tablet_bootstrap.cc:492] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:07.187116 22457 ts_tablet_manager.cc:1403] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.482s	user 0.021s	sys 0.014s
W20260709 13:08:07.148725 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 114430us
I20260709 13:08:07.197482 22457 raft_consensus.cc:359] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.204598 22457 raft_consensus.cc:385] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.205240 22457 raft_consensus.cc:740] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.206307 22457 consensus_queue.cc:260] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
W20260709 13:08:07.259845 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 58848.9us
I20260709 13:08:07.271967 22455 tablet_bootstrap.cc:492] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:07.272758 22455 ts_tablet_manager.cc:1403] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.713s	user 0.037s	sys 0.047s
W20260709 13:08:07.260596 22555 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.008s	sys 0.007s
W20260709 13:08:07.243819 22458 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.007s	sys 0.005s
W20260709 13:08:07.288653 22458 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.008s	sys 0.005s
W20260709 13:08:07.288260 22555 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.009s	sys 0.007s
I20260709 13:08:07.364370 22457 ts_tablet_manager.cc:1434] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.177s	user 0.009s	sys 0.000s
I20260709 13:08:07.368110 22457 tablet_bootstrap.cc:492] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:07.372103 22458 tablet_bootstrap.cc:492] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:07.372813 22458 ts_tablet_manager.cc:1403] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.710s	user 0.018s	sys 0.009s
I20260709 13:08:07.295126 22465 raft_consensus.cc:515] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
W20260709 13:08:07.373262 22374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 492634us
W20260709 13:08:07.379025 22226 leader_election.cc:343] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:07.388774 22607 consensus_queue.cc:1048] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:07.395686 22455 raft_consensus.cc:359] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.397404 22455 raft_consensus.cc:385] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.397629 22455 raft_consensus.cc:740] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.406976 22456 tablet_bootstrap.cc:492] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:07.407699 22456 ts_tablet_manager.cc:1403] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.739s	user 0.026s	sys 0.030s
I20260709 13:08:07.408769 22458 raft_consensus.cc:359] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.409593 22458 raft_consensus.cc:385] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.410233 22457 tablet_bootstrap.cc:654] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.409852 22458 raft_consensus.cc:740] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.412271 22581 consensus_queue.cc:1048] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:07.412467 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ad2369e00484995e87d12f574dfdc" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:07.413947 22335 raft_consensus.cc:3060] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:07.415027 22455 consensus_queue.cc:260] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.412494 22458 consensus_queue.cc:260] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.429459 22456 raft_consensus.cc:359] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:07.430189 22456 raft_consensus.cc:385] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.430534 22456 raft_consensus.cc:740] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.431486 22456 consensus_queue.cc:260] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:07.433630 22456 ts_tablet_manager.cc:1434] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:08:07.434764 22456 tablet_bootstrap.cc:492] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:07.417346 22465 leader_election.cc:290] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:07.448158 22455 ts_tablet_manager.cc:1434] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.175s	user 0.009s	sys 0.001s
I20260709 13:08:07.449846 22455 tablet_bootstrap.cc:492] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:07.458370 22456 tablet_bootstrap.cc:654] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.452504 22607 raft_consensus.cc:493] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.459250 22607 raft_consensus.cc:515] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.461604 22581 raft_consensus.cc:493] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.462193 22581 raft_consensus.cc:515] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:07.464186 22457 tablet_bootstrap.cc:492] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:07.464506 22581 leader_election.cc:290] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:07.464781 22457 ts_tablet_manager.cc:1403] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.099s	user 0.015s	sys 0.002s
I20260709 13:08:07.466437 22607 leader_election.cc:290] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:07.472352 22457 raft_consensus.cc:359] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.473342 22457 raft_consensus.cc:385] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.473917 22457 raft_consensus.cc:740] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.477552 22457 consensus_queue.cc:260] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.478893 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:07.485319 22457 ts_tablet_manager.cc:1434] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:08:07.487264 22457 tablet_bootstrap.cc:492] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:07.489409 22410 raft_consensus.cc:2393] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93815010a884438c86fc6479359b69e8 in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:07.501729 22335 raft_consensus.cc:2468] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:07.503217 22457 tablet_bootstrap.cc:654] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.503480 22458 ts_tablet_manager.cc:1434] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.130s	user 0.012s	sys 0.025s
I20260709 13:08:07.505268 22612 raft_consensus.cc:493] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.509172 22458 tablet_bootstrap.cc:492] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:07.513202 22375 leader_election.cc:304] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:07.513868 22456 tablet_bootstrap.cc:492] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
W20260709 13:08:07.502576 22227 consensus_peers.cc:597] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.514447 22456 ts_tablet_manager.cc:1403] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.080s	user 0.021s	sys 0.027s
I20260709 13:08:07.514288 22605 raft_consensus.cc:2804] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:07.518478 22605 raft_consensus.cc:697] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:07.498986 22607 consensus_queue.cc:1048] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:08:07.519387 22597 raft_consensus.cc:493] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.519492 22605 consensus_queue.cc:237] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:07.520023 22597 raft_consensus.cc:515] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.521556 22456 raft_consensus.cc:359] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.522298 22456 raft_consensus.cc:385] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.522588 22456 raft_consensus.cc:740] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.523753 22456 consensus_queue.cc:260] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.525373 22597 leader_election.cc:290] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:07.533061 22456 ts_tablet_manager.cc:1434] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:08:07.509917 22612 raft_consensus.cc:515] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:07.534284 22456 tablet_bootstrap.cc:492] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:07.548792 22334 raft_consensus.cc:3060] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:07.549190 22455 tablet_bootstrap.cc:654] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.556622 22612 leader_election.cc:290] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:07.561363 22616 raft_consensus.cc:493] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bb26db1bd5a64429848b5d63b157c6f0)
W20260709 13:08:07.583535 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:07.579071 22616 raft_consensus.cc:515] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.659178 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ad2369e00484995e87d12f574dfdc" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:07.660408 22184 raft_consensus.cc:2468] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:07.661034 22616 leader_election.cc:290] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:07.711560 22616 raft_consensus.cc:493] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.712149 22616 raft_consensus.cc:515] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.714543 22616 leader_election.cc:290] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:07.732020 22617 raft_consensus.cc:493] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.732647 22617 raft_consensus.cc:515] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.734571 22617 leader_election.cc:290] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:07.737037 22185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b51a3b1dc444319751dd92054d5dc3" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:07.737825 22185 raft_consensus.cc:2468] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 0.
I20260709 13:08:07.740237 22185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:07.740803 22182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:07.740759 22457 tablet_bootstrap.cc:492] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:07.740921 22185 raft_consensus.cc:2468] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 0.
I20260709 13:08:07.742408 22457 ts_tablet_manager.cc:1403] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.256s	user 0.036s	sys 0.008s
I20260709 13:08:07.745718 22457 raft_consensus.cc:359] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.746476 22457 raft_consensus.cc:385] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.746781 22457 raft_consensus.cc:740] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.747529 22457 consensus_queue.cc:260] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.749158 22457 ts_tablet_manager.cc:1434] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:08:07.750213 22457 tablet_bootstrap.cc:492] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:07.757316 22457 tablet_bootstrap.cc:654] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.760371 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ad2369e00484995e87d12f574dfdc" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:07.761216 22181 raft_consensus.cc:3060] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:07.764770 22458 tablet_bootstrap.cc:700] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607: Time spent opening tablet: real 0.184s	user 0.001s	sys 0.002s
I20260709 13:08:07.765401 22457 tablet_bootstrap.cc:492] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:07.765508 22458 tablet_bootstrap.cc:654] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.766094 22457 ts_tablet_manager.cc:1403] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 13:08:07.769457 22457 raft_consensus.cc:359] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:07.770203 22457 raft_consensus.cc:385] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.770431 22457 raft_consensus.cc:740] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.771001 22457 consensus_queue.cc:260] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:07.771816 22458 tablet_bootstrap.cc:492] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:07.772414 22458 ts_tablet_manager.cc:1403] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.266s	user 0.009s	sys 0.009s
I20260709 13:08:07.772444 22457 ts_tablet_manager.cc:1434] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:08:07.773519 22457 tablet_bootstrap.cc:492] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:07.775803 22458 raft_consensus.cc:359] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.776580 22458 raft_consensus.cc:385] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.776918 22458 raft_consensus.cc:740] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.777848 22458 consensus_queue.cc:260] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.779747 22457 tablet_bootstrap.cc:654] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.780135 22617 raft_consensus.cc:493] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 67fc9e7fc4b34621ae1cd3e1621b89dc)
I20260709 13:08:07.780467 22458 ts_tablet_manager.cc:1434] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:08:07.780694 22617 raft_consensus.cc:515] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.782825 22458 tablet_bootstrap.cc:492] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:07.782871 22617 leader_election.cc:290] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:07.785005 22181 raft_consensus.cc:2468] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:07.784930 22406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ad2369e00484995e87d12f574dfdc" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:07.788245 22625 raft_consensus.cc:493] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:07.788872 22625 raft_consensus.cc:515] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:07.791955 22458 tablet_bootstrap.cc:654] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.791879 22625 leader_election.cc:290] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:07.792539 22625 raft_consensus.cc:493] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.792994 22625 raft_consensus.cc:515] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.795317 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:07.795650 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c995e1a8ce9c48aab8235f258f77d080" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:07.796746 22457 tablet_bootstrap.cc:492] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:07.797278 22457 ts_tablet_manager.cc:1403] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.024s	user 0.021s	sys 0.000s
W20260709 13:08:07.797425 22151 leader_election.cc:343] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:07.798084 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2144e27a177e41b0b22e474d9f209102" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:07.799203 22181 raft_consensus.cc:2393] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93815010a884438c86fc6479359b69e8 in current term 1: Already voted for candidate bb26db1bd5a64429848b5d63b157c6f0 in this term.
I20260709 13:08:07.799706 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c995e1a8ce9c48aab8235f258f77d080" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:07.800395 22404 raft_consensus.cc:2468] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:07.800735 22457 raft_consensus.cc:359] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.801443 22457 raft_consensus.cc:385] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.801740 22457 raft_consensus.cc:740] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.801841 22625 leader_election.cc:290] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:07.801968 22300 leader_election.cc:304] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607
I20260709 13:08:07.802620 22457 consensus_queue.cc:260] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.804332 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:07.804549 22457 ts_tablet_manager.cc:1434] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:08:07.805330 22625 raft_consensus.cc:2804] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:07.805853 22625 raft_consensus.cc:493] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:07.806414 22625 raft_consensus.cc:3060] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.808636 22458 tablet_bootstrap.cc:492] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:07.809243 22458 ts_tablet_manager.cc:1403] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.009s
I20260709 13:08:07.812458 22458 raft_consensus.cc:359] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.813163 22458 raft_consensus.cc:385] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.813472 22458 raft_consensus.cc:740] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.814347 22458 consensus_queue.cc:260] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.815905 22605 raft_consensus.cc:493] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:07.805819 22457 tablet_bootstrap.cc:492] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:07.816303 22616 consensus_queue.cc:1048] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:07.816553 22605 raft_consensus.cc:515] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.817318 22458 ts_tablet_manager.cc:1434] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:08:07.818511 22458 tablet_bootstrap.cc:492] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:07.822356 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ad2369e00484995e87d12f574dfdc" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:07.823192 22336 raft_consensus.cc:2393] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 652d70a0b765447792ab9b4383712607 in current term 1: Already voted for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in this term.
I20260709 13:08:07.824047 22457 tablet_bootstrap.cc:654] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.824563 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3a61ae21bab486cbfa407942fdeac35" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:07.825295 22458 tablet_bootstrap.cc:654] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.816363 22625 raft_consensus.cc:515] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:07.826354 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7f82cbc4e8544bbac74ba96797d687c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:07.827107 22334 raft_consensus.cc:1240] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Rejecting Update request from peer 652d70a0b765447792ab9b4383712607 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:07.828945 22625 leader_election.cc:290] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:07.833593 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c995e1a8ce9c48aab8235f258f77d080" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:07.834486 22404 raft_consensus.cc:3060] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.838977 22458 tablet_bootstrap.cc:492] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:07.839619 22458 ts_tablet_manager.cc:1403] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 13:08:07.843257 22605 leader_election.cc:290] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:07.843451 22152 leader_election.cc:304] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:07.843930 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c995e1a8ce9c48aab8235f258f77d080" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:07.845104 22617 raft_consensus.cc:2749] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:08:07.847169 22152 leader_election.cc:343] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:07.847043 22625 consensus_queue.cc:1048] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:07.847581 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:07.848631 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:07.850090 22301 leader_election.cc:304] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:07.850844 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:07.853269 22402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:07.857600 22625 raft_consensus.cc:2749] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:07.861534 22605 consensus_queue.cc:1048] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:07.867764 22404 raft_consensus.cc:2468] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 2.
I20260709 13:08:07.868796 22410 raft_consensus.cc:1240] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Rejecting Update request from peer 652d70a0b765447792ab9b4383712607 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:07.870008 22300 leader_election.cc:304] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607
I20260709 13:08:07.870918 22581 raft_consensus.cc:2804] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:07.871429 22581 raft_consensus.cc:697] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 2 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
I20260709 13:08:07.868139 22458 raft_consensus.cc:359] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.872376 22581 consensus_queue.cc:237] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:07.872810 22457 tablet_bootstrap.cc:492] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:07.873477 22457 ts_tablet_manager.cc:1403] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.068s	user 0.024s	sys 0.025s
I20260709 13:08:07.872893 22458 raft_consensus.cc:385] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.874982 22615 consensus_queue.cc:1048] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:07.876863 22457 raft_consensus.cc:359] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:07.877588 22457 raft_consensus.cc:385] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.877859 22457 raft_consensus.cc:740] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.878597 22457 consensus_queue.cc:260] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:07.880165 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:07.880712 22457 ts_tablet_manager.cc:1434] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:08:07.880846 22261 raft_consensus.cc:2468] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:07.881829 22457 tablet_bootstrap.cc:492] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:07.891335 22458 raft_consensus.cc:740] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.894412 22458 consensus_queue.cc:260] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.897713 22458 ts_tablet_manager.cc:1434] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.002s
I20260709 13:08:07.899974 22458 tablet_bootstrap.cc:492] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:07.910419 22458 tablet_bootstrap.cc:654] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.892611 22457 tablet_bootstrap.cc:654] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.930472 22616 consensus_queue.cc:1059] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:07.931758 22616 raft_consensus.cc:3055] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:07.932159 22616 raft_consensus.cc:740] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:07.933369 22616 consensus_queue.cc:260] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:07.936411 22616 raft_consensus.cc:3060] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.941890 22458 tablet_bootstrap.cc:492] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:07.942790 22458 ts_tablet_manager.cc:1403] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.007s
I20260709 13:08:07.943373 22457 tablet_bootstrap.cc:492] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:07.943893 22457 ts_tablet_manager.cc:1403] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.062s	user 0.004s	sys 0.008s
I20260709 13:08:07.947120 22457 raft_consensus.cc:359] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:07.947850 22457 raft_consensus.cc:385] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.948158 22457 raft_consensus.cc:740] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.948848 22457 consensus_queue.cc:260] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:07.950896 22457 ts_tablet_manager.cc:1434] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:08:07.951967 22457 tablet_bootstrap.cc:492] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:07.953405 22465 consensus_queue.cc:1048] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:08:07.954202 22226 leader_election.cc:343] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:07.955075 22226 leader_election.cc:343] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:07.958634 22457 tablet_bootstrap.cc:654] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.962766 22458 raft_consensus.cc:359] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:07.966022 22615 raft_consensus.cc:493] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.966683 22615 raft_consensus.cc:515] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.968787 22458 raft_consensus.cc:385] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.971380 22613 raft_consensus.cc:493] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.972450 22615 leader_election.cc:290] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:07.972016 22613 raft_consensus.cc:515] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:07.969460 22458 raft_consensus.cc:740] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.988729 22456 tablet_bootstrap.cc:700] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8: Time spent opening tablet: real 0.341s	user 0.003s	sys 0.000s
I20260709 13:08:07.989584 22456 tablet_bootstrap.cc:654] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.992043 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9f843a01847a4b475b6dbcb704a6f" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:07.993902 22458 consensus_queue.cc:260] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.010015 22455 tablet_bootstrap.cc:492] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:08.013423 22455 ts_tablet_manager.cc:1403] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.564s	user 0.021s	sys 0.012s
I20260709 13:08:08.018011 22458 ts_tablet_manager.cc:1434] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.075s	user 0.009s	sys 0.001s
I20260709 13:08:07.981225 22613 leader_election.cc:290] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:07.999274 22607 raft_consensus.cc:493] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.027336 22458 tablet_bootstrap.cc:492] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:08.033692 22613 raft_consensus.cc:493] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.051620 22457 tablet_bootstrap.cc:492] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:08.052161 22457 ts_tablet_manager.cc:1403] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.100s	user 0.029s	sys 0.041s
I20260709 13:08:08.033277 22607 raft_consensus.cc:515] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.061372 22606 raft_consensus.cc:493] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:08.062636 22152 leader_election.cc:343] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.062027 22606 raft_consensus.cc:515] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.063710 22334 raft_consensus.cc:3060] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.071712 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01099e972e9b4d08a64552be3fd52276" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.072153 22606 leader_election.cc:290] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:08.048362 22613 raft_consensus.cc:515] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.080727 22607 leader_election.cc:290] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
W20260709 13:08:08.087354 22152 leader_election.cc:343] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.087569 22183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b51a3b1dc444319751dd92054d5dc3" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:08.099349 22183 raft_consensus.cc:2468] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
W20260709 13:08:08.074203 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 78110.9us
I20260709 13:08:08.097856 22613 leader_election.cc:290] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:08.108317 22458 tablet_bootstrap.cc:654] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.106632 22375 leader_election.cc:304] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:08.119911 22605 raft_consensus.cc:2804] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.120371 22605 raft_consensus.cc:493] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.120729 22605 raft_consensus.cc:3060] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.122253 22455 raft_consensus.cc:359] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.123041 22455 raft_consensus.cc:385] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.123340 22455 raft_consensus.cc:740] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.129798 22455 consensus_queue.cc:260] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.135324 22617 raft_consensus.cc:493] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.141455 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4c4a9cd1164b7997f7188f9da9f70b" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.142344 22617 raft_consensus.cc:515] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.144747 22617 leader_election.cc:290] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:08.146095 22555 consensus_queue.cc:1048] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:08.146543 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f204603e14c438bb549d1116eaac788" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:08.148659 22648 raft_consensus.cc:493] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.149296 22648 raft_consensus.cc:515] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
W20260709 13:08:08.149710 22151 leader_election.cc:343] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.151700 22648 leader_election.cc:290] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:08.152163 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2144e27a177e41b0b22e474d9f209102" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.155699 22374 leader_election.cc:304] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607
I20260709 13:08:08.158660 22606 raft_consensus.cc:2804] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:08.158407 22457 raft_consensus.cc:359] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.159121 22606 raft_consensus.cc:493] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:08.159672 22606 raft_consensus.cc:3060] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.162343 22085 catalog_manager.cc:5697] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:08.159276 22457 raft_consensus.cc:385] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.164639 22085 catalog_manager.cc:5697] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:08.163470 22494 consensus_queue.cc:1048] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:08.165280 22457 raft_consensus.cc:740] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.164408 22083 catalog_manager.cc:5697] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:08.171609 22457 consensus_queue.cc:260] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.173907 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:08.174695 22455 ts_tablet_manager.cc:1434] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.160s	user 0.014s	sys 0.002s
I20260709 13:08:08.174674 22300 leader_election.cc:304] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
W20260709 13:08:08.175612 22226 leader_election.cc:343] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.175879 22455 tablet_bootstrap.cc:492] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:08.177726 22457 ts_tablet_manager.cc:1434] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.125s	user 0.009s	sys 0.000s
I20260709 13:08:08.178857 22457 tablet_bootstrap.cc:492] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:08.185621 22613 raft_consensus.cc:493] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.187743 22455 tablet_bootstrap.cc:654] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.186316 22613 raft_consensus.cc:515] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.187718 22082 catalog_manager.cc:5697] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:08.185421 22606 raft_consensus.cc:515] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.192699 22456 tablet_bootstrap.cc:492] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:08.198321 22456 ts_tablet_manager.cc:1403] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.664s	user 0.022s	sys 0.017s
I20260709 13:08:08.200248 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2144e27a177e41b0b22e474d9f209102" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.201640 22456 raft_consensus.cc:359] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.202399 22613 leader_election.cc:290] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:08.202611 22456 raft_consensus.cc:385] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.203184 22456 raft_consensus.cc:740] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
W20260709 13:08:08.203922 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 87876.8us
I20260709 13:08:08.204831 22227 leader_election.cc:304] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:08.190403 22494 consensus_queue.cc:1048] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:08:08.205848 22456 consensus_queue.cc:260] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.207029 22605 raft_consensus.cc:515] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.208416 22456 ts_tablet_manager.cc:1434] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:08:08.209486 22456 tablet_bootstrap.cc:492] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:08.214995 22646 raft_consensus.cc:2804] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.214920 22597 consensus_queue.cc:1048] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:08.215605 22183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b51a3b1dc444319751dd92054d5dc3" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:08.216406 22183 raft_consensus.cc:3060] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.216795 22605 leader_election.cc:290] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:08.215672 22646 raft_consensus.cc:493] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.218647 22646 raft_consensus.cc:3060] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.219772 22597 raft_consensus.cc:493] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.220268 22597 raft_consensus.cc:515] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.225960 22597 leader_election.cc:290] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:08.226828 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f204603e14c438bb549d1116eaac788" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:08.227154 22455 tablet_bootstrap.cc:492] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:08.227413 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b51a3b1dc444319751dd92054d5dc3" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.228019 22606 leader_election.cc:290] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:08.227662 22455 ts_tablet_manager.cc:1403] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.020s
W20260709 13:08:08.228749 22226 leader_election.cc:343] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.231523 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
W20260709 13:08:08.231700 22227 leader_election.cc:343] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.232250 22227 leader_election.cc:304] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:08.232530 22455 raft_consensus.cc:359] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.233325 22455 raft_consensus.cc:385] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.233603 22455 raft_consensus.cc:740] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.234805 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b51a3b1dc444319751dd92054d5dc3" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:08.235509 22607 raft_consensus.cc:2749] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:08.236236 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:08.236938 22262 raft_consensus.cc:3060] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.236174 22607 consensus_queue.cc:1048] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:08.239564 22456 tablet_bootstrap.cc:654] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.240685 22455 consensus_queue.cc:260] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.243368 22455 ts_tablet_manager.cc:1434] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.002s
I20260709 13:08:08.244606 22455 tablet_bootstrap.cc:492] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:08.263449 22455 tablet_bootstrap.cc:654] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.264750 22262 raft_consensus.cc:2468] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 2.
I20260709 13:08:08.264377 22613 raft_consensus.cc:493] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.266592 22613 raft_consensus.cc:515] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.269415 22605 raft_consensus.cc:493] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.233881 22227 leader_election.cc:304] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:08.270198 22605 raft_consensus.cc:515] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.269901 22374 leader_election.cc:304] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607
I20260709 13:08:08.275871 22578 raft_consensus.cc:2749] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:08.279670 22655 raft_consensus.cc:2804] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:08.281529 22613 leader_election.cc:290] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:08.282896 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:08.284852 22151 leader_election.cc:343] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.288940 22605 leader_election.cc:290] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:08.288712 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:08.289692 22607 raft_consensus.cc:2804] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.289702 22184 raft_consensus.cc:2468] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:08.290874 22655 raft_consensus.cc:697] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:08.291740 22375 leader_election.cc:304] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:08.291863 22655 consensus_queue.cc:237] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.293286 22465 raft_consensus.cc:2804] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.293810 22465 raft_consensus.cc:493] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.294262 22465 raft_consensus.cc:3060] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.296059 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.290354 22607 raft_consensus.cc:493] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.305820 22457 tablet_bootstrap.cc:654] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.306725 22183 raft_consensus.cc:2468] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:08.310364 22375 leader_election.cc:304] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:08.311883 22605 raft_consensus.cc:2804] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:08.312548 22605 raft_consensus.cc:697] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:08.312837 22465 raft_consensus.cc:515] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.313756 22605 consensus_queue.cc:237] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.299980 22607 raft_consensus.cc:3060] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.317508 22465 leader_election.cc:290] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:08.319031 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:08.319658 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:08.320111 22456 tablet_bootstrap.cc:492] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:08.327437 22456 ts_tablet_manager.cc:1403] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.118s	user 0.013s	sys 0.005s
I20260709 13:08:08.329596 22184 raft_consensus.cc:3060] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.329983 22458 tablet_bootstrap.cc:492] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:08.330557 22458 ts_tablet_manager.cc:1403] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.304s	user 0.019s	sys 0.024s
I20260709 13:08:08.333905 22458 raft_consensus.cc:359] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:08.334682 22458 raft_consensus.cc:385] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:08:08.300931 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 66404.2us
I20260709 13:08:08.348598 22408 raft_consensus.cc:1275] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer 93815010a884438c86fc6479359b69e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:08.348963 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3a61ae21bab486cbfa407942fdeac35" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.351716 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7f82cbc4e8544bbac74ba96797d687c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.352387 22260 raft_consensus.cc:2468] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:08.334929 22458 raft_consensus.cc:740] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.337147 22597 raft_consensus.cc:493] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.354074 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.354127 22597 raft_consensus.cc:515] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.339902 22456 raft_consensus.cc:359] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.300396 22646 raft_consensus.cc:515] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.355427 22258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.365042 22456 raft_consensus.cc:385] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.366751 22456 raft_consensus.cc:740] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.367589 22456 consensus_queue.cc:260] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.368108 22458 consensus_queue.cc:260] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:08.374311 22456 ts_tablet_manager.cc:1434] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.005s
I20260709 13:08:08.376782 22456 tablet_bootstrap.cc:492] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:08.366122 22578 consensus_queue.cc:1048] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:08:08.378443 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9f843a01847a4b475b6dbcb704a6f" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.379333 22260 raft_consensus.cc:2468] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:08.380116 22597 leader_election.cc:290] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:08.376231 22259 raft_consensus.cc:1240] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Rejecting Update request from peer 652d70a0b765447792ab9b4383712607 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:08.345880 22184 raft_consensus.cc:2468] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:08.384119 22458 ts_tablet_manager.cc:1434] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.001s
I20260709 13:08:08.386771 22458 tablet_bootstrap.cc:492] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:08.384653 22261 raft_consensus.cc:2393] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 1: Already voted for candidate bb26db1bd5a64429848b5d63b157c6f0 in this term.
I20260709 13:08:08.390728 22375 leader_election.cc:304] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:08.391743 22605 raft_consensus.cc:2804] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:08.392387 22605 raft_consensus.cc:697] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:08.393353 22605 consensus_queue.cc:237] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.394551 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01099e972e9b4d08a64552be3fd52276" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.397100 22260 raft_consensus.cc:2468] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:08.395473 22455 tablet_bootstrap.cc:492] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:08.398368 22455 ts_tablet_manager.cc:1403] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.154s	user 0.017s	sys 0.029s
I20260709 13:08:08.402181 22455 raft_consensus.cc:359] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.403016 22455 raft_consensus.cc:385] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.403340 22455 raft_consensus.cc:740] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.405793 22407 raft_consensus.cc:1240] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Rejecting Update request from peer 652d70a0b765447792ab9b4383712607 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:08.408581 22613 consensus_queue.cc:1059] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:08.409122 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b51a3b1dc444319751dd92054d5dc3" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:08.409428 22613 consensus_queue.cc:1243] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [LEADER]: Peer 67fc9e7fc4b34621ae1cd3e1621b89dc log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:08:08.409962 22613 raft_consensus.cc:3055] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:08.410323 22613 raft_consensus.cc:740] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:08.410352 22455 consensus_queue.cc:260] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.411314 22613 consensus_queue.cc:260] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.412508 22613 raft_consensus.cc:3060] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.415591 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4c4a9cd1164b7997f7188f9da9f70b" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.416210 22597 raft_consensus.cc:493] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.416765 22597 raft_consensus.cc:515] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.417388 22646 leader_election.cc:290] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:08.420612 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f757dd6e4b94a8a84bfaddd602685e4" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:08.396461 22457 tablet_bootstrap.cc:492] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:08.421613 22457 ts_tablet_manager.cc:1403] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.243s	user 0.020s	sys 0.010s
W20260709 13:08:08.422431 22226 leader_election.cc:343] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.423031 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f204603e14c438bb549d1116eaac788" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.423771 22260 raft_consensus.cc:2468] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:08.423245 22597 leader_election.cc:290] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:08.425019 22457 raft_consensus.cc:359] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.425776 22457 raft_consensus.cc:385] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.414510 22455 ts_tablet_manager.cc:1434] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:08:08.426749 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2144e27a177e41b0b22e474d9f209102" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.427016 22455 tablet_bootstrap.cc:492] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:08.382174 22607 raft_consensus.cc:515] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.432999 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.433264 22262 raft_consensus.cc:2393] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 1: Already voted for candidate bb26db1bd5a64429848b5d63b157c6f0 in this term.
I20260709 13:08:08.433813 22259 raft_consensus.cc:2468] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:08.426044 22457 raft_consensus.cc:740] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.451094 22456 tablet_bootstrap.cc:654] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.435340 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:08.452589 22457 consensus_queue.cc:260] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.455823 22458 tablet_bootstrap.cc:654] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.459438 22457 ts_tablet_manager.cc:1434] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:08:08.459869 22455 tablet_bootstrap.cc:654] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.449272 22597 raft_consensus.cc:493] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.461107 22457 tablet_bootstrap.cc:492] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:08.458261 22658 raft_consensus.cc:493] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.461557 22597 raft_consensus.cc:515] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.468184 22597 leader_election.cc:290] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:08.469206 22578 raft_consensus.cc:493] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.469094 22183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9f843a01847a4b475b6dbcb704a6f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:08.469988 22183 raft_consensus.cc:2468] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 0.
I20260709 13:08:08.469826 22578 raft_consensus.cc:515] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.471984 22457 tablet_bootstrap.cc:654] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.462945 22658 raft_consensus.cc:515] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.472162 22578 leader_election.cc:290] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:08.474011 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01b83067f9fa498487aeed4f60e8bfde" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.474305 22658 leader_election.cc:290] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:08.474813 22260 raft_consensus.cc:2468] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 0.
I20260709 13:08:08.475562 22607 leader_election.cc:290] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:08.479221 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9f843a01847a4b475b6dbcb704a6f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.482470 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01b83067f9fa498487aeed4f60e8bfde" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.485831 22300 leader_election.cc:304] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:08.489660 22183 raft_consensus.cc:1275] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Refusing update from remote peer 93815010a884438c86fc6479359b69e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:08.494457 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0418b53b8c2e4786b445d55f0ea5a723" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:08.494989 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01b83067f9fa498487aeed4f60e8bfde" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
W20260709 13:08:08.495955 22226 leader_election.cc:343] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.495994 22260 raft_consensus.cc:2468] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:08.498461 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01b83067f9fa498487aeed4f60e8bfde" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:08.513234 22374 leader_election.cc:304] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:08.499306 22334 raft_consensus.cc:2468] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:08.521379 22578 raft_consensus.cc:2804] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.522790 22578 raft_consensus.cc:493] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.523262 22578 raft_consensus.cc:3060] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.528354 22626 consensus_queue.cc:1048] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:08.528966 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01099e972e9b4d08a64552be3fd52276" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.530330 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f757dd6e4b94a8a84bfaddd602685e4" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.532977 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.499228 22407 raft_consensus.cc:2468] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 0.
W20260709 13:08:08.545675 22227 leader_election.cc:343] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.553478 22605 raft_consensus.cc:2804] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.572458 22605 raft_consensus.cc:493] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.572999 22605 raft_consensus.cc:3060] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.573756 22455 tablet_bootstrap.cc:492] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:08.574361 22455 ts_tablet_manager.cc:1403] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.148s	user 0.022s	sys 0.043s
I20260709 13:08:08.556567 22151 leader_election.cc:304] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:08.579038 22455 raft_consensus.cc:359] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:08.579751 22455 raft_consensus.cc:385] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.580996 22455 raft_consensus.cc:740] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.581786 22455 consensus_queue.cc:260] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:08.582334 22457 tablet_bootstrap.cc:492] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:08.582857 22457 ts_tablet_manager.cc:1403] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.123s	user 0.051s	sys 0.032s
I20260709 13:08:08.584043 22455 ts_tablet_manager.cc:1434] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:08:08.583750 22578 raft_consensus.cc:515] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.585052 22455 tablet_bootstrap.cc:492] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:08.586007 22457 raft_consensus.cc:359] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.586797 22457 raft_consensus.cc:385] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.587003 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01b83067f9fa498487aeed4f60e8bfde" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:08.587431 22457 raft_consensus.cc:740] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.587736 22260 raft_consensus.cc:3060] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.588443 22457 consensus_queue.cc:260] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.599025 22458 tablet_bootstrap.cc:492] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:08.600641 22458 ts_tablet_manager.cc:1403] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.215s	user 0.014s	sys 0.020s
I20260709 13:08:08.604750 22456 tablet_bootstrap.cc:492] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:08.580168 22616 raft_consensus.cc:2749] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.605522 22456 ts_tablet_manager.cc:1403] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.229s	user 0.022s	sys 0.007s
I20260709 13:08:08.600414 22626 raft_consensus.cc:493] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.608625 22457 ts_tablet_manager.cc:1434] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.001s
W20260709 13:08:08.609186 22153 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.609833 22457 tablet_bootstrap.cc:492] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:08.579535 22151 leader_election.cc:304] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:08.611172 22613 raft_consensus.cc:2804] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.611541 22613 raft_consensus.cc:493] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.611873 22613 raft_consensus.cc:3060] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.612946 22456 raft_consensus.cc:359] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:08.613683 22456 raft_consensus.cc:385] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.608814 22626 raft_consensus.cc:515] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.613982 22456 raft_consensus.cc:740] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.615881 22456 consensus_queue.cc:260] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
W20260709 13:08:08.594790 22227 leader_election.cc:343] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.614806 22151 leader_election.cc:304] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:08.623818 22616 raft_consensus.cc:2749] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.624469 22151 leader_election.cc:304] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:08.625360 22616 raft_consensus.cc:2749] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:08.629464 22626 leader_election.cc:290] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:08.629855 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7f82cbc4e8544bbac74ba96797d687c" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:08.633468 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7f82cbc4e8544bbac74ba96797d687c" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:08.633859 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01b83067f9fa498487aeed4f60e8bfde" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:08.634243 22259 raft_consensus.cc:2468] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 0.
I20260709 13:08:08.625829 22151 leader_election.cc:304] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:08.635605 22300 leader_election.cc:304] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:08.635759 22616 raft_consensus.cc:2804] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.636233 22616 raft_consensus.cc:493] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.636495 22626 raft_consensus.cc:2804] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.636605 22616 raft_consensus.cc:3060] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.636999 22626 raft_consensus.cc:493] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.637461 22626 raft_consensus.cc:3060] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.636711 22151 leader_election.cc:304] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
W20260709 13:08:08.631943 22227 leader_election.cc:343] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.621978 22578 leader_election.cc:290] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:08.639943 22617 raft_consensus.cc:2804] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.646761 22617 raft_consensus.cc:493] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.646428 22613 raft_consensus.cc:515] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.647171 22617 raft_consensus.cc:3060] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.647754 22184 raft_consensus.cc:2393] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93815010a884438c86fc6479359b69e8 in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:08.640609 22151 leader_election.cc:304] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:08.637909 22456 ts_tablet_manager.cc:1434] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:08:08.644719 22457 tablet_bootstrap.cc:654] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.652082 22456 tablet_bootstrap.cc:492] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:08.653167 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7f82cbc4e8544bbac74ba96797d687c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:08.657358 22260 raft_consensus.cc:2468] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:08.642633 22227 leader_election.cc:304] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:08.675045 22607 raft_consensus.cc:493] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:08.664459 22208 debug-util.cc:398] Leaking SignalData structure 0x7b0800304180 after lost signal to thread 12728
I20260709 13:08:08.675678 22607 raft_consensus.cc:515] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:08.676707 22208 debug-util.cc:398] Leaking SignalData structure 0x7b08003aa3a0 after lost signal to thread 22119
I20260709 13:08:08.678433 22656 raft_consensus.cc:2749] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:08.667308 22655 raft_consensus.cc:493] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.665057 22626 raft_consensus.cc:515] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.667419 22458 raft_consensus.cc:359] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:08.679651 22458 raft_consensus.cc:385] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.679374 22655 raft_consensus.cc:515] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.679978 22458 raft_consensus.cc:740] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.681164 22626 leader_election.cc:290] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:08.654237 22613 leader_election.cc:290] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:08.681624 22655 leader_election.cc:290] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:08.682037 22334 raft_consensus.cc:2393] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 652d70a0b765447792ab9b4383712607 in current term 1: Already voted for candidate 93815010a884438c86fc6479359b69e8 in this term.
I20260709 13:08:08.682192 22613 raft_consensus.cc:2749] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.665947 22084 catalog_manager.cc:5697] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:08.684098 22458 consensus_queue.cc:260] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:08.685935 22458 ts_tablet_manager.cc:1434] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.085s	user 0.006s	sys 0.001s
I20260709 13:08:08.686836 22656 raft_consensus.cc:493] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.687417 22656 raft_consensus.cc:515] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.690455 22656 leader_election.cc:290] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:08.691675 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e49aee5d99347029a0f5e51fe01b896" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:08.692181 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.691195 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
W20260709 13:08:08.674989 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 47145.2us
I20260709 13:08:08.693210 22458 tablet_bootstrap.cc:492] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:08.677809 22607 leader_election.cc:290] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:08.670902 22617 raft_consensus.cc:515] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.703895 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7f82cbc4e8544bbac74ba96797d687c" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:08.704586 22262 raft_consensus.cc:3060] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.704835 22300 leader_election.cc:304] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:08.706012 22626 raft_consensus.cc:2804] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:08.706534 22626 raft_consensus.cc:697] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
W20260709 13:08:08.707211 22208 debug-util.cc:398] Leaking SignalData structure 0x7b0800126e80 after lost signal to thread 22211
W20260709 13:08:08.707880 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:08:08.708774 22208 debug-util.cc:398] Leaking SignalData structure 0x7b080012b6e0 after lost signal to thread 22286
I20260709 13:08:08.707435 22626 consensus_queue.cc:237] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
W20260709 13:08:08.699857 22151 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 59347.6us
W20260709 13:08:08.709261 22208 debug-util.cc:398] Leaking SignalData structure 0x7b080012b700 after lost signal to thread 22287
I20260709 13:08:08.719763 22617 leader_election.cc:290] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:08.725124 22456 tablet_bootstrap.cc:654] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:08.710027 22226 leader_election.cc:343] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.702711 22259 raft_consensus.cc:2468] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
W20260709 13:08:08.711684 22375 leader_election.cc:343] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:08.715304 22227 leader_election.cc:343] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.699939 22605 raft_consensus.cc:515] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.737560 22408 raft_consensus.cc:2393] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 93815010a884438c86fc6479359b69e8 in current term 1: Already voted for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in this term.
I20260709 13:08:08.733079 22616 raft_consensus.cc:515] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.738654 22374 leader_election.cc:304] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:08.739648 22655 raft_consensus.cc:2804] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.740033 22655 raft_consensus.cc:493] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.740371 22655 raft_consensus.cc:3060] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.741343 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9f843a01847a4b475b6dbcb704a6f" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:08.741757 22455 tablet_bootstrap.cc:654] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:08.743729 22152 leader_election.cc:343] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:08.734419 22208 debug-util.cc:398] Leaking SignalData structure 0x7b080024da60 after lost signal to thread 22360
I20260709 13:08:08.734565 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01099e972e9b4d08a64552be3fd52276" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:08.736227 22227 leader_election.cc:304] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:08.750402 22656 raft_consensus.cc:2749] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:08.752682 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.754119 22262 raft_consensus.cc:2468] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:08.756229 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0418b53b8c2e4786b445d55f0ea5a723" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.773749 22605 leader_election.cc:290] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:08.774178 22300 leader_election.cc:304] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:08.774953 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01b83067f9fa498487aeed4f60e8bfde" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:08.775465 22581 raft_consensus.cc:2804] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:08.775907 22259 raft_consensus.cc:2393] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 1: Already voted for candidate 93815010a884438c86fc6479359b69e8 in this term.
I20260709 13:08:08.758260 22227 leader_election.cc:304] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
W20260709 13:08:08.763767 22152 leader_election.cc:343] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.776970 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01b83067f9fa498487aeed4f60e8bfde" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
W20260709 13:08:08.778592 22227 leader_election.cc:343] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:08.745947 22208 debug-util.cc:398] Leaking SignalData structure 0x7b080012c480 after lost signal to thread 22434
W20260709 13:08:08.779554 22208 debug-util.cc:398] Leaking SignalData structure 0x7b08001322e0 after lost signal to thread 22496
I20260709 13:08:08.779922 22607 raft_consensus.cc:2804] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.765816 22617 raft_consensus.cc:493] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.771692 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7f82cbc4e8544bbac74ba96797d687c" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:08.780354 22607 raft_consensus.cc:493] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.780798 22607 raft_consensus.cc:3060] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.781127 22184 raft_consensus.cc:2393] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 93815010a884438c86fc6479359b69e8 in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:08.776046 22581 raft_consensus.cc:697] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
I20260709 13:08:08.783440 22374 leader_election.cc:304] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:08.783007 22581 consensus_queue.cc:237] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.784492 22617 raft_consensus.cc:515] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:08.786310 22658 raft_consensus.cc:2749] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:08.787019 22617 leader_election.cc:290] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:08.764971 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e49aee5d99347029a0f5e51fe01b896" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.788307 22616 leader_election.cc:290] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
W20260709 13:08:08.793179 22227 leader_election.cc:343] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.793808 22227 leader_election.cc:304] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:08.794847 22656 raft_consensus.cc:2749] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:08.804071 22655 raft_consensus.cc:515] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.806994 22655 leader_election.cc:290] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:08.807969 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:08.808758 22259 raft_consensus.cc:3060] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.796481 22151 leader_election.cc:304] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:08.815039 22617 raft_consensus.cc:2804] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.815434 22617 raft_consensus.cc:493] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:08.815382 22151 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 83278.9us
I20260709 13:08:08.815996 22617 raft_consensus.cc:3060] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.816706 22151 leader_election.cc:304] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:08.817647 22683 raft_consensus.cc:2749] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.817752 22259 raft_consensus.cc:2468] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:08.818305 22151 leader_election.cc:304] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:08.823530 22683 raft_consensus.cc:2804] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.823935 22683 raft_consensus.cc:493] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.824294 22683 raft_consensus.cc:3060] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.819252 22374 leader_election.cc:304] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:08.825742 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8f7472c97049ca98e597c7d4b4eb1c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.826655 22333 raft_consensus.cc:2468] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:08.827152 22605 raft_consensus.cc:2804] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:08.820072 22658 consensus_queue.cc:1048] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:08.834663 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:08.846694 22183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f204603e14c438bb549d1116eaac788" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:08.835256 22607 raft_consensus.cc:515] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:08.859398 22465 consensus_queue.cc:1048] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:08.859891 22456 tablet_bootstrap.cc:492] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:08.860781 22456 ts_tablet_manager.cc:1403] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.209s	user 0.012s	sys 0.014s
I20260709 13:08:08.849920 22457 tablet_bootstrap.cc:492] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:08.850040 22458 tablet_bootstrap.cc:654] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.835813 22152 leader_election.cc:304] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:08.863360 22457 ts_tablet_manager.cc:1403] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.254s	user 0.046s	sys 0.011s
I20260709 13:08:08.843786 22082 catalog_manager.cc:5697] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:08.836067 22227 leader_election.cc:343] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:08.868333 22375 leader_election.cc:343] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.868887 22227 leader_election.cc:304] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:08.869194 22607 leader_election.cc:290] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:08.869798 22607 raft_consensus.cc:2749] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:08.873330 22684 raft_consensus.cc:2804] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.873811 22684 raft_consensus.cc:493] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.874194 22684 raft_consensus.cc:3060] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.875272 22457 raft_consensus.cc:359] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.876083 22457 raft_consensus.cc:385] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.876353 22457 raft_consensus.cc:740] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.877148 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9f843a01847a4b475b6dbcb704a6f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:08.879195 22187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01099e972e9b4d08a64552be3fd52276" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:08.855686 22605 raft_consensus.cc:697] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:08.864804 22082 catalog_manager.cc:5697] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 2 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:08.886696 22457 consensus_queue.cc:260] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.885316 22082 catalog_manager.cc:5697] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:08.891659 22455 tablet_bootstrap.cc:492] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:08.892516 22455 ts_tablet_manager.cc:1403] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.308s	user 0.016s	sys 0.017s
I20260709 13:08:08.899410 22683 raft_consensus.cc:515] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.901964 22187 raft_consensus.cc:2393] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb26db1bd5a64429848b5d63b157c6f0 in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:08.901835 22683 leader_election.cc:290] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:08.902884 22605 consensus_queue.cc:237] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.903597 22465 raft_consensus.cc:493] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.904130 22465 raft_consensus.cc:515] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:08.905253 22259 raft_consensus.cc:1275] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Refusing update from remote peer 67fc9e7fc4b34621ae1cd3e1621b89dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:08.907020 22658 consensus_queue.cc:1048] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:08:08.908519 22227 leader_election.cc:343] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.909029 22185 raft_consensus.cc:1275] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Refusing update from remote peer 67fc9e7fc4b34621ae1cd3e1621b89dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:08.917523 22457 ts_tablet_manager.cc:1434] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 13:08:08.919160 22457 tablet_bootstrap.cc:492] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:08.873279 22617 raft_consensus.cc:515] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.920770 22179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b51a3b1dc444319751dd92054d5dc3" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:08.921896 22617 leader_election.cc:290] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:08.924924 22179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:08.925077 22655 consensus_queue.cc:1048] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:08.925881 22179 raft_consensus.cc:2393] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bb26db1bd5a64429848b5d63b157c6f0 in current term 1: Already voted for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in this term.
I20260709 13:08:08.911867 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8f7472c97049ca98e597c7d4b4eb1c" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:08.926514 22334 raft_consensus.cc:2468] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:08.928629 22183 raft_consensus.cc:2393] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb26db1bd5a64429848b5d63b157c6f0 in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:08.929833 22082 catalog_manager.cc:5697] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:08.940307 22655 consensus_queue.cc:1048] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:08.941509 22465 leader_election.cc:290] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:08.939926 22456 raft_consensus.cc:359] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.943441 22456 raft_consensus.cc:385] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.947361 22607 raft_consensus.cc:493] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.950531 22607 raft_consensus.cc:515] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:08.954370 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa44244e76604221bcbd08e0f3bbdb82" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:08.955461 22179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9f843a01847a4b475b6dbcb704a6f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:08.956416 22179 raft_consensus.cc:2393] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bb26db1bd5a64429848b5d63b157c6f0 in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
W20260709 13:08:08.956672 22226 leader_election.cc:343] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.954497 22185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:08.958591 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7f82cbc4e8544bbac74ba96797d687c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:08.959511 22260 raft_consensus.cc:2393] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 652d70a0b765447792ab9b4383712607 in current term 1: Already voted for candidate 93815010a884438c86fc6479359b69e8 in this term.
I20260709 13:08:08.961459 22258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9f843a01847a4b475b6dbcb704a6f" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:08.961498 22684 raft_consensus.cc:515] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:08.962006 22607 leader_election.cc:290] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:08.962384 22258 raft_consensus.cc:2393] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 652d70a0b765447792ab9b4383712607 in current term 1: Already voted for candidate bb26db1bd5a64429848b5d63b157c6f0 in this term.
I20260709 13:08:08.949599 22456 raft_consensus.cc:740] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.968683 22456 consensus_queue.cc:260] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:08.977416 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01099e972e9b4d08a64552be3fd52276" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:08.976266 22465 consensus_queue.cc:1048] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:08:08.992832 22456 ts_tablet_manager.cc:1434] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.132s	user 0.012s	sys 0.002s
I20260709 13:08:08.994644 22456 tablet_bootstrap.cc:492] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:08.998291 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8f7472c97049ca98e597c7d4b4eb1c" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:08.999213 22186 raft_consensus.cc:2393] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:09.001132 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f204603e14c438bb549d1116eaac788" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:09.001788 22257 raft_consensus.cc:3060] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.997543 22455 raft_consensus.cc:359] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:09.002662 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8f7472c97049ca98e597c7d4b4eb1c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:09.002965 22455 raft_consensus.cc:385] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.003301 22455 raft_consensus.cc:740] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.991813 22260 raft_consensus.cc:3060] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.004005 22455 consensus_queue.cc:260] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:09.006516 22455 ts_tablet_manager.cc:1434] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.114s	user 0.008s	sys 0.000s
I20260709 13:08:09.007514 22455 tablet_bootstrap.cc:492] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:09.008694 22457 tablet_bootstrap.cc:654] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.003338 22334 raft_consensus.cc:3060] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.990104 22258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:09.016924 22258 raft_consensus.cc:3060] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.018591 22375 leader_election.cc:304] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:09.022955 22691 raft_consensus.cc:2804] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.023453 22691 raft_consensus.cc:493] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.023829 22691 raft_consensus.cc:3060] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.021332 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f204603e14c438bb549d1116eaac788" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:09.024850 22404 raft_consensus.cc:3060] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.025344 22684 leader_election.cc:290] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:09.011788 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8f7472c97049ca98e597c7d4b4eb1c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:09.023737 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:09.038823 22458 tablet_bootstrap.cc:492] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:09.039397 22458 ts_tablet_manager.cc:1403] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.349s	user 0.019s	sys 0.018s
W20260709 13:08:09.053336 22151 leader_election.cc:343] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.054942 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8f7472c97049ca98e597c7d4b4eb1c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:09.043452 22227 leader_election.cc:304] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:09.060845 22227 leader_election.cc:304] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:09.062249 22656 raft_consensus.cc:2749] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.064106 22455 tablet_bootstrap.cc:654] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.094959 22260 raft_consensus.cc:2468] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:09.116957 22607 raft_consensus.cc:2749] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.092741 22334 raft_consensus.cc:2468] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:09.139266 22085 catalog_manager.cc:5697] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.099962 22227 leader_election.cc:304] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:09.155890 22152 leader_election.cc:304] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:09.153723 22404 raft_consensus.cc:2468] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:09.157828 22684 raft_consensus.cc:2804] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.161408 22656 raft_consensus.cc:2749] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.168740 22151 leader_election.cc:304] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:09.169556 22646 raft_consensus.cc:493] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.165094 22684 raft_consensus.cc:697] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:09.177944 22646 raft_consensus.cc:515] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:09.178679 22684 consensus_queue.cc:237] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:09.176333 22458 raft_consensus.cc:359] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:09.182454 22458 raft_consensus.cc:385] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.182781 22458 raft_consensus.cc:740] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.177448 22456 tablet_bootstrap.cc:654] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.181902 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808e97b948be44ee895c060c4329964e" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:09.183671 22458 consensus_queue.cc:260] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:09.180428 22646 leader_election.cc:290] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:09.192035 22458 ts_tablet_manager.cc:1434] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.152s	user 0.009s	sys 0.000s
I20260709 13:08:09.193362 22458 tablet_bootstrap.cc:492] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:09.188689 22616 raft_consensus.cc:2804] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.194676 22616 raft_consensus.cc:697] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:09.195649 22616 consensus_queue.cc:237] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:09.197263 22691 raft_consensus.cc:515] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:09.199222 22455 tablet_bootstrap.cc:492] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:09.199543 22691 leader_election.cc:290] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:09.199774 22455 ts_tablet_manager.cc:1403] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.192s	user 0.023s	sys 0.003s
I20260709 13:08:09.200338 22408 raft_consensus.cc:2393] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 652d70a0b765447792ab9b4383712607 in current term 1: Already voted for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in this term.
W20260709 13:08:09.201900 22226 leader_election.cc:343] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.202487 22257 raft_consensus.cc:2468] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:09.203867 22581 consensus_queue.cc:1048] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:09.201763 22683 raft_consensus.cc:493] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.205406 22683 raft_consensus.cc:515] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:09.207723 22683 leader_election.cc:290] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
W20260709 13:08:09.166072 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 74449.3us
I20260709 13:08:09.210211 22227 leader_election.cc:304] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:09.210626 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e95f83f20904ff692e70a1872c3e77a" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:09.211062 22656 raft_consensus.cc:2749] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:08:09.211477 22227 leader_election.cc:343] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:09.212126 22152 leader_election.cc:343] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.212954 22258 raft_consensus.cc:2468] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:09.215178 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e95f83f20904ff692e70a1872c3e77a" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:09.234467 22455 raft_consensus.cc:359] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:09.236652 22455 raft_consensus.cc:385] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.237047 22455 raft_consensus.cc:740] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.237912 22455 consensus_queue.cc:260] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
W20260709 13:08:09.247104 22151 leader_election.cc:343] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.247743 22151 leader_election.cc:304] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:09.248682 22684 raft_consensus.cc:2749] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.212155 22227 leader_election.cc:304] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:09.256294 22656 raft_consensus.cc:2749] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.249835 22458 tablet_bootstrap.cc:654] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.253356 22457 tablet_bootstrap.cc:492] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:09.258749 22457 ts_tablet_manager.cc:1403] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.340s	user 0.018s	sys 0.008s
I20260709 13:08:09.259364 22455 ts_tablet_manager.cc:1434] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.001s
W20260709 13:08:09.257293 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 45851.3us
W20260709 13:08:09.233225 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 129453us
I20260709 13:08:09.261140 22455 tablet_bootstrap.cc:492] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:09.267881 22227 leader_election.cc:304] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
W20260709 13:08:09.434827 22716 log.cc:927] Time spent T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607: Append to log took a long time: real 0.163s	user 0.009s	sys 0.000s
I20260709 13:08:09.275818 22656 raft_consensus.cc:2749] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:08:09.438109 22714 log.cc:927] Time spent T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0: Append to log took a long time: real 0.124s	user 0.001s	sys 0.008s
W20260709 13:08:09.439404 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 172426us
I20260709 13:08:09.441771 22457 raft_consensus.cc:359] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:09.445339 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8f7472c97049ca98e597c7d4b4eb1c" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:09.455091 22334 raft_consensus.cc:2393] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:09.454227 22457 raft_consensus.cc:385] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.456595 22457 raft_consensus.cc:740] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.457242 22457 consensus_queue.cc:260] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:09.459267 22455 tablet_bootstrap.cc:654] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.460310 22718 consensus_queue.cc:1048] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:09.462255 22336 raft_consensus.cc:3060] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.470319 22578 consensus_queue.cc:1048] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:09.476531 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa44244e76604221bcbd08e0f3bbdb82" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:09.482477 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8f7472c97049ca98e597c7d4b4eb1c" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:09.485208 22456 tablet_bootstrap.cc:492] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:09.485216 22457 ts_tablet_manager.cc:1434] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.226s	user 0.009s	sys 0.030s
I20260709 13:08:09.485893 22456 ts_tablet_manager.cc:1403] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.492s	user 0.019s	sys 0.007s
I20260709 13:08:09.486852 22457 tablet_bootstrap.cc:492] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:09.512914 22458 tablet_bootstrap.cc:492] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:09.513690 22458 ts_tablet_manager.cc:1403] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.321s	user 0.017s	sys 0.007s
I20260709 13:08:09.487039 22151 leader_election.cc:304] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:09.515956 22151 leader_election.cc:304] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:09.516795 22684 raft_consensus.cc:2749] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.517462 22684 raft_consensus.cc:2749] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.517307 22151 leader_election.cc:304] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:09.519551 22151 leader_election.cc:304] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:09.519177 22084 catalog_manager.cc:5697] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:09.520426 22616 raft_consensus.cc:2804] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.520946 22616 raft_consensus.cc:697] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:09.522305 22683 raft_consensus.cc:2804] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.513120 22456 raft_consensus.cc:359] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:09.524255 22456 raft_consensus.cc:385] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.524717 22456 raft_consensus.cc:740] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.525717 22456 consensus_queue.cc:260] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:09.523008 22458 raft_consensus.cc:359] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:09.529431 22458 raft_consensus.cc:385] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.529775 22458 raft_consensus.cc:740] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.521811 22616 consensus_queue.cc:237] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:09.528704 22683 raft_consensus.cc:697] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:09.534415 22683 consensus_queue.cc:237] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:09.539526 22617 raft_consensus.cc:493] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.540161 22617 raft_consensus.cc:515] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:09.543663 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808e97b948be44ee895c060c4329964e" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:09.544299 22617 leader_election.cc:290] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:09.546096 22458 consensus_queue.cc:260] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:09.548821 22458 ts_tablet_manager.cc:1434] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:08:09.549907 22458 tablet_bootstrap.cc:492] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:09.550105 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf2d6d73fe4593b96ead779d12207e" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:09.550697 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf2d6d73fe4593b96ead779d12207e" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:09.502341 22626 raft_consensus.cc:493] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.572046 22456 ts_tablet_manager.cc:1434] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.086s	user 0.008s	sys 0.001s
I20260709 13:08:09.573726 22456 tablet_bootstrap.cc:492] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:09.569805 22085 catalog_manager.cc:5697] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:09.583902 22085 catalog_manager.cc:5697] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:09.570531 22151 leader_election.cc:343] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.600462 22658 raft_consensus.cc:493] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.564594 22626 raft_consensus.cc:515] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:09.606278 22691 consensus_queue.cc:1048] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:09.609724 22455 tablet_bootstrap.cc:492] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:09.611153 22455 ts_tablet_manager.cc:1403] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.350s	user 0.019s	sys 0.014s
W20260709 13:08:09.618178 22152 leader_election.cc:343] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.618801 22152 leader_election.cc:304] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:09.619717 22617 raft_consensus.cc:2749] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.572249 22718 consensus_queue.cc:1048] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.055s
I20260709 13:08:09.601388 22658 raft_consensus.cc:515] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:09.629357 22658 leader_election.cc:290] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:09.633256 22626 leader_election.cc:290] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:09.637157 22617 consensus_queue.cc:1048] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:09.640419 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:09.641297 22257 raft_consensus.cc:2393] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:09.653122 22457 tablet_bootstrap.cc:654] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.662720 22455 raft_consensus.cc:359] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:09.663887 22455 raft_consensus.cc:385] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.664260 22455 raft_consensus.cc:740] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.665377 22455 consensus_queue.cc:260] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:09.667795 22375 leader_election.cc:304] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:09.692135 22455 ts_tablet_manager.cc:1434] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.081s	user 0.010s	sys 0.000s
I20260709 13:08:09.693689 22455 tablet_bootstrap.cc:492] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:09.701067 22458 tablet_bootstrap.cc:654] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.692936 22605 raft_consensus.cc:2749] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:08:09.721541 22227 leader_election.cc:343] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.727660 22701 raft_consensus.cc:493] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.751850 22083 catalog_manager.cc:5697] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:09.762321 22701 raft_consensus.cc:515] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:09.764843 22701 leader_election.cc:290] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:09.756901 22227 leader_election.cc:304] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
W20260709 13:08:09.766371 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 44909.7us
I20260709 13:08:09.766639 22456 tablet_bootstrap.cc:654] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:09.740085 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 84040us
I20260709 13:08:09.773157 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f757dd6e4b94a8a84bfaddd602685e4" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:09.773952 22257 raft_consensus.cc:2468] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:09.775508 22374 leader_election.cc:304] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
W20260709 13:08:09.739619 22301 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 175608us
I20260709 13:08:09.776660 22708 raft_consensus.cc:2804] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.777066 22708 raft_consensus.cc:493] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.777428 22708 raft_consensus.cc:3060] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.784973 22690 consensus_queue.cc:1048] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:09.771509 22607 raft_consensus.cc:2749] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.811519 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:09.811980 22455 tablet_bootstrap.cc:654] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.813094 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:09.821774 22457 tablet_bootstrap.cc:492] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:09.824043 22457 ts_tablet_manager.cc:1403] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.337s	user 0.022s	sys 0.020s
I20260709 13:08:09.832671 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:09.813902 22257 raft_consensus.cc:2468] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
W20260709 13:08:09.797685 22227 leader_election.cc:343] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.849722 22227 leader_election.cc:304] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:09.850665 22683 raft_consensus.cc:493] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.852406 22083 catalog_manager.cc:5697] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.854072 22646 raft_consensus.cc:2749] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.859390 22683 raft_consensus.cc:515] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:09.874444 22375 leader_election.cc:304] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:09.875464 22701 raft_consensus.cc:3060] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.858430 22083 catalog_manager.cc:5697] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.875629 22683 leader_election.cc:290] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:09.881459 22457 raft_consensus.cc:359] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:09.884733 22457 raft_consensus.cc:385] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.885090 22457 raft_consensus.cc:740] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.894706 22457 consensus_queue.cc:260] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:09.906376 22457 ts_tablet_manager.cc:1434] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.082s	user 0.008s	sys 0.001s
I20260709 13:08:09.901062 22626 raft_consensus.cc:493] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.908509 22457 tablet_bootstrap.cc:492] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:09.855026 22684 consensus_queue.cc:1048] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.047s
W20260709 13:08:09.883801 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 88212us
I20260709 13:08:09.909947 22458 tablet_bootstrap.cc:492] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:09.914040 22458 ts_tablet_manager.cc:1403] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.364s	user 0.012s	sys 0.014s
W20260709 13:08:09.916036 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:09.926461 22616 consensus_queue.cc:1048] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:09.869515 22300 leader_election.cc:304] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:09.928673 22581 raft_consensus.cc:2804] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:08:09.928851 22300 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 59458.4us
I20260709 13:08:09.929117 22581 raft_consensus.cc:493] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.929510 22581 raft_consensus.cc:3060] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.941413 22455 tablet_bootstrap.cc:492] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:09.942075 22455 ts_tablet_manager.cc:1403] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.249s	user 0.034s	sys 0.030s
I20260709 13:08:09.945086 22701 raft_consensus.cc:2749] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.945302 22455 raft_consensus.cc:359] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:09.946323 22455 raft_consensus.cc:385] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.946658 22455 raft_consensus.cc:740] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.947700 22455 consensus_queue.cc:260] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:09.948448 22458 raft_consensus.cc:359] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:09.949205 22458 raft_consensus.cc:385] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.949481 22458 raft_consensus.cc:740] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.951021 22455 ts_tablet_manager.cc:1434] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:08:09.914919 22626 raft_consensus.cc:515] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:09.957041 22607 raft_consensus.cc:493] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.957619 22607 raft_consensus.cc:515] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:09.958263 22626 leader_election.cc:290] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:09.960484 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01099e972e9b4d08a64552be3fd52276" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:09.961361 22607 leader_election.cc:290] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:09.952967 22458 consensus_queue.cc:260] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:09.955322 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:09.966013 22458 ts_tablet_manager.cc:1434] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 13:08:09.965925 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffdd1d227bbe48c8a46c4fefd02eab0b" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:09.967114 22458 tablet_bootstrap.cc:492] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:09.960755 22455 tablet_bootstrap.cc:492] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
W20260709 13:08:09.971637 22226 leader_election.cc:343] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.973244 22260 raft_consensus.cc:2468] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:09.974622 22458 tablet_bootstrap.cc:654] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.974623 22455 tablet_bootstrap.cc:654] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.961404 22257 raft_consensus.cc:2393] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93815010a884438c86fc6479359b69e8 in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:09.982550 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01099e972e9b4d08a64552be3fd52276" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:10.021355 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f757dd6e4b94a8a84bfaddd602685e4" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:10.022917 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:10.024317 22617 consensus_queue.cc:1048] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:10.010480 22301 leader_election.cc:304] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:10.019851 22581 raft_consensus.cc:515] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.000746 22708 raft_consensus.cc:515] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.026621 22336 raft_consensus.cc:1240] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Rejecting Update request from peer 67fc9e7fc4b34621ae1cd3e1621b89dc for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:08:10.028780 22581 leader_election.cc:290] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:10.029412 22581 raft_consensus.cc:3060] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:10.041460 22375 leader_election.cc:343] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.045171 22458 tablet_bootstrap.cc:492] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:10.045822 22458 ts_tablet_manager.cc:1403] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.079s	user 0.014s	sys 0.015s
I20260709 13:08:10.045874 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f757dd6e4b94a8a84bfaddd602685e4" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:10.046674 22262 raft_consensus.cc:3060] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.049202 22458 raft_consensus.cc:359] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.051718 22456 tablet_bootstrap.cc:492] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:10.052277 22456 ts_tablet_manager.cc:1403] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.479s	user 0.016s	sys 0.015s
I20260709 13:08:10.049935 22458 raft_consensus.cc:385] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.055259 22458 raft_consensus.cc:740] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.055971 22458 consensus_queue.cc:260] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.057644 22458 ts_tablet_manager.cc:1434] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:08:10.059376 22457 tablet_bootstrap.cc:654] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.061510 22458 tablet_bootstrap.cc:492] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:10.064761 22455 tablet_bootstrap.cc:492] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:10.065287 22455 ts_tablet_manager.cc:1403] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.105s	user 0.027s	sys 0.012s
I20260709 13:08:10.068260 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffdd1d227bbe48c8a46c4fefd02eab0b" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:10.063911 22701 raft_consensus.cc:493] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.070128 22151 leader_election.cc:304] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:10.070308 22701 raft_consensus.cc:515] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.070792 22708 leader_election.cc:290] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:10.071122 22616 raft_consensus.cc:2804] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:10.071527 22616 raft_consensus.cc:493] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.071870 22616 raft_consensus.cc:3060] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.072595 22701 leader_election.cc:290] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
W20260709 13:08:10.075579 22152 leader_election.cc:343] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.077477 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e95f83f20904ff692e70a1872c3e77a" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:10.078259 22186 raft_consensus.cc:2468] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:10.078964 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:10.079792 22375 leader_election.cc:304] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:10.081035 22605 raft_consensus.cc:2804] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:10.081509 22605 raft_consensus.cc:493] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.081934 22605 raft_consensus.cc:3060] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.102267 22456 raft_consensus.cc:359] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.103353 22456 raft_consensus.cc:385] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.103605 22456 raft_consensus.cc:740] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.104521 22456 consensus_queue.cc:260] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.101119 22684 raft_consensus.cc:493] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.106287 22684 raft_consensus.cc:515] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.108444 22684 leader_election.cc:290] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:10.100976 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f757dd6e4b94a8a84bfaddd602685e4" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:10.111680 22262 raft_consensus.cc:2468] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:10.119994 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:10.124541 22708 consensus_queue.cc:1059] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.126863 22456 ts_tablet_manager.cc:1434] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.074s	user 0.008s	sys 0.000s
I20260709 13:08:10.129532 22374 leader_election.cc:304] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:10.131225 22300 leader_election.cc:304] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:10.133586 22456 tablet_bootstrap.cc:492] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:10.125913 22708 raft_consensus.cc:3055] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:10.136921 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e95f83f20904ff692e70a1872c3e77a" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:10.144855 22333 raft_consensus.cc:2468] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:10.139519 22626 raft_consensus.cc:2749] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:10.150550 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96821e05d1924572a73e450bc7dcb265" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:10.142524 22701 raft_consensus.cc:2804] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:08:10.152676 22375 leader_election.cc:343] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.141361 22708 raft_consensus.cc:740] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:10.153501 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96821e05d1924572a73e450bc7dcb265" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:10.165968 22741 consensus_queue.cc:1048] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:08:10.168081 22708 consensus_queue.cc:260] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.169652 22708 raft_consensus.cc:3060] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:10.165016 22152 leader_election.cc:343] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.152678 22701 raft_consensus.cc:697] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:10.171577 22701 consensus_queue.cc:237] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
W20260709 13:08:10.172241 22227 leader_election.cc:343] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.172880 22227 leader_election.cc:304] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:10.178766 22605 raft_consensus.cc:515] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.183804 22457 tablet_bootstrap.cc:492] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:10.184514 22457 ts_tablet_manager.cc:1403] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.276s	user 0.031s	sys 0.035s
I20260709 13:08:10.187655 22457 raft_consensus.cc:359] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.202337 22457 raft_consensus.cc:385] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.214107 22457 raft_consensus.cc:740] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.203394 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e95f83f20904ff692e70a1872c3e77a" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:10.204883 22741 consensus_queue.cc:1048] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.192114 22458 tablet_bootstrap.cc:654] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.201234 22455 raft_consensus.cc:359] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.217008 22456 tablet_bootstrap.cc:654] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.218971 22455 raft_consensus.cc:385] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.219359 22455 raft_consensus.cc:740] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.225797 22457 consensus_queue.cc:260] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.231128 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e95f83f20904ff692e70a1872c3e77a" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:10.233805 22455 consensus_queue.cc:260] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.231925 22331 raft_consensus.cc:3060] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.235415 22605 leader_election.cc:290] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:10.222653 22607 raft_consensus.cc:2749] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:10.255039 22455 ts_tablet_manager.cc:1434] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.189s	user 0.007s	sys 0.003s
I20260709 13:08:10.239857 22404 raft_consensus.cc:2468] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:10.239526 22186 raft_consensus.cc:3060] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.258045 22455 tablet_bootstrap.cc:492] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:10.232920 22616 raft_consensus.cc:515] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.275426 22331 raft_consensus.cc:2468] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:10.281373 22151 leader_election.cc:304] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:10.287699 22457 ts_tablet_manager.cc:1434] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.103s	user 0.016s	sys 0.001s
I20260709 13:08:10.300683 22457 tablet_bootstrap.cc:492] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:10.306067 22375 leader_election.cc:304] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:10.308547 22581 raft_consensus.cc:2749] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:10.307381 22691 raft_consensus.cc:2804] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:10.323891 22455 tablet_bootstrap.cc:654] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.328238 22616 leader_election.cc:290] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:10.316854 22683 raft_consensus.cc:2804] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:10.344455 22684 consensus_queue.cc:1048] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.347043 22684 raft_consensus.cc:493] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.347548 22684 raft_consensus.cc:515] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.332454 22691 raft_consensus.cc:697] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
W20260709 13:08:10.342764 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 111378us
I20260709 13:08:10.335762 22683 raft_consensus.cc:493] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.356473 22646 raft_consensus.cc:493] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.357041 22646 raft_consensus.cc:515] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.358381 22457 tablet_bootstrap.cc:654] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.359171 22646 leader_election.cc:290] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:10.359359 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:10.360100 22257 raft_consensus.cc:3060] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.365818 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:10.372551 22691 consensus_queue.cc:237] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.360488 22683 raft_consensus.cc:3060] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:10.374359 22152 leader_election.cc:343] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.377241 22456 tablet_bootstrap.cc:492] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:10.377822 22456 ts_tablet_manager.cc:1403] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.245s	user 0.030s	sys 0.006s
I20260709 13:08:10.388192 22684 leader_election.cc:290] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:10.389011 22186 raft_consensus.cc:2468] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:10.390506 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:10.402916 22456 raft_consensus.cc:359] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.403993 22456 raft_consensus.cc:385] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.406179 22456 raft_consensus.cc:740] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.407783 22458 tablet_bootstrap.cc:492] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:10.408489 22458 ts_tablet_manager.cc:1403] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.347s	user 0.017s	sys 0.006s
I20260709 13:08:10.407387 22456 consensus_queue.cc:260] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.414947 22456 ts_tablet_manager.cc:1434] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.001s
I20260709 13:08:10.402606 22656 raft_consensus.cc:493] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.418967 22083 catalog_manager.cc:5697] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:10.419121 22227 leader_election.cc:343] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.422389 22456 tablet_bootstrap.cc:492] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
W20260709 13:08:10.400521 22151 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:41261, user_credentials={real_user=slave}} blocked reactor thread for 53743.9us
I20260709 13:08:10.453300 22455 tablet_bootstrap.cc:492] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:10.453972 22455 ts_tablet_manager.cc:1403] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.196s	user 0.019s	sys 0.002s
I20260709 13:08:10.424196 22656 raft_consensus.cc:515] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.462837 22683 raft_consensus.cc:515] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.469580 22616 consensus_queue.cc:1048] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:10.470016 22683 leader_election.cc:290] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:10.471472 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa44244e76604221bcbd08e0f3bbdb82" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:10.472275 22260 raft_consensus.cc:2468] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:10.472404 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:10.466794 22455 raft_consensus.cc:359] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.473273 22455 raft_consensus.cc:385] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.473625 22455 raft_consensus.cc:740] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.474629 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96821e05d1924572a73e450bc7dcb265" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:10.474556 22455 consensus_queue.cc:260] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.475653 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa44244e76604221bcbd08e0f3bbdb82" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:10.478044 22402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96821e05d1924572a73e450bc7dcb265" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:10.478778 22402 raft_consensus.cc:3060] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:10.479478 22152 leader_election.cc:343] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:10.486904 22227 leader_election.cc:343] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.487488 22227 leader_election.cc:304] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:10.488097 22402 raft_consensus.cc:2468] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:10.488339 22750 raft_consensus.cc:2749] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:10.478773 22455 ts_tablet_manager.cc:1434] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:08:10.490233 22455 tablet_bootstrap.cc:492] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:10.499787 22151 leader_election.cc:304] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:10.500834 22683 raft_consensus.cc:2804] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:08:10.501214 22151 leader_election.cc:343] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.501447 22683 raft_consensus.cc:493] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.502031 22683 raft_consensus.cc:3060] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.507138 22458 raft_consensus.cc:359] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.507982 22458 raft_consensus.cc:385] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.508267 22458 raft_consensus.cc:740] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.509059 22458 consensus_queue.cc:260] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.521670 22458 ts_tablet_manager.cc:1434] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.113s	user 0.006s	sys 0.002s
I20260709 13:08:10.524241 22458 tablet_bootstrap.cc:492] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
W20260709 13:08:10.529342 22759 log.cc:927] Time spent T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:08:10.557109 22752 raft_consensus.cc:493] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.557907 22752 raft_consensus.cc:515] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.557674 22151 leader_election.cc:304] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:10.563377 22617 raft_consensus.cc:2804] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:10.563948 22617 raft_consensus.cc:697] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:10.564846 22617 consensus_queue.cc:237] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.561847 22456 tablet_bootstrap.cc:654] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.565941 22455 tablet_bootstrap.cc:654] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.568377 22656 leader_election.cc:290] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:10.566951 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:10.594154 22458 tablet_bootstrap.cc:654] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.595086 22257 raft_consensus.cc:2468] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:10.611464 22457 tablet_bootstrap.cc:492] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:10.612944 22457 ts_tablet_manager.cc:1403] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.312s	user 0.030s	sys 0.003s
I20260709 13:08:10.615558 22741 raft_consensus.cc:493] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.616206 22741 raft_consensus.cc:515] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.616999 22457 raft_consensus.cc:359] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.617866 22457 raft_consensus.cc:385] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.618799 22741 leader_election.cc:290] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:10.620151 22402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffdd1d227bbe48c8a46c4fefd02eab0b" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:10.618268 22457 raft_consensus.cc:740] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
W20260709 13:08:10.626453 22151 leader_election.cc:343] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.620052 22683 raft_consensus.cc:515] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.634271 22726 raft_consensus.cc:493] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.629825 22752 leader_election.cc:290] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:10.634846 22726 raft_consensus.cc:515] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.636247 22683 leader_election.cc:290] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:10.636878 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:10.638927 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e49aee5d99347029a0f5e51fe01b896" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:10.627841 22457 consensus_queue.cc:260] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.640187 22402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e49aee5d99347029a0f5e51fe01b896" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:10.642129 22458 tablet_bootstrap.cc:492] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:10.642766 22458 ts_tablet_manager.cc:1403] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.119s	user 0.010s	sys 0.006s
W20260709 13:08:10.641959 22226 leader_election.cc:343] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.637115 22726 leader_election.cc:290] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:10.631454 22151 leader_election.cc:304] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:10.644788 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0418b53b8c2e4786b445d55f0ea5a723" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:10.645431 22260 raft_consensus.cc:2468] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 0.
I20260709 13:08:10.646651 22741 raft_consensus.cc:2804] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:10.646497 22458 raft_consensus.cc:359] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.647406 22458 raft_consensus.cc:385] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.647333 22741 raft_consensus.cc:697] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:10.647825 22458 raft_consensus.cc:740] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.648348 22300 leader_election.cc:304] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:10.648480 22402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0418b53b8c2e4786b445d55f0ea5a723" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:10.649367 22626 raft_consensus.cc:2804] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:10.649962 22626 raft_consensus.cc:493] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.650342 22626 raft_consensus.cc:3060] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.648911 22458 consensus_queue.cc:260] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.637115 22455 tablet_bootstrap.cc:492] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:10.655175 22455 ts_tablet_manager.cc:1403] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.165s	user 0.020s	sys 0.007s
I20260709 13:08:10.655570 22457 ts_tablet_manager.cc:1434] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 13:08:10.656636 22457 tablet_bootstrap.cc:492] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:10.649334 22402 raft_consensus.cc:2468] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 0.
I20260709 13:08:10.658330 22455 raft_consensus.cc:359] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.659039 22455 raft_consensus.cc:385] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.659288 22455 raft_consensus.cc:740] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.660416 22458 ts_tablet_manager.cc:1434] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 13:08:10.661590 22458 tablet_bootstrap.cc:492] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:10.663678 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa44244e76604221bcbd08e0f3bbdb82" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:10.664237 22457 tablet_bootstrap.cc:654] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.665297 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffdd1d227bbe48c8a46c4fefd02eab0b" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:10.666266 22260 raft_consensus.cc:2468] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:10.666188 22455 consensus_queue.cc:260] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
W20260709 13:08:10.668277 22227 leader_election.cc:343] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:10.665517 22151 leader_election.cc:343] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.677865 22455 ts_tablet_manager.cc:1434] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
W20260709 13:08:10.676770 22227 leader_election.cc:343] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.681782 22227 leader_election.cc:304] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:10.683183 22455 tablet_bootstrap.cc:492] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
W20260709 13:08:10.685397 22227 leader_election.cc:343] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.686044 22456 tablet_bootstrap.cc:492] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:10.686270 22227 leader_election.cc:304] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:10.686836 22456 ts_tablet_manager.cc:1403] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.265s	user 0.014s	sys 0.006s
I20260709 13:08:10.656512 22741 consensus_queue.cc:237] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.695384 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa44244e76604221bcbd08e0f3bbdb82" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:10.696226 22260 raft_consensus.cc:3060] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.700208 22458 tablet_bootstrap.cc:654] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.701143 22691 consensus_queue.cc:1048] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.694291 22083 catalog_manager.cc:5697] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:10.689165 22646 raft_consensus.cc:2749] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:10.688932 22751 raft_consensus.cc:2749] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:10.711228 22151 leader_election.cc:304] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:10.714313 22683 raft_consensus.cc:2804] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:10.714830 22683 raft_consensus.cc:493] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.715246 22683 raft_consensus.cc:3060] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.716899 22260 raft_consensus.cc:2468] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:10.724792 22455 tablet_bootstrap.cc:654] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.725646 22457 tablet_bootstrap.cc:492] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:10.726416 22457 ts_tablet_manager.cc:1403] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.070s	user 0.021s	sys 0.011s
I20260709 13:08:10.730522 22626 raft_consensus.cc:515] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
W20260709 13:08:10.734526 22375 consensus_peers.cc:597] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:10.736300 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0418b53b8c2e4786b445d55f0ea5a723" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:10.737097 22260 raft_consensus.cc:3060] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.738147 22456 raft_consensus.cc:359] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.739090 22456 raft_consensus.cc:385] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.739383 22456 raft_consensus.cc:740] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.740295 22456 consensus_queue.cc:260] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:10.730377 22457 raft_consensus.cc:359] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.743155 22457 raft_consensus.cc:385] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.743476 22457 raft_consensus.cc:740] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.744753 22457 consensus_queue.cc:260] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.748096 22457 ts_tablet_manager.cc:1434] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:08:10.748764 22402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0418b53b8c2e4786b445d55f0ea5a723" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:10.749519 22402 raft_consensus.cc:3060] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.751983 22084 catalog_manager.cc:5697] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:10.754511 22456 ts_tablet_manager.cc:1434] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.064s	user 0.002s	sys 0.005s
I20260709 13:08:10.755771 22456 tablet_bootstrap.cc:492] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:10.758291 22457 tablet_bootstrap.cc:492] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:10.762897 22455 tablet_bootstrap.cc:492] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:10.763548 22455 ts_tablet_manager.cc:1403] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.084s	user 0.007s	sys 0.007s
I20260709 13:08:10.764470 22402 raft_consensus.cc:2468] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:10.765954 22300 leader_election.cc:304] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:10.766921 22726 raft_consensus.cc:2804] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:10.767500 22726 raft_consensus.cc:697] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
I20260709 13:08:10.769169 22458 tablet_bootstrap.cc:492] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:10.768749 22726 consensus_queue.cc:237] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.767500 22626 leader_election.cc:290] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:10.770385 22457 tablet_bootstrap.cc:654] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.770164 22151 leader_election.cc:304] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:10.773383 22260 raft_consensus.cc:2468] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:10.780005 22455 raft_consensus.cc:359] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.803458 22455 raft_consensus.cc:385] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.803862 22455 raft_consensus.cc:740] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.804673 22455 consensus_queue.cc:260] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.780725 22456 tablet_bootstrap.cc:654] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.812830 22455 ts_tablet_manager.cc:1434] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.001s
I20260709 13:08:10.814327 22455 tablet_bootstrap.cc:492] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:10.814483 22758 raft_consensus.cc:2804] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:10.815129 22758 raft_consensus.cc:697] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:10.815990 22758 consensus_queue.cc:237] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.816244 22458 ts_tablet_manager.cc:1403] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.155s	user 0.012s	sys 0.000s
I20260709 13:08:10.825501 22701 consensus_queue.cc:1048] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.831799 22683 raft_consensus.cc:515] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.868686 22402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffdd1d227bbe48c8a46c4fefd02eab0b" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:10.870134 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffdd1d227bbe48c8a46c4fefd02eab0b" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
W20260709 13:08:10.870482 22151 leader_election.cc:343] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.870910 22260 raft_consensus.cc:3060] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.874264 22683 leader_election.cc:290] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:10.885195 22260 raft_consensus.cc:2468] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:10.887477 22151 leader_election.cc:304] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:10.888325 22684 raft_consensus.cc:2804] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:10.888897 22684 raft_consensus.cc:697] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:10.889644 22684 consensus_queue.cc:237] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.895231 22458 raft_consensus.cc:359] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.896821 22458 raft_consensus.cc:385] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.897284 22458 raft_consensus.cc:740] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.896339 22457 tablet_bootstrap.cc:492] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:10.904445 22457 ts_tablet_manager.cc:1403] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.146s	user 0.021s	sys 0.022s
I20260709 13:08:10.899471 22456 tablet_bootstrap.cc:492] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:10.900133 22455 tablet_bootstrap.cc:654] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.907848 22456 ts_tablet_manager.cc:1403] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.152s	user 0.032s	sys 0.018s
I20260709 13:08:10.906740 22458 consensus_queue.cc:260] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.915143 22458 ts_tablet_manager.cc:1434] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.097s	user 0.008s	sys 0.000s
I20260709 13:08:10.915501 22456 raft_consensus.cc:359] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.916425 22456 raft_consensus.cc:385] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.916751 22456 raft_consensus.cc:740] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.916880 22758 raft_consensus.cc:493] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.917600 22758 raft_consensus.cc:515] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.921877 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:10.917982 22456 consensus_queue.cc:260] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.922605 22260 raft_consensus.cc:2468] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:10.923758 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:10.923959 22151 leader_election.cc:304] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:10.924814 22683 raft_consensus.cc:2804] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:10.925302 22683 raft_consensus.cc:493] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.925707 22683 raft_consensus.cc:3060] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:10.925719 22152 leader_election.cc:343] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.944690 22605 raft_consensus.cc:493] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.945336 22605 raft_consensus.cc:515] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:10.946949 22701 consensus_queue.cc:1048] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.948499 22758 leader_election.cc:290] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:10.912382 22457 raft_consensus.cc:359] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.953495 22457 raft_consensus.cc:385] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.953909 22457 raft_consensus.cc:740] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.955402 22458 tablet_bootstrap.cc:492] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:10.955637 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa44244e76604221bcbd08e0f3bbdb82" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:10.956785 22260 raft_consensus.cc:2393] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:10.957437 22457 consensus_queue.cc:260] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.958086 22605 leader_election.cc:290] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:10.966841 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa44244e76604221bcbd08e0f3bbdb82" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:10.968560 22375 leader_election.cc:304] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:10.971035 22457 ts_tablet_manager.cc:1434] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.001s
I20260709 13:08:10.972548 22457 tablet_bootstrap.cc:492] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:10.975332 22456 ts_tablet_manager.cc:1434] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 13:08:10.976259 22456 tablet_bootstrap.cc:492] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:10.978226 22605 raft_consensus.cc:3060] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.984555 22751 raft_consensus.cc:493] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.985169 22751 raft_consensus.cc:515] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:10.981199 22082 catalog_manager.cc:5697] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:10.994333 22646 raft_consensus.cc:493] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.995002 22646 raft_consensus.cc:515] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:10.995275 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:10.998349 22458 tablet_bootstrap.cc:654] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.997689 22402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:11.000624 22226 leader_election.cc:343] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.002902 22646 leader_election.cc:290] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:10.995487 22751 leader_election.cc:290] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:11.010789 22751 raft_consensus.cc:493] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.011420 22751 raft_consensus.cc:515] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:11.013792 22751 leader_election.cc:290] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
W20260709 13:08:11.016558 22227 leader_election.cc:343] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.019902 22227 leader_election.cc:304] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:11.018824 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9f843a01847a4b475b6dbcb704a6f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:11.022914 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3808206385a0458abc04050aeab1cea7" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:11.023350 22333 raft_consensus.cc:2468] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 0.
I20260709 13:08:11.024441 22183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9f843a01847a4b475b6dbcb704a6f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:11.024995 22752 raft_consensus.cc:2749] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:11.025323 22183 raft_consensus.cc:2468] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:11.026424 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3808206385a0458abc04050aeab1cea7" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:11.033526 22227 leader_election.cc:343] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.035779 22227 leader_election.cc:304] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:11.047066 22626 raft_consensus.cc:493] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:11.051317 22227 leader_election.cc:343] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.058362 22227 leader_election.cc:304] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:11.060240 22751 raft_consensus.cc:2749] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:11.058497 22752 raft_consensus.cc:2804] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:11.068389 22752 raft_consensus.cc:493] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.072465 22752 raft_consensus.cc:3060] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.073424 22455 tablet_bootstrap.cc:492] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:11.074090 22455 ts_tablet_manager.cc:1403] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.260s	user 0.046s	sys 0.038s
I20260709 13:08:11.076260 22455 raft_consensus.cc:359] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:11.076970 22455 raft_consensus.cc:385] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:11.077217 22455 raft_consensus.cc:740] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:11.077847 22455 consensus_queue.cc:260] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:11.079507 22455 ts_tablet_manager.cc:1434] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:08:11.080535 22455 tablet_bootstrap.cc:492] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:11.084302 22457 tablet_bootstrap.cc:654] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:11.086673 22684 consensus_queue.cc:1048] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:11.052713 22626 raft_consensus.cc:515] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:11.090188 22455 tablet_bootstrap.cc:654] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:11.093699 22183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf2d6d73fe4593b96ead779d12207e" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:11.094507 22183 raft_consensus.cc:2468] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 0.
I20260709 13:08:11.096249 22301 leader_election.cc:304] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:11.097318 22726 raft_consensus.cc:2804] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:11.097810 22726 raft_consensus.cc:493] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.098304 22726 raft_consensus.cc:3060] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:11.099751 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf2d6d73fe4593b96ead779d12207e" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:11.100603 22407 raft_consensus.cc:2468] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 0.
I20260709 13:08:11.107096 22626 leader_election.cc:290] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:11.021406 22683 raft_consensus.cc:515] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:11.122112 22646 raft_consensus.cc:493] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.122804 22646 raft_consensus.cc:515] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:11.140949 22456 tablet_bootstrap.cc:654] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:11.135154 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7f96ef51941e39bc97c0b769c7770" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:11.173007 22646 leader_election.cc:290] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:11.178126 22683 leader_election.cc:290] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:11.174000 22183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7f96ef51941e39bc97c0b769c7770" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:11.181074 22455 tablet_bootstrap.cc:492] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:11.181298 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:11.181749 22455 ts_tablet_manager.cc:1403] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.101s	user 0.017s	sys 0.005s
I20260709 13:08:11.185034 22458 tablet_bootstrap.cc:492] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:11.185568 22458 ts_tablet_manager.cc:1403] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.231s	user 0.041s	sys 0.021s
I20260709 13:08:11.186942 22605 raft_consensus.cc:2749] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:08:11.187855 22152 leader_election.cc:343] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:11.189029 22152 consensus_peers.cc:597] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:11.189692 22226 leader_election.cc:343] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.196882 22455 raft_consensus.cc:359] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:11.197654 22455 raft_consensus.cc:385] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:11.197901 22455 raft_consensus.cc:740] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
W20260709 13:08:11.200332 22152 consensus_peers.cc:597] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:11.200065 22726 raft_consensus.cc:515] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:11.193504 22458 raft_consensus.cc:359] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:11.203141 22458 raft_consensus.cc:385] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:11.203492 22458 raft_consensus.cc:740] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:11.204480 22458 consensus_queue.cc:260] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:11.205930 22455 consensus_queue.cc:260] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:11.206955 22752 raft_consensus.cc:515] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:11.209460 22455 ts_tablet_manager.cc:1434] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:08:11.209590 22752 leader_election.cc:290] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:11.211066 22455 tablet_bootstrap.cc:492] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
W20260709 13:08:11.238102 22227 leader_election.cc:343] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.243393 22458 ts_tablet_manager.cc:1434] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.005s
I20260709 13:08:11.243536 22227 leader_election.cc:304] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:11.244428 22458 tablet_bootstrap.cc:492] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
W20260709 13:08:11.247730 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:11.245990 22684 raft_consensus.cc:493] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.249820 22183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9f843a01847a4b475b6dbcb704a6f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:11.247047 22646 raft_consensus.cc:2749] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:11.247329 22701 raft_consensus.cc:493] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.250028 22684 raft_consensus.cc:515] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:11.252542 22701 raft_consensus.cc:515] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:11.250466 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf2d6d73fe4593b96ead779d12207e" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:11.253865 22407 raft_consensus.cc:3060] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:11.254690 22684 leader_election.cc:290] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:11.254783 22701 leader_election.cc:290] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:11.255446 22701 raft_consensus.cc:493] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.257426 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e6db323fdb4762a2e92b729b051111" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:11.258139 22181 raft_consensus.cc:2468] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:11.257992 22701 raft_consensus.cc:515] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:11.250530 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:11.259606 22257 raft_consensus.cc:3060] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:11.260811 22701 leader_election.cc:290] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:11.261458 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffdd1d227bbe48c8a46c4fefd02eab0b" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:11.262519 22260 raft_consensus.cc:2393] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:11.264159 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e6db323fdb4762a2e92b729b051111" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:11.265487 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:11.250578 22183 raft_consensus.cc:3060] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.267131 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:11.267841 22260 raft_consensus.cc:2468] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:11.252101 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9f843a01847a4b475b6dbcb704a6f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:11.268988 22626 consensus_queue.cc:1048] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:11.269433 22333 raft_consensus.cc:3060] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.272926 22458 tablet_bootstrap.cc:654] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:11.275302 22726 leader_election.cc:290] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:11.265748 22455 tablet_bootstrap.cc:654] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:11.277300 22375 leader_election.cc:343] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.282889 22375 leader_election.cc:304] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:11.279177 22083 catalog_manager.cc:5697] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:11.278177 22151 leader_election.cc:304] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:11.284682 22083 catalog_manager.cc:5697] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:11.286911 22083 catalog_manager.cc:5697] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:11.286952 22758 raft_consensus.cc:2804] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:11.298552 22758 raft_consensus.cc:493] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.298945 22758 raft_consensus.cc:3060] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:11.268813 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf2d6d73fe4593b96ead779d12207e" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:11.311336 22181 raft_consensus.cc:3060] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:11.313381 22456 tablet_bootstrap.cc:492] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:11.314045 22456 ts_tablet_manager.cc:1403] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.338s	user 0.012s	sys 0.012s
I20260709 13:08:11.316108 22405 raft_consensus.cc:3060] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:11.330260 22407 raft_consensus.cc:2468] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:11.288340 22701 raft_consensus.cc:2804] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:11.335742 22701 raft_consensus.cc:493] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.335773 22300 leader_election.cc:304] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:11.336352 22701 raft_consensus.cc:3060] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:11.336781 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffdd1d227bbe48c8a46c4fefd02eab0b" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:11.337998 22626 raft_consensus.cc:2804] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:11.338616 22626 raft_consensus.cc:697] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
I20260709 13:08:11.328682 22741 consensus_queue.cc:1048] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:08:11.339701 22626 consensus_queue.cc:237] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:11.342870 22375 leader_election.cc:304] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:11.333966 22617 consensus_queue.cc:1048] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:11.335310 22457 tablet_bootstrap.cc:492] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:11.353690 22457 ts_tablet_manager.cc:1403] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.381s	user 0.038s	sys 0.020s
W20260709 13:08:11.350205 22152 leader_election.cc:343] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.358431 22456 raft_consensus.cc:359] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:11.368530 22456 raft_consensus.cc:385] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:11.368865 22456 raft_consensus.cc:740] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:11.369686 22456 consensus_queue.cc:260] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:11.380259 22456 ts_tablet_manager.cc:1434] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 13:08:11.357601 22758 raft_consensus.cc:515] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:11.372776 22741 consensus_queue.cc:1048] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:08:11.386099 22758 leader_election.cc:290] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:11.360350 22333 raft_consensus.cc:2468] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 2.
I20260709 13:08:11.395570 22655 raft_consensus.cc:2749] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:11.406629 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:11.407852 22260 raft_consensus.cc:3060] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:11.410650 22336 raft_consensus.cc:3060] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:11.418732 22257 raft_consensus.cc:2468] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:11.420776 22151 leader_election.cc:304] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:11.422010 22758 raft_consensus.cc:2804] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:11.422566 22758 raft_consensus.cc:697] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:11.423509 22758 consensus_queue.cc:237] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:11.426353 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:11.435405 22227 leader_election.cc:304] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:11.436324 22456 tablet_bootstrap.cc:492] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:11.436889 22751 raft_consensus.cc:2804] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:11.437474 22751 raft_consensus.cc:697] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:11.438462 22751 consensus_queue.cc:237] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:11.439546 22152 leader_election.cc:343] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.429143 22457 raft_consensus.cc:359] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:11.444669 22457 raft_consensus.cc:385] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:11.445139 22457 raft_consensus.cc:740] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:11.446300 22457 consensus_queue.cc:260] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:11.478652 22457 ts_tablet_manager.cc:1434] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.125s	user 0.008s	sys 0.000s
I20260709 13:08:11.482657 22458 tablet_bootstrap.cc:492] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:11.483497 22458 ts_tablet_manager.cc:1403] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.239s	user 0.037s	sys 0.019s
W20260709 13:08:11.455760 22300 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 90348.8us
I20260709 13:08:11.479954 22752 raft_consensus.cc:493] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.513307 22183 raft_consensus.cc:2468] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 2.
I20260709 13:08:11.516546 22456 tablet_bootstrap.cc:654] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:11.516794 22458 raft_consensus.cc:359] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:11.521620 22458 raft_consensus.cc:385] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:11.522465 22458 raft_consensus.cc:740] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:11.522912 22457 tablet_bootstrap.cc:492] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:11.523200 22458 consensus_queue.cc:260] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:11.541344 22790 raft_consensus.cc:493] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.542013 22790 raft_consensus.cc:515] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:11.542871 22458 ts_tablet_manager.cc:1434] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.001s
I20260709 13:08:11.544037 22458 tablet_bootstrap.cc:492] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:11.544269 22790 leader_election.cc:290] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:11.524639 22752 raft_consensus.cc:515] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:11.528262 22791 consensus_queue.cc:1048] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
W20260709 13:08:11.562755 22374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 100555us
I20260709 13:08:11.604681 22455 tablet_bootstrap.cc:492] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:11.605610 22455 ts_tablet_manager.cc:1403] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.395s	user 0.008s	sys 0.010s
I20260709 13:08:11.575430 22181 raft_consensus.cc:2468] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:11.629206 22083 catalog_manager.cc:5697] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:11.623382 22655 raft_consensus.cc:493] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.638165 22457 tablet_bootstrap.cc:654] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:11.598047 22260 raft_consensus.cc:2468] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
W20260709 13:08:11.630929 22798 log.cc:927] Time spent T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:08:11.643543 22655 raft_consensus.cc:515] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:11.636261 22801 raft_consensus.cc:493] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 67fc9e7fc4b34621ae1cd3e1621b89dc)
I20260709 13:08:11.629206 22701 raft_consensus.cc:515] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:11.647174 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7f96ef51941e39bc97c0b769c7770" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:11.661149 22456 tablet_bootstrap.cc:492] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:11.660039 22801 raft_consensus.cc:515] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:11.674141 22458 tablet_bootstrap.cc:654] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:11.674686 22456 ts_tablet_manager.cc:1403] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.239s	user 0.022s	sys 0.012s
I20260709 13:08:11.669965 22455 raft_consensus.cc:359] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:11.677220 22455 raft_consensus.cc:385] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:11.677547 22455 raft_consensus.cc:740] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:11.678308 22455 consensus_queue.cc:260] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
W20260709 13:08:11.666770 22795 log.cc:927] Time spent T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:08:11.685887 22801 leader_election.cc:290] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:11.686324 22793 consensus_queue.cc:1048] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:11.670063 22684 consensus_queue.cc:1048] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 13:08:11.682000 22262 raft_consensus.cc:2468] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:11.687707 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:11.683441 22151 leader_election.cc:304] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:11.666952 22655 leader_election.cc:290] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:11.668243 22752 leader_election.cc:290] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:11.704833 22183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9b737a6d0d24895b749f55ac078ac5f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:11.715548 22151 leader_election.cc:304] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:11.717144 22683 raft_consensus.cc:2804] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:11.717648 22683 raft_consensus.cc:493] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.718065 22683 raft_consensus.cc:3060] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:11.711309 22331 raft_consensus.cc:2393] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb26db1bd5a64429848b5d63b157c6f0 in current term 2: Already voted for candidate 93815010a884438c86fc6479359b69e8 in this term.
W20260709 13:08:11.718617 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 43627us
I20260709 13:08:11.723658 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7f96ef51941e39bc97c0b769c7770" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:11.710559 22617 raft_consensus.cc:2804] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:11.725682 22617 raft_consensus.cc:697] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:11.718055 22456 raft_consensus.cc:359] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:11.726656 22617 consensus_queue.cc:237] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:11.727993 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:11.728725 22332 raft_consensus.cc:2468] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 2.
I20260709 13:08:11.727691 22456 raft_consensus.cc:385] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:11.737704 22701 leader_election.cc:290] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:11.732391 22456 raft_consensus.cc:740] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:11.734195 22375 leader_election.cc:304] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:11.757962 22455 ts_tablet_manager.cc:1434] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.152s	user 0.017s	sys 0.007s
I20260709 13:08:11.759595 22455 tablet_bootstrap.cc:492] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:11.765611 22457 tablet_bootstrap.cc:492] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:11.766283 22457 ts_tablet_manager.cc:1403] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.262s	user 0.023s	sys 0.003s
I20260709 13:08:11.746831 22456 consensus_queue.cc:260] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:11.773955 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9b737a6d0d24895b749f55ac078ac5f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:11.774791 22458 tablet_bootstrap.cc:492] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:11.775288 22458 ts_tablet_manager.cc:1403] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.231s	user 0.015s	sys 0.004s
I20260709 13:08:11.776407 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:11.768761 22701 raft_consensus.cc:2804] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:11.777525 22701 raft_consensus.cc:493] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.777884 22701 raft_consensus.cc:3060] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:11.777766 22458 raft_consensus.cc:359] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:11.778592 22458 raft_consensus.cc:385] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:11.778815 22458 raft_consensus.cc:740] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:11.783443 22456 ts_tablet_manager.cc:1434] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.108s	user 0.008s	sys 0.002s
I20260709 13:08:11.780073 22683 raft_consensus.cc:515] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:11.845674 22084 catalog_manager.cc:5697] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:11.857847 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e6db323fdb4762a2e92b729b051111" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:11.859711 22186 raft_consensus.cc:3060] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:11.877032 22456 tablet_bootstrap.cc:492] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
W20260709 13:08:11.906873 22226 leader_election.cc:343] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.907419 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e6db323fdb4762a2e92b729b051111" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:11.921365 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:11.922708 22261 raft_consensus.cc:2468] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:11.908167 22607 raft_consensus.cc:493] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.930648 22607 raft_consensus.cc:515] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:11.935247 22607 leader_election.cc:290] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:11.944471 22683 leader_election.cc:290] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:11.784508 22458 consensus_queue.cc:260] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:11.963452 22408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808e97b948be44ee895c060c4329964e" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:11.971011 22301 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 72796.9us
W20260709 13:08:11.992354 22226 leader_election.cc:343] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.994871 22456 tablet_bootstrap.cc:654] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:11.979851 22457 raft_consensus.cc:359] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:11.997479 22457 raft_consensus.cc:385] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:12.001978 22457 raft_consensus.cc:740] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:12.003891 22457 consensus_queue.cc:260] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:11.986295 22151 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:41261, user_credentials={real_user=slave}} blocked reactor thread for 61040.2us
I20260709 13:08:11.959388 22802 raft_consensus.cc:493] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:11.985668 22375 leader_election.cc:343] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.006392 22726 consensus_queue.cc:1048] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:12.010325 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808e97b948be44ee895c060c4329964e" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:12.013793 22458 ts_tablet_manager.cc:1434] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.194s	user 0.008s	sys 0.001s
I20260709 13:08:12.015296 22455 tablet_bootstrap.cc:654] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:12.018548 22151 leader_election.cc:343] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.033263 22458 tablet_bootstrap.cc:492] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:12.019987 22802 raft_consensus.cc:515] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.036638 22802 leader_election.cc:290] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:12.040944 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e49aee5d99347029a0f5e51fe01b896" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:12.042845 22457 ts_tablet_manager.cc:1434] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.276s	user 0.007s	sys 0.003s
I20260709 13:08:12.045221 22726 consensus_queue.cc:1048] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:08:12.053474 22227 leader_election.cc:343] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.054068 22227 leader_election.cc:304] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:12.054716 22258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e49aee5d99347029a0f5e51fe01b896" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:12.055658 22258 raft_consensus.cc:2468] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
W20260709 13:08:12.055889 22375 leader_election.cc:343] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.056104 22752 raft_consensus.cc:2749] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:12.051148 22085 catalog_manager.cc:5697] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 0 to 2, leader changed from <none> to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:12.058571 22227 leader_election.cc:343] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.059132 22227 leader_election.cc:304] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:12.059245 22333 raft_consensus.cc:1275] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Refusing update from remote peer bb26db1bd5a64429848b5d63b157c6f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:12.067298 22186 raft_consensus.cc:2468] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:12.068686 22752 raft_consensus.cc:2749] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:12.057055 22457 tablet_bootstrap.cc:492] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:12.091341 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7f96ef51941e39bc97c0b769c7770" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:12.092317 22455 tablet_bootstrap.cc:492] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:12.093129 22455 ts_tablet_manager.cc:1403] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.334s	user 0.018s	sys 0.003s
I20260709 13:08:12.093298 22456 tablet_bootstrap.cc:492] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:12.092501 22259 raft_consensus.cc:3060] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.095757 22375 leader_election.cc:304] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:12.096868 22802 raft_consensus.cc:2804] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:12.097455 22802 raft_consensus.cc:697] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:12.102814 22456 ts_tablet_manager.cc:1403] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.319s	user 0.018s	sys 0.015s
I20260709 13:08:12.103534 22802 consensus_queue.cc:237] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:12.117964 22803 consensus_queue.cc:1048] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:08:12.122793 22458 tablet_bootstrap.cc:654] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:12.127089 22803 raft_consensus.cc:493] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.141362 22803 raft_consensus.cc:515] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:12.149683 22726 raft_consensus.cc:493] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.150678 22726 raft_consensus.cc:515] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:12.134737 22374 leader_election.cc:304] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:12.160422 22815 raft_consensus.cc:2804] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:12.160975 22815 raft_consensus.cc:493] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.161418 22815 raft_consensus.cc:3060] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:12.162752 22374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 105410us
I20260709 13:08:12.164090 22803 leader_election.cc:290] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:12.129781 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7f96ef51941e39bc97c0b769c7770" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
W20260709 13:08:12.170984 22152 consensus_peers.cc:597] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:12.170689 22456 raft_consensus.cc:359] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:12.171568 22456 raft_consensus.cc:385] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:12.171921 22456 raft_consensus.cc:740] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:12.175765 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d38314054d8474eb00ca91785ea945d" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:12.179107 22184 raft_consensus.cc:1275] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Refusing update from remote peer bb26db1bd5a64429848b5d63b157c6f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:12.184144 22332 raft_consensus.cc:3060] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.180907 22456 consensus_queue.cc:260] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:12.217693 22457 tablet_bootstrap.cc:654] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:12.226847 22456 ts_tablet_manager.cc:1434] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.124s	user 0.008s	sys 0.002s
I20260709 13:08:12.231720 22457 tablet_bootstrap.cc:492] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:12.232424 22457 ts_tablet_manager.cc:1403] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.176s	user 0.016s	sys 0.002s
W20260709 13:08:12.236644 22227 leader_election.cc:343] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.231613 22455 raft_consensus.cc:359] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.243971 22455 raft_consensus.cc:385] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:12.244249 22455 raft_consensus.cc:740] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:12.250301 22455 consensus_queue.cc:260] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.252192 22455 ts_tablet_manager.cc:1434] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.159s	user 0.009s	sys 0.000s
I20260709 13:08:12.253597 22455 tablet_bootstrap.cc:492] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:12.255213 22701 raft_consensus.cc:515] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.255736 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:12.256839 22403 raft_consensus.cc:2368] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb26db1bd5a64429848b5d63b157c6f0 for earlier term 2. Current term is 3.
I20260709 13:08:12.258134 22701 leader_election.cc:290] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:12.260089 22226 leader_election.cc:400] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261) with higher term. Message: Invalid argument: T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb26db1bd5a64429848b5d63b157c6f0 for earlier term 2. Current term is 3.
I20260709 13:08:12.260501 22226 leader_election.cc:403] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:12.262468 22607 raft_consensus.cc:3060] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:12.266487 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:12.267369 22333 raft_consensus.cc:3060] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:12.243094 22726 leader_election.cc:290] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:12.286458 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:12.288838 22300 leader_election.cc:304] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:12.293268 22752 consensus_queue.cc:1048] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:12.301225 22333 raft_consensus.cc:2468] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 3.
I20260709 13:08:12.316159 22456 tablet_bootstrap.cc:492] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:12.317463 22375 leader_election.cc:304] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:12.319350 22655 raft_consensus.cc:2804] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:12.320307 22655 raft_consensus.cc:697] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:12.322791 22655 consensus_queue.cc:237] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.323992 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d38314054d8474eb00ca91785ea945d" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:12.324659 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
W20260709 13:08:12.327749 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:08:12.307093 22151 leader_election.cc:343] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.332587 22815 raft_consensus.cc:515] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.335812 22456 tablet_bootstrap.cc:654] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:12.336086 22082 catalog_manager.cc:5697] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:12.346400 22684 raft_consensus.cc:493] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.347122 22684 raft_consensus.cc:515] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:12.348093 22458 tablet_bootstrap.cc:492] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:12.348716 22458 ts_tablet_manager.cc:1403] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.316s	user 0.017s	sys 0.021s
I20260709 13:08:12.349704 22684 leader_election.cc:290] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:12.351624 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3808206385a0458abc04050aeab1cea7" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:12.353420 22152 leader_election.cc:343] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.353677 22457 raft_consensus.cc:359] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:12.346235 22259 raft_consensus.cc:2468] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:12.361366 22458 raft_consensus.cc:359] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:12.362260 22458 raft_consensus.cc:385] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:12.362555 22458 raft_consensus.cc:740] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:12.363386 22458 consensus_queue.cc:260] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:12.367551 22458 ts_tablet_manager.cc:1434] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:08:12.375186 22455 tablet_bootstrap.cc:654] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:12.376040 22793 raft_consensus.cc:2749] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:12.377156 22815 leader_election.cc:290] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:12.379359 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e49aee5d99347029a0f5e51fe01b896" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
W20260709 13:08:12.387884 22375 leader_election.cc:343] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.354492 22457 raft_consensus.cc:385] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:12.381086 22084 catalog_manager.cc:5697] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:12.388921 22456 tablet_bootstrap.cc:492] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:12.389218 22457 raft_consensus.cc:740] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:12.389714 22456 ts_tablet_manager.cc:1403] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.074s	user 0.033s	sys 0.010s
I20260709 13:08:12.390367 22458 tablet_bootstrap.cc:492] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:12.382588 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e49aee5d99347029a0f5e51fe01b896" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
W20260709 13:08:12.385210 22227 leader_election.cc:343] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.393674 22227 leader_election.cc:304] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:12.393922 22456 raft_consensus.cc:359] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:12.394892 22456 raft_consensus.cc:385] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:12.395231 22456 raft_consensus.cc:740] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:12.395455 22752 raft_consensus.cc:2749] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:12.396173 22456 consensus_queue.cc:260] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:12.398627 22456 ts_tablet_manager.cc:1434] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:08:12.399968 22456 tablet_bootstrap.cc:492] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:12.403898 22457 consensus_queue.cc:260] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:12.418409 22457 ts_tablet_manager.cc:1434] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.186s	user 0.009s	sys 0.000s
I20260709 13:08:12.419839 22457 tablet_bootstrap.cc:492] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:12.414914 22259 raft_consensus.cc:3060] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.421574 22456 tablet_bootstrap.cc:654] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:12.427139 22458 tablet_bootstrap.cc:654] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:12.428139 22617 consensus_queue.cc:1048] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:08:12.429766 22809 consensus_queue.cc:1048] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:12.435385 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3808206385a0458abc04050aeab1cea7" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:12.436254 22260 raft_consensus.cc:2468] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:12.445113 22456 tablet_bootstrap.cc:492] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:12.445749 22456 ts_tablet_manager.cc:1403] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.046s	user 0.006s	sys 0.006s
I20260709 13:08:12.442029 22815 raft_consensus.cc:493] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.453852 22457 tablet_bootstrap.cc:654] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:12.441407 22151 leader_election.cc:304] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:12.460114 22809 raft_consensus.cc:2804] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:12.461100 22684 raft_consensus.cc:493] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.449136 22815 raft_consensus.cc:515] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.461613 22684 raft_consensus.cc:515] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:12.461248 22082 catalog_manager.cc:5697] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.465441 22684 leader_election.cc:290] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:12.462764 22809 raft_consensus.cc:697] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:12.467599 22809 consensus_queue.cc:237] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:12.468071 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e65d47217943239d60b88b4b70ba68" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:12.469529 22152 leader_election.cc:343] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.473242 22151 leader_election.cc:304] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:12.474568 22617 raft_consensus.cc:2804] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:12.475082 22617 raft_consensus.cc:493] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.475512 22617 raft_consensus.cc:3060] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.489928 22456 raft_consensus.cc:359] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:12.493400 22456 raft_consensus.cc:385] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:12.494165 22456 raft_consensus.cc:740] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:12.494876 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e65d47217943239d60b88b4b70ba68" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:12.495328 22456 consensus_queue.cc:260] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:12.499688 22455 tablet_bootstrap.cc:492] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:12.500202 22455 ts_tablet_manager.cc:1403] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.247s	user 0.010s	sys 0.004s
W20260709 13:08:12.504668 22151 leader_election.cc:343] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.505074 22151 leader_election.cc:304] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
W20260709 13:08:12.511940 22152 consensus_peers.cc:597] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:12.512493 22683 raft_consensus.cc:2749] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:12.516660 22332 raft_consensus.cc:3060] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.512447 22458 tablet_bootstrap.cc:492] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:12.517611 22458 ts_tablet_manager.cc:1403] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.149s	user 0.017s	sys 0.012s
I20260709 13:08:12.517681 22258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808e97b948be44ee895c060c4329964e" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:12.518523 22258 raft_consensus.cc:2468] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:12.520386 22374 leader_election.cc:304] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:12.520903 22655 consensus_queue.cc:1048] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:12.521536 22811 raft_consensus.cc:2804] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:12.522014 22811 raft_consensus.cc:493] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.522387 22811 raft_consensus.cc:3060] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.528343 22455 raft_consensus.cc:359] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:12.529181 22455 raft_consensus.cc:385] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:12.529453 22455 raft_consensus.cc:740] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:12.533679 22815 leader_election.cc:290] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:12.542347 22456 ts_tablet_manager.cc:1434] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.096s	user 0.011s	sys 0.000s
I20260709 13:08:12.503822 22607 raft_consensus.cc:2749] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261) with higher term. Message: Invalid argument: T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb26db1bd5a64429848b5d63b157c6f0 for earlier term 2. Current term is 3.
I20260709 13:08:12.564986 22617 raft_consensus.cc:515] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:12.574826 22456 tablet_bootstrap.cc:492] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:12.535220 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808e97b948be44ee895c060c4329964e" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:12.636222 22617 leader_election.cc:290] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:12.628954 22455 consensus_queue.cc:260] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:12.689052 22458 raft_consensus.cc:359] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:12.690941 22457 tablet_bootstrap.cc:492] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:12.691351 22811 raft_consensus.cc:515] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.722096 22333 raft_consensus.cc:3060] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.702386 22458 raft_consensus.cc:385] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:12.702764 22457 ts_tablet_manager.cc:1403] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.283s	user 0.013s	sys 0.002s
I20260709 13:08:12.702977 22726 raft_consensus.cc:493] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.703742 22258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3808206385a0458abc04050aeab1cea7" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:12.706250 22455 ts_tablet_manager.cc:1434] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.206s	user 0.009s	sys 0.000s
I20260709 13:08:12.734102 22726 raft_consensus.cc:515] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:12.734648 22258 raft_consensus.cc:3060] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.735447 22455 tablet_bootstrap.cc:492] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:12.736457 22726 leader_election.cc:290] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:12.733345 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808e97b948be44ee895c060c4329964e" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:12.737715 22260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:12.738441 22262 raft_consensus.cc:3060] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.738732 22458 raft_consensus.cc:740] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:12.743088 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3808206385a0458abc04050aeab1cea7" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:12.744746 22330 raft_consensus.cc:3060] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.741351 22458 consensus_queue.cc:260] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:12.752723 22082 catalog_manager.cc:5697] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:12.758445 22375 leader_election.cc:343] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.762566 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
W20260709 13:08:12.770092 22152 leader_election.cc:343] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.774863 22301 leader_election.cc:304] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:12.773597 22815 consensus_queue.cc:1048] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 13:08:12.799686 22458 ts_tablet_manager.cc:1434] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.282s	user 0.009s	sys 0.019s
I20260709 13:08:12.795147 22259 raft_consensus.cc:2468] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:12.809587 22456 tablet_bootstrap.cc:654] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:12.795467 22811 leader_election.cc:290] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:12.814869 22457 raft_consensus.cc:359] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.820307 22457 raft_consensus.cc:385] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:12.820663 22457 raft_consensus.cc:740] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:12.831020 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808e97b948be44ee895c060c4329964e" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:12.846645 22626 raft_consensus.cc:493] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:12.847282 22626 raft_consensus.cc:515] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:12.849596 22331 raft_consensus.cc:1275] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Refusing update from remote peer 67fc9e7fc4b34621ae1cd3e1621b89dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:12.851974 22802 raft_consensus.cc:493] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.852586 22802 raft_consensus.cc:515] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:12.849581 22258 raft_consensus.cc:2468] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
W20260709 13:08:12.854403 22375 leader_election.cc:343] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.856114 22802 leader_election.cc:290] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:12.860581 22455 tablet_bootstrap.cc:654] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:12.861886 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7f96ef51941e39bc97c0b769c7770" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:12.870904 22458 tablet_bootstrap.cc:492] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:12.874342 22812 consensus_queue.cc:1048] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:08:12.895823 22752 raft_consensus.cc:493] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.916195 22752 raft_consensus.cc:515] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.916908 22258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7f96ef51941e39bc97c0b769c7770" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:12.917817 22258 raft_consensus.cc:2393] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:12.919544 22752 leader_election.cc:290] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:12.920737 22410 raft_consensus.cc:3060] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.920773 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:12.922407 22226 leader_election.cc:343] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.925376 22262 raft_consensus.cc:2468] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:12.892985 22374 leader_election.cc:304] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:12.914932 22801 raft_consensus.cc:493] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.929713 22801 raft_consensus.cc:515] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:12.933074 22257 raft_consensus.cc:2468] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 3.
I20260709 13:08:12.935493 22801 leader_election.cc:290] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:12.936959 22456 tablet_bootstrap.cc:492] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:12.936870 22853 raft_consensus.cc:493] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.937619 22853 raft_consensus.cc:515] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:12.933387 22455 tablet_bootstrap.cc:492] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:12.938917 22455 ts_tablet_manager.cc:1403] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.204s	user 0.021s	sys 0.000s
I20260709 13:08:12.939054 22456 ts_tablet_manager.cc:1403] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.365s	user 0.025s	sys 0.005s
I20260709 13:08:12.940685 22728 raft_consensus.cc:2749] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:12.941325 22683 consensus_queue.cc:1048] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:08:12.942528 22374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 60720.7us
I20260709 13:08:12.946970 22458 tablet_bootstrap.cc:654] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:12.948487 22853 leader_election.cc:290] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:12.953372 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9b737a6d0d24895b749f55ac078ac5f" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:12.953614 22374 leader_election.cc:304] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:12.955229 22374 leader_election.cc:304] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:12.956243 22811 raft_consensus.cc:2804] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:12.956773 22811 raft_consensus.cc:697] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:12.957803 22811 consensus_queue.cc:237] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.958940 22701 raft_consensus.cc:2804] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:12.959488 22701 raft_consensus.cc:697] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:12.960400 22701 consensus_queue.cc:237] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
W20260709 13:08:12.962803 22151 leader_election.cc:343] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:12.968937 22456 raft_consensus.cc:359] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.969913 22456 raft_consensus.cc:385] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:12.970317 22456 raft_consensus.cc:740] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:12.973325 22456 consensus_queue.cc:260] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.978267 22185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:12.986907 22151 leader_election.cc:304] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:12.993885 22457 consensus_queue.cc:260] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.999114 22455 raft_consensus.cc:359] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:13.000151 22455 raft_consensus.cc:385] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.000488 22455 raft_consensus.cc:740] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.001237 22457 ts_tablet_manager.cc:1434] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.264s	user 0.010s	sys 0.005s
I20260709 13:08:13.001602 22455 consensus_queue.cc:260] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.994997 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96821e05d1924572a73e450bc7dcb265" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:12.997222 22726 raft_consensus.cc:493] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.004192 22726 raft_consensus.cc:515] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:12.994313 22456 ts_tablet_manager.cc:1434] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.000s
I20260709 13:08:13.006654 22456 tablet_bootstrap.cc:492] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:13.007777 22455 ts_tablet_manager.cc:1434] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.001s
I20260709 13:08:13.008977 22455 tablet_bootstrap.cc:492] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:12.994175 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:13.011706 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96821e05d1924572a73e450bc7dcb265" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:12.974179 22626 leader_election.cc:290] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
W20260709 13:08:13.018255 22227 leader_election.cc:343] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.026257 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:13.019923 22683 raft_consensus.cc:2804] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:13.055433 22726 leader_election.cc:290] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:13.058941 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9b737a6d0d24895b749f55ac078ac5f" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:13.061867 22300 leader_election.cc:304] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:13.065045 22728 raft_consensus.cc:2749] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:13.066527 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e49aee5d99347029a0f5e51fe01b896" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:13.067634 22256 raft_consensus.cc:2393] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93815010a884438c86fc6479359b69e8 in current term 1: Already voted for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in this term.
I20260709 13:08:13.041671 22815 raft_consensus.cc:2749] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:13.090791 22456 tablet_bootstrap.cc:654] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.071832 22261 raft_consensus.cc:2468] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:13.075074 22683 raft_consensus.cc:697] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
W20260709 13:08:13.097563 22152 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 67794.3us
I20260709 13:08:13.108032 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e49aee5d99347029a0f5e51fe01b896" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:13.109412 22226 leader_election.cc:343] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.109983 22226 leader_election.cc:304] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:13.110978 22751 raft_consensus.cc:2749] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:13.112739 22802 raft_consensus.cc:493] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.113268 22802 raft_consensus.cc:515] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:13.115880 22457 tablet_bootstrap.cc:492] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:13.117897 22256 raft_consensus.cc:1275] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Refusing update from remote peer 67fc9e7fc4b34621ae1cd3e1621b89dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:13.120762 22802 leader_election.cc:290] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:13.123658 22458 tablet_bootstrap.cc:492] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:13.126847 22457 tablet_bootstrap.cc:654] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:13.129444 22227 leader_election.cc:343] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.130484 22458 ts_tablet_manager.cc:1403] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.265s	user 0.021s	sys 0.011s
I20260709 13:08:13.130618 22227 leader_election.cc:304] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
W20260709 13:08:13.150379 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 115182us
I20260709 13:08:13.143914 22801 raft_consensus.cc:2749] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:13.169800 22683 consensus_queue.cc:237] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:13.142500 22617 consensus_queue.cc:1048] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:08:13.197978 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:13.200203 22811 consensus_queue.cc:1048] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:08:13.244243 22861 raft_consensus.cc:493] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.244957 22861 raft_consensus.cc:515] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
W20260709 13:08:13.246549 22375 leader_election.cc:343] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.247382 22861 leader_election.cc:290] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:13.247565 22300 leader_election.cc:304] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:13.250902 22626 raft_consensus.cc:3060] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:13.259513 22458 raft_consensus.cc:359] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:13.263233 22458 raft_consensus.cc:385] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.263618 22458 raft_consensus.cc:740] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.281642 22455 tablet_bootstrap.cc:654] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.288370 22458 consensus_queue.cc:260] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:13.298173 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96356b5b69fc405cb1678dbfaec2df1f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:13.298532 22258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:13.299188 22258 raft_consensus.cc:2468] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
W20260709 13:08:13.300802 22226 leader_election.cc:343] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.301344 22458 ts_tablet_manager.cc:1434] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.171s	user 0.017s	sys 0.018s
I20260709 13:08:13.302973 22458 tablet_bootstrap.cc:492] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:13.308121 22151 leader_election.cc:304] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:13.309147 22683 raft_consensus.cc:2804] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:13.309557 22683 raft_consensus.cc:493] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.309900 22683 raft_consensus.cc:3060] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:13.319255 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96356b5b69fc405cb1678dbfaec2df1f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:13.324630 22227 leader_election.cc:343] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.329635 22227 leader_election.cc:304] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:13.341756 22626 raft_consensus.cc:2749] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:13.361477 22876 raft_consensus.cc:2749] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:13.366705 22870 raft_consensus.cc:493] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.367295 22870 raft_consensus.cc:515] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:13.369634 22870 leader_election.cc:290] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:13.391626 22456 tablet_bootstrap.cc:492] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:13.392146 22456 ts_tablet_manager.cc:1403] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.386s	user 0.017s	sys 0.002s
I20260709 13:08:13.390977 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0de2f3edc854918bb42aa68cdc2c9fa" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:13.399410 22457 tablet_bootstrap.cc:492] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:13.400002 22457 ts_tablet_manager.cc:1403] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.284s	user 0.033s	sys 0.018s
W20260709 13:08:13.400619 22152 leader_election.cc:343] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.405004 22457 raft_consensus.cc:359] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:13.405829 22457 raft_consensus.cc:385] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.406167 22457 raft_consensus.cc:740] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.406911 22457 consensus_queue.cc:260] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:13.408890 22457 ts_tablet_manager.cc:1434] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:08:13.410163 22457 tablet_bootstrap.cc:492] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
W20260709 13:08:13.410226 22300 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 85675.4us
I20260709 13:08:13.411907 22456 raft_consensus.cc:359] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:13.412575 22456 raft_consensus.cc:385] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.412843 22456 raft_consensus.cc:740] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.413570 22456 consensus_queue.cc:260] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:13.421411 22085 catalog_manager.cc:5697] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:13.438792 22085 catalog_manager.cc:5697] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:13.446448 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:13.458257 22752 raft_consensus.cc:493] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.458945 22752 raft_consensus.cc:515] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:13.459550 22374 leader_election.cc:304] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:13.460631 22455 tablet_bootstrap.cc:492] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:13.461179 22455 ts_tablet_manager.cc:1403] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.452s	user 0.016s	sys 0.018s
I20260709 13:08:13.461800 22752 leader_election.cc:290] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:13.463438 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdbf4608aa9f48d68cb77a4907dff6a6" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:13.465282 22226 leader_election.cc:343] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.467491 22458 tablet_bootstrap.cc:654] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.467967 22457 tablet_bootstrap.cc:654] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.443933 22683 raft_consensus.cc:515] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:13.474251 22812 raft_consensus.cc:2804] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:13.474659 22812 raft_consensus.cc:493] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.474979 22812 raft_consensus.cc:3060] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:13.479890 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdbf4608aa9f48d68cb77a4907dff6a6" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:13.493498 22456 ts_tablet_manager.cc:1434] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.101s	user 0.008s	sys 0.000s
W20260709 13:08:13.495857 22227 leader_election.cc:343] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.496412 22227 leader_election.cc:304] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:13.498685 22455 raft_consensus.cc:359] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:13.499481 22455 raft_consensus.cc:385] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.499764 22455 raft_consensus.cc:740] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.500447 22455 consensus_queue.cc:260] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:13.501726 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0de2f3edc854918bb42aa68cdc2c9fa" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:13.503259 22607 raft_consensus.cc:2749] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:13.516017 22455 ts_tablet_manager.cc:1434] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.000s
I20260709 13:08:13.523837 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9b737a6d0d24895b749f55ac078ac5f" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
W20260709 13:08:13.538178 22374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 63074.3us
I20260709 13:08:13.541280 22410 raft_consensus.cc:3060] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:13.528740 22456 tablet_bootstrap.cc:492] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
W20260709 13:08:13.494591 22300 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 82555.5us
I20260709 13:08:13.529450 22683 leader_election.cc:290] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
W20260709 13:08:13.538422 22151 leader_election.cc:343] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.546373 22151 leader_election.cc:304] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:13.546569 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9b737a6d0d24895b749f55ac078ac5f" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:13.543579 22812 raft_consensus.cc:515] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:13.547739 22741 raft_consensus.cc:2749] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:13.548002 22257 raft_consensus.cc:3060] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:13.549752 22812 leader_election.cc:290] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:13.551012 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:13.552377 22261 raft_consensus.cc:3060] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:13.554979 22455 tablet_bootstrap.cc:492] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:13.555428 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
W20260709 13:08:13.556890 22375 leader_election.cc:343] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.553879 22410 raft_consensus.cc:2468] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:13.566453 22151 leader_election.cc:304] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:13.567288 22741 raft_consensus.cc:2804] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:13.567806 22741 raft_consensus.cc:697] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:13.569154 22456 tablet_bootstrap.cc:654] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.568683 22741 consensus_queue.cc:237] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:13.580037 22458 tablet_bootstrap.cc:492] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:13.580940 22458 ts_tablet_manager.cc:1403] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.278s	user 0.014s	sys 0.003s
I20260709 13:08:13.584461 22261 raft_consensus.cc:2468] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:13.586163 22374 leader_election.cc:304] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:13.587056 22701 raft_consensus.cc:2804] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:13.587602 22701 raft_consensus.cc:697] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:13.591245 22701 consensus_queue.cc:237] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:13.594707 22457 tablet_bootstrap.cc:492] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:13.595476 22457 ts_tablet_manager.cc:1403] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.186s	user 0.010s	sys 0.004s
I20260709 13:08:13.596701 22728 raft_consensus.cc:493] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.598701 22728 raft_consensus.cc:515] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:13.600756 22456 tablet_bootstrap.cc:492] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:13.601306 22456 ts_tablet_manager.cc:1403] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.073s	user 0.004s	sys 0.020s
I20260709 13:08:13.603296 22728 leader_election.cc:290] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:13.603533 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:13.604465 22456 raft_consensus.cc:359] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:13.605291 22456 raft_consensus.cc:385] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.605592 22456 raft_consensus.cc:740] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.606292 22456 consensus_queue.cc:260] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:13.607522 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:13.607911 22458 raft_consensus.cc:359] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:13.608635 22456 ts_tablet_manager.cc:1434] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:08:13.608727 22458 raft_consensus.cc:385] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.609176 22458 raft_consensus.cc:740] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.596076 22257 raft_consensus.cc:2468] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:13.609853 22300 leader_election.cc:304] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:13.611055 22728 raft_consensus.cc:2749] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:13.612445 22457 raft_consensus.cc:359] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:13.633006 22457 raft_consensus.cc:385] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.633394 22457 raft_consensus.cc:740] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.618409 22458 consensus_queue.cc:260] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:13.609845 22456 tablet_bootstrap.cc:492] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:13.636214 22458 ts_tablet_manager.cc:1434] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.055s	user 0.005s	sys 0.001s
I20260709 13:08:13.637763 22458 tablet_bootstrap.cc:492] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:13.625536 22455 tablet_bootstrap.cc:654] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.625903 22334 raft_consensus.cc:3060] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:13.642438 22456 tablet_bootstrap.cc:654] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.649219 22457 consensus_queue.cc:260] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:13.668442 22457 ts_tablet_manager.cc:1434] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.073s	user 0.014s	sys 0.008s
I20260709 13:08:13.670167 22458 tablet_bootstrap.cc:654] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.671669 22457 tablet_bootstrap.cc:492] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:13.662202 22701 consensus_queue.cc:1048] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:08:13.679265 22456 tablet_bootstrap.cc:492] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:13.679893 22456 ts_tablet_manager.cc:1403] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.070s	user 0.013s	sys 0.015s
I20260709 13:08:13.686172 22455 tablet_bootstrap.cc:492] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:13.686764 22455 ts_tablet_manager.cc:1403] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.132s	user 0.014s	sys 0.007s
I20260709 13:08:13.676375 22886 consensus_queue.cc:1048] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:08:13.693318 22811 consensus_queue.cc:1048] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:08:13.696801 22701 consensus_queue.cc:1048] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:13.717514 22455 raft_consensus.cc:359] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
W20260709 13:08:13.721469 22152 consensus_peers.cc:597] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:13.722307 22455 raft_consensus.cc:385] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.722693 22455 raft_consensus.cc:740] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.723284 22455 consensus_queue.cc:260] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:13.728430 22457 tablet_bootstrap.cc:654] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.757614 22336 raft_consensus.cc:3060] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:13.770243 22083 catalog_manager.cc:5697] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:13.771639 22683 consensus_queue.cc:1048] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:13.781482 22455 ts_tablet_manager.cc:1434] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.094s	user 0.007s	sys 0.001s
I20260709 13:08:13.802742 22741 raft_consensus.cc:493] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.803218 22458 tablet_bootstrap.cc:492] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:13.803807 22458 ts_tablet_manager.cc:1403] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.166s	user 0.008s	sys 0.011s
I20260709 13:08:13.803411 22741 raft_consensus.cc:515] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:13.811293 22741 leader_election.cc:290] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:13.815023 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:13.815793 22256 raft_consensus.cc:2468] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:13.816435 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:13.818226 22151 leader_election.cc:343] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.819047 22151 leader_election.cc:304] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:13.823182 22683 raft_consensus.cc:2804] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:13.823555 22683 raft_consensus.cc:493] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.823907 22683 raft_consensus.cc:3060] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:13.831146 22458 raft_consensus.cc:359] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:13.831367 22456 raft_consensus.cc:359] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:13.832020 22458 raft_consensus.cc:385] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.832152 22456 raft_consensus.cc:385] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.832368 22458 raft_consensus.cc:740] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.832578 22456 raft_consensus.cc:740] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.833343 22458 consensus_queue.cc:260] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:13.833549 22456 consensus_queue.cc:260] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:13.843719 22455 tablet_bootstrap.cc:492] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:13.850277 22084 catalog_manager.cc:5697] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:13.858465 22456 ts_tablet_manager.cc:1434] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.178s	user 0.007s	sys 0.002s
I20260709 13:08:13.869079 22458 ts_tablet_manager.cc:1434] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.002s
I20260709 13:08:13.881620 22456 tablet_bootstrap.cc:492] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:13.884254 22458 tablet_bootstrap.cc:492] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:13.905857 22456 tablet_bootstrap.cc:654] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.916494 22683 raft_consensus.cc:515] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:13.918941 22683 leader_election.cc:290] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:13.929121 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:13.930039 22457 tablet_bootstrap.cc:492] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:13.931144 22457 ts_tablet_manager.cc:1403] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.262s	user 0.022s	sys 0.021s
I20260709 13:08:13.923894 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
W20260709 13:08:13.932063 22151 leader_election.cc:343] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.944893 22752 raft_consensus.cc:493] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.945339 22886 raft_consensus.cc:493] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.945644 22752 raft_consensus.cc:515] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:13.945983 22886 raft_consensus.cc:515] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:13.948477 22752 leader_election.cc:290] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:13.934930 22257 raft_consensus.cc:3060] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:13.950486 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d38314054d8474eb00ca91785ea945d" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:13.950913 22886 leader_election.cc:290] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:13.951169 22185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e65d47217943239d60b88b4b70ba68" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:13.951906 22185 raft_consensus.cc:2468] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:13.952991 22185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d38314054d8474eb00ca91785ea945d" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:13.953141 22375 leader_election.cc:304] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:13.953646 22185 raft_consensus.cc:2468] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 0.
W20260709 13:08:13.954931 22227 leader_election.cc:343] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.959734 22227 leader_election.cc:304] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:13.960928 22336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e65d47217943239d60b88b4b70ba68" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:13.961313 22457 raft_consensus.cc:359] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:13.962103 22457 raft_consensus.cc:385] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.962466 22457 raft_consensus.cc:740] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.964583 22607 raft_consensus.cc:2804] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:13.970580 22457 consensus_queue.cc:260] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:13.969787 22607 raft_consensus.cc:493] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.973940 22607 raft_consensus.cc:3060] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:13.976583 22617 consensus_queue.cc:1048] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:13.989609 22455 tablet_bootstrap.cc:654] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.958993 22886 raft_consensus.cc:2804] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:13.991757 22457 ts_tablet_manager.cc:1434] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.001s
W20260709 13:08:13.995174 22375 leader_election.cc:343] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.010952 22457 tablet_bootstrap.cc:492] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:13.995980 22886 raft_consensus.cc:493] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.014504 22886 raft_consensus.cc:3060] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.039132 22458 tablet_bootstrap.cc:654] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.072001 22457 tablet_bootstrap.cc:654] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.077057 22085 catalog_manager.cc:5697] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } }
I20260709 13:08:14.089607 22741 consensus_queue.cc:1048] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:14.100538 22456 tablet_bootstrap.cc:492] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:14.101249 22456 ts_tablet_manager.cc:1403] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.223s	user 0.016s	sys 0.053s
I20260709 13:08:14.119987 22752 raft_consensus.cc:493] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.120850 22752 raft_consensus.cc:515] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
W20260709 13:08:14.122836 22375 consensus_peers.cc:597] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:14.123922 22333 raft_consensus.cc:3060] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.124817 22257 raft_consensus.cc:2468] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:14.136324 22861 raft_consensus.cc:493] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.141232 22861 raft_consensus.cc:515] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:14.143638 22861 leader_election.cc:290] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:14.149029 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4bb1a452944c59aa3dc6717c52be6a" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:14.150532 22227 leader_election.cc:343] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.137717 22752 leader_election.cc:290] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:14.140192 22151 leader_election.cc:304] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:14.160454 22809 raft_consensus.cc:2804] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.161232 22809 raft_consensus.cc:697] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:14.162489 22809 consensus_queue.cc:237] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:14.165699 22185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:14.168080 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4bb1a452944c59aa3dc6717c52be6a" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:14.168586 22886 raft_consensus.cc:515] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
W20260709 13:08:14.174273 22227 leader_election.cc:343] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:14.176770 22227 leader_election.cc:343] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.177153 22227 leader_election.cc:304] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:14.153378 22456 raft_consensus.cc:359] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:14.179165 22456 raft_consensus.cc:385] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.179495 22456 raft_consensus.cc:740] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.182463 22456 consensus_queue.cc:260] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:14.181406 22861 raft_consensus.cc:2749] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.195701 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:14.197448 22886 leader_election.cc:290] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:14.199959 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e65d47217943239d60b88b4b70ba68" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:14.203730 22456 ts_tablet_manager.cc:1434] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.102s	user 0.009s	sys 0.001s
I20260709 13:08:14.209794 22186 raft_consensus.cc:3060] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.222151 22607 raft_consensus.cc:515] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:14.226358 22226 leader_election.cc:343] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.234998 22226 leader_election.cc:304] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:14.235809 22185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d38314054d8474eb00ca91785ea945d" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:14.236528 22185 raft_consensus.cc:3060] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.236770 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d38314054d8474eb00ca91785ea945d" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:14.237305 22457 tablet_bootstrap.cc:492] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:14.237844 22457 ts_tablet_manager.cc:1403] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.227s	user 0.022s	sys 0.028s
I20260709 13:08:14.241930 22456 tablet_bootstrap.cc:492] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
W20260709 13:08:14.243086 22227 leader_election.cc:343] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.233508 22607 leader_election.cc:290] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:14.249292 22752 raft_consensus.cc:2749] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.242800 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e65d47217943239d60b88b4b70ba68" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:14.276707 22186 raft_consensus.cc:2468] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:14.280927 22456 tablet_bootstrap.cc:654] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.281680 22455 tablet_bootstrap.cc:492] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:14.289458 22455 ts_tablet_manager.cc:1403] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.469s	user 0.010s	sys 0.012s
I20260709 13:08:14.289562 22375 leader_election.cc:304] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
W20260709 13:08:14.303453 22375 leader_election.cc:343] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:14.272696 22226 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:41261, user_credentials={real_user=slave}} blocked reactor thread for 51147.4us
I20260709 13:08:14.307806 22908 raft_consensus.cc:2804] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.320561 22457 raft_consensus.cc:359] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:14.324242 22457 raft_consensus.cc:385] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.324694 22457 raft_consensus.cc:740] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.325428 22457 consensus_queue.cc:260] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:14.327368 22455 raft_consensus.cc:359] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.327826 22457 ts_tablet_manager.cc:1434] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.090s	user 0.005s	sys 0.002s
I20260709 13:08:14.328179 22455 raft_consensus.cc:385] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.328504 22455 raft_consensus.cc:740] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.329182 22455 consensus_queue.cc:260] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.331561 22455 ts_tablet_manager.cc:1434] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.001s
I20260709 13:08:14.333389 22455 tablet_bootstrap.cc:492] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:14.343135 22908 raft_consensus.cc:697] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:14.344296 22908 consensus_queue.cc:237] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.348079 22185 raft_consensus.cc:2468] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:14.351953 22227 leader_election.cc:304] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:14.359503 22457 tablet_bootstrap.cc:492] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:14.360517 22752 raft_consensus.cc:2804] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.361192 22752 raft_consensus.cc:697] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:14.365535 22752 consensus_queue.cc:237] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:14.379308 22456 tablet_bootstrap.cc:492] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:14.379793 22458 tablet_bootstrap.cc:492] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:14.379956 22456 ts_tablet_manager.cc:1403] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.138s	user 0.016s	sys 0.014s
I20260709 13:08:14.380339 22458 ts_tablet_manager.cc:1403] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.501s	user 0.014s	sys 0.004s
I20260709 13:08:14.383229 22456 raft_consensus.cc:359] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.384063 22456 raft_consensus.cc:385] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.384383 22456 raft_consensus.cc:740] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.385093 22456 consensus_queue.cc:260] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.372754 22083 catalog_manager.cc:5697] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:14.395426 22457 tablet_bootstrap.cc:654] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.419631 22456 ts_tablet_manager.cc:1434] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 13:08:14.421340 22458 raft_consensus.cc:359] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.429785 22458 raft_consensus.cc:385] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.430243 22458 raft_consensus.cc:740] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.432696 22456 tablet_bootstrap.cc:492] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:14.436806 22455 tablet_bootstrap.cc:654] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.414247 22815 consensus_queue.cc:1048] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260709 13:08:14.477538 22728 raft_consensus.cc:493] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.478183 22728 raft_consensus.cc:515] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:14.480724 22728 leader_election.cc:290] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:14.480279 22458 consensus_queue.cc:260] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.482604 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:14.484207 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:14.489723 22458 ts_tablet_manager.cc:1434] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.109s	user 0.011s	sys 0.001s
I20260709 13:08:14.490964 22458 tablet_bootstrap.cc:492] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:14.491847 22300 leader_election.cc:304] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:14.495754 22085 catalog_manager.cc:5697] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:14.497100 22884 raft_consensus.cc:2749] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.505270 22458 tablet_bootstrap.cc:654] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.506667 22332 raft_consensus.cc:3060] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.520428 22455 tablet_bootstrap.cc:492] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:14.521070 22455 ts_tablet_manager.cc:1403] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.188s	user 0.012s	sys 0.003s
I20260709 13:08:14.524047 22455 raft_consensus.cc:359] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:14.524806 22455 raft_consensus.cc:385] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.525098 22455 raft_consensus.cc:740] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.528155 22082 catalog_manager.cc:5697] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:14.529495 22455 consensus_queue.cc:260] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:14.531271 22455 ts_tablet_manager.cc:1434] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:08:14.532373 22455 tablet_bootstrap.cc:492] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:14.538350 22457 tablet_bootstrap.cc:492] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:14.538774 22458 tablet_bootstrap.cc:492] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:14.539309 22458 ts_tablet_manager.cc:1403] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.049s	user 0.008s	sys 0.006s
I20260709 13:08:14.538952 22457 ts_tablet_manager.cc:1403] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.180s	user 0.017s	sys 0.020s
I20260709 13:08:14.544272 22457 raft_consensus.cc:359] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:14.545020 22457 raft_consensus.cc:385] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.545286 22457 raft_consensus.cc:740] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.546969 22458 raft_consensus.cc:359] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:14.547878 22458 raft_consensus.cc:385] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.548178 22458 raft_consensus.cc:740] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.549080 22458 consensus_queue.cc:260] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:14.550321 22457 consensus_queue.cc:260] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:14.550937 22455 tablet_bootstrap.cc:654] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.552299 22458 ts_tablet_manager.cc:1434] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:08:14.552995 22457 ts_tablet_manager.cc:1434] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:08:14.553651 22458 tablet_bootstrap.cc:492] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:14.554236 22457 tablet_bootstrap.cc:492] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0: Bootstrap starting.
I20260709 13:08:14.556620 22455 tablet_bootstrap.cc:492] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:14.557090 22455 ts_tablet_manager.cc:1403] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.000s
I20260709 13:08:14.561252 22455 raft_consensus.cc:359] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:14.562186 22455 raft_consensus.cc:385] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.562481 22455 raft_consensus.cc:740] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.563331 22455 consensus_queue.cc:260] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:14.568359 22458 tablet_bootstrap.cc:654] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.582134 22457 tablet_bootstrap.cc:654] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.582137 22456 tablet_bootstrap.cc:654] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.595528 22458 tablet_bootstrap.cc:492] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:14.596076 22458 ts_tablet_manager.cc:1403] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.002s
I20260709 13:08:14.611049 22455 ts_tablet_manager.cc:1434] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.054s	user 0.004s	sys 0.004s
I20260709 13:08:14.623893 22458 raft_consensus.cc:359] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:14.624804 22458 raft_consensus.cc:385] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.625070 22458 raft_consensus.cc:740] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.625782 22458 consensus_queue.cc:260] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:14.641201 22455 tablet_bootstrap.cc:492] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:14.643507 22458 ts_tablet_manager.cc:1434] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.002s
I20260709 13:08:14.645292 22458 tablet_bootstrap.cc:492] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
W20260709 13:08:14.653861 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:14.668529 22607 raft_consensus.cc:493] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.671465 22607 raft_consensus.cc:515] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:14.671901 22456 tablet_bootstrap.cc:492] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:14.674216 22456 ts_tablet_manager.cc:1403] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.242s	user 0.007s	sys 0.019s
I20260709 13:08:14.679059 22607 leader_election.cc:290] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:14.679968 22457 tablet_bootstrap.cc:492] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0: No bootstrap required, opened a new log
I20260709 13:08:14.680117 22458 tablet_bootstrap.cc:654] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.680549 22457 ts_tablet_manager.cc:1403] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0: Time spent bootstrapping tablet: real 0.127s	user 0.013s	sys 0.019s
I20260709 13:08:14.681769 22185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
W20260709 13:08:14.683531 22227 leader_election.cc:343] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.683633 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:14.685437 22226 leader_election.cc:343] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.685526 22923 raft_consensus.cc:493] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.685956 22226 leader_election.cc:304] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:14.686085 22923 raft_consensus.cc:515] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.689842 22752 raft_consensus.cc:2749] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.690122 22457 raft_consensus.cc:359] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:14.703065 22457 raft_consensus.cc:385] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.703408 22457 raft_consensus.cc:740] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.691493 22923 leader_election.cc:290] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:14.705538 22456 raft_consensus.cc:359] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.706379 22456 raft_consensus.cc:385] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.706710 22456 raft_consensus.cc:740] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.707800 22456 consensus_queue.cc:260] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.696908 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb4bb4b406fa4f4f9a55ba3b7c005cf0" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:14.690786 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb4bb4b406fa4f4f9a55ba3b7c005cf0" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:14.712491 22752 raft_consensus.cc:493] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:14.712603 22151 leader_election.cc:343] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.713187 22752 raft_consensus.cc:515] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:14.715471 22752 leader_election.cc:290] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:14.716272 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d9ed71058164d51a8aa5b9980536bd8" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:14.717806 22227 leader_election.cc:343] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.719151 22456 ts_tablet_manager.cc:1434] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 13:08:14.719322 22886 raft_consensus.cc:493] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.704756 22457 consensus_queue.cc:260] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:14.720870 22886 raft_consensus.cc:515] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.721882 22884 raft_consensus.cc:493] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:14.719815 22152 leader_election.cc:343] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.722460 22884 raft_consensus.cc:515] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.722886 22152 leader_election.cc:304] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:14.724325 22741 raft_consensus.cc:2749] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.725601 22886 leader_election.cc:290] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:14.726258 22185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e65d47217943239d60b88b4b70ba68" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:14.720192 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d9ed71058164d51a8aa5b9980536bd8" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:14.727694 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0de2f3edc854918bb42aa68cdc2c9fa" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:14.731369 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e65d47217943239d60b88b4b70ba68" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:14.733347 22457 ts_tablet_manager.cc:1434] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.001s
I20260709 13:08:14.735366 22181 raft_consensus.cc:2468] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:14.729530 22884 leader_election.cc:290] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:14.740341 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0de2f3edc854918bb42aa68cdc2c9fa" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:14.742731 22334 raft_consensus.cc:2468] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
W20260709 13:08:14.745357 22226 leader_election.cc:343] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.745896 22226 leader_election.cc:304] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:14.747457 22913 raft_consensus.cc:2749] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.737360 22456 tablet_bootstrap.cc:492] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:14.736999 22375 leader_election.cc:304] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:14.738269 22185 raft_consensus.cc:2393] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93815010a884438c86fc6479359b69e8 in current term 1: Already voted for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in this term.
I20260709 13:08:14.750352 22301 leader_election.cc:304] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:14.750845 22410 raft_consensus.cc:3060] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.752171 22728 raft_consensus.cc:3060] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.759449 22908 raft_consensus.cc:2804] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:14.759996 22908 raft_consensus.cc:493] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.760435 22908 raft_consensus.cc:3060] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.773015 22728 raft_consensus.cc:2749] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.783227 22886 raft_consensus.cc:493] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.790545 22886 raft_consensus.cc:515] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:14.791770 22617 consensus_queue.cc:1048] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:14.795346 22458 tablet_bootstrap.cc:492] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:14.795710 22886 leader_election.cc:290] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:14.795920 22458 ts_tablet_manager.cc:1403] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.151s	user 0.017s	sys 0.009s
I20260709 13:08:14.796309 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:14.783722 22455 tablet_bootstrap.cc:654] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.796988 22261 raft_consensus.cc:2468] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:14.797329 22185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:14.799007 22374 leader_election.cc:304] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:14.800830 22458 raft_consensus.cc:359] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.801553 22458 raft_consensus.cc:385] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.801833 22458 raft_consensus.cc:740] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.804036 22458 consensus_queue.cc:260] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.804430 22811 raft_consensus.cc:2804] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:14.806423 22811 raft_consensus.cc:493] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:14.806914 22811 raft_consensus.cc:3060] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:14.811993 22456 tablet_bootstrap.cc:654] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.806504 22458 ts_tablet_manager.cc:1434] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:08:14.804318 22921 consensus_queue.cc:1048] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:14.816708 22455 tablet_bootstrap.cc:492] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:14.817337 22455 ts_tablet_manager.cc:1403] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.183s	user 0.008s	sys 0.007s
I20260709 13:08:14.818518 22458 tablet_bootstrap.cc:492] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:14.823596 22815 consensus_queue.cc:1048] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:14.825081 22455 raft_consensus.cc:359] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.825712 22815 raft_consensus.cc:493] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.825873 22455 raft_consensus.cc:385] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.826188 22752 raft_consensus.cc:493] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.826457 22455 raft_consensus.cc:740] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.826351 22815 raft_consensus.cc:515] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:14.827682 22455 consensus_queue.cc:260] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.826841 22752 raft_consensus.cc:515] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:14.829191 22815 leader_election.cc:290] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:14.831207 22908 raft_consensus.cc:515] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.831575 22752 leader_election.cc:290] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:14.831774 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdbf4608aa9f48d68cb77a4907dff6a6" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:14.832468 22261 raft_consensus.cc:2468] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:14.833866 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dfb3dc107a469c880393583ee0c218" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:14.839087 22374 leader_election.cc:304] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:14.840296 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdbf4608aa9f48d68cb77a4907dff6a6" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:14.836710 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dfb3dc107a469c880393583ee0c218" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:14.842986 22227 leader_election.cc:343] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:14.843794 22227 consensus_peers.cc:597] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:14.848371 22886 raft_consensus.cc:2804] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:14.848914 22886 raft_consensus.cc:493] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.849267 22886 raft_consensus.cc:3060] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.852591 22458 tablet_bootstrap.cc:654] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.853193 22456 tablet_bootstrap.cc:492] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:14.853813 22456 ts_tablet_manager.cc:1403] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.120s	user 0.011s	sys 0.000s
W20260709 13:08:14.855043 22375 leader_election.cc:343] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:14.856652 22227 leader_election.cc:343] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.857158 22227 leader_election.cc:304] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:14.857298 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0de2f3edc854918bb42aa68cdc2c9fa" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:14.867494 22181 raft_consensus.cc:3060] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.858321 22923 raft_consensus.cc:493] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.860847 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0de2f3edc854918bb42aa68cdc2c9fa" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:14.854898 22908 leader_election.cc:290] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:14.872643 22333 raft_consensus.cc:3060] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.869649 22752 raft_consensus.cc:2749] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.879791 22455 ts_tablet_manager.cc:1434] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.062s	user 0.005s	sys 0.002s
I20260709 13:08:14.882957 22741 raft_consensus.cc:493] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.883523 22741 raft_consensus.cc:515] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:14.878191 22923 raft_consensus.cc:515] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.887301 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42127cae8f494cb081ec04febe577130" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:14.890337 22152 leader_election.cc:343] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.885943 22741 leader_election.cc:290] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:14.861740 22811 raft_consensus.cc:515] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:14.898728 22683 consensus_queue.cc:1048] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:08:14.924978 22455 tablet_bootstrap.cc:492] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:14.911666 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:14.924810 22921 raft_consensus.cc:493] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.925778 22405 raft_consensus.cc:1240] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Rejecting Update request from peer 652d70a0b765447792ab9b4383712607 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:14.936276 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:14.937075 22921 raft_consensus.cc:515] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:14.945168 22186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb4bb4b406fa4f4f9a55ba3b7c005cf0" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:14.946147 22186 raft_consensus.cc:2468] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:14.948575 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb4bb4b406fa4f4f9a55ba3b7c005cf0" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:14.937969 22261 raft_consensus.cc:2468] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:14.951251 22811 leader_election.cc:290] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:14.955878 22181 raft_consensus.cc:2468] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:14.961596 22455 tablet_bootstrap.cc:654] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:14.971580 22151 leader_election.cc:343] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.963347 22886 raft_consensus.cc:515] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:14.972308 22923 leader_election.cc:290] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:14.943405 22921 leader_election.cc:290] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:14.961668 22333 raft_consensus.cc:2468] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:14.958061 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:14.977037 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42127cae8f494cb081ec04febe577130" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:14.979207 22151 leader_election.cc:304] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:14.981127 22942 raft_consensus.cc:2804] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:14.981665 22942 raft_consensus.cc:493] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.982095 22809 raft_consensus.cc:493] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.982244 22886 leader_election.cc:290] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:14.979298 22456 raft_consensus.cc:359] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:14.982956 22456 raft_consensus.cc:385] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.982685 22809 raft_consensus.cc:515] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:14.983240 22456 raft_consensus.cc:740] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.984083 22456 consensus_queue.cc:260] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:14.985133 22809 leader_election.cc:290] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:14.947517 22375 leader_election.cc:304] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
W20260709 13:08:14.985517 22151 leader_election.cc:343] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.982096 22942 raft_consensus.cc:3060] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.986033 22151 leader_election.cc:304] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:14.986259 22607 consensus_queue.cc:1048] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:14.987108 22809 raft_consensus.cc:2749] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.988327 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdbf4608aa9f48d68cb77a4907dff6a6" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:14.989001 22257 raft_consensus.cc:3060] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.989503 22456 ts_tablet_manager.cc:1434] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.134s	user 0.007s	sys 0.000s
I20260709 13:08:14.993557 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4bb1a452944c59aa3dc6717c52be6a" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:14.994443 22261 raft_consensus.cc:2468] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:14.994870 22617 consensus_queue.cc:1059] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:14.999483 22809 raft_consensus.cc:3055] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:14.999920 22809 raft_consensus.cc:740] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:15.001106 22456 tablet_bootstrap.cc:492] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:15.000973 22809 consensus_queue.cc:260] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:15.002375 22809 raft_consensus.cc:3060] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:15.004854 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 57430.8us
I20260709 13:08:15.005194 22921 raft_consensus.cc:2804] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:15.005663 22921 raft_consensus.cc:493] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.005995 22921 raft_consensus.cc:3060] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.008986 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdbf4608aa9f48d68cb77a4907dff6a6" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:15.009222 22375 leader_election.cc:304] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
W20260709 13:08:15.010131 22375 leader_election.cc:343] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:15.010917 22815 raft_consensus.cc:2804] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:15.010975 22257 raft_consensus.cc:2468] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:15.011631 22815 raft_consensus.cc:697] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:15.012595 22815 consensus_queue.cc:237] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:15.013031 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4bb1a452944c59aa3dc6717c52be6a" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:15.014117 22374 leader_election.cc:304] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:15.015242 22812 raft_consensus.cc:2804] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:15.015740 22812 raft_consensus.cc:697] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:15.016638 22812 consensus_queue.cc:237] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:15.019492 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:15.009720 22333 raft_consensus.cc:3060] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.028712 22921 raft_consensus.cc:515] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:15.036850 22607 raft_consensus.cc:493] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.037882 22607 raft_consensus.cc:515] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:15.036453 22151 leader_election.cc:304] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:15.042341 22607 leader_election.cc:290] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:15.043169 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96356b5b69fc405cb1678dbfaec2df1f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:15.046101 22152 leader_election.cc:343] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:15.046911 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96356b5b69fc405cb1678dbfaec2df1f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:15.045450 22226 leader_election.cc:343] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:15.051048 22227 leader_election.cc:343] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:15.051519 22227 leader_election.cc:304] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:15.052276 22607 raft_consensus.cc:2749] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:08:15.092571 22152 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 55834.1us
I20260709 13:08:15.093820 22741 raft_consensus.cc:2804] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:15.104853 22741 raft_consensus.cc:493] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.107056 22741 raft_consensus.cc:3060] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.119513 22455 tablet_bootstrap.cc:492] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:15.120111 22455 ts_tablet_manager.cc:1403] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.204s	user 0.014s	sys 0.001s
W20260709 13:08:15.133227 22151 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 78360.3us
I20260709 13:08:15.129155 22333 raft_consensus.cc:2468] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:15.140607 22332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb4bb4b406fa4f4f9a55ba3b7c005cf0" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:15.147648 22921 leader_election.cc:290] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:15.188969 22456 tablet_bootstrap.cc:654] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:15.193538 22083 catalog_manager.cc:5697] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:15.204367 22083 catalog_manager.cc:5697] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:15.207886 22152 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 64110.5us
I20260709 13:08:15.228765 22458 tablet_bootstrap.cc:492] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:15.229456 22458 ts_tablet_manager.cc:1403] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.414s	user 0.012s	sys 0.021s
W20260709 13:08:15.234313 22943 log.cc:927] Time spent T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:08:15.234009 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb4bb4b406fa4f4f9a55ba3b7c005cf0" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
W20260709 13:08:15.250656 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 76284.9us
I20260709 13:08:15.251247 22184 raft_consensus.cc:3060] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.237943 22942 raft_consensus.cc:515] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:15.260964 22942 leader_election.cc:290] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:15.271490 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
W20260709 13:08:15.276250 22375 leader_election.cc:343] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:15.290356 22151 leader_election.cc:343] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:15.293360 22458 raft_consensus.cc:359] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:15.294346 22458 raft_consensus.cc:385] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:15.294642 22458 raft_consensus.cc:740] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:15.295348 22458 consensus_queue.cc:260] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:15.283968 22455 raft_consensus.cc:359] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:15.298659 22455 raft_consensus.cc:385] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:15.299032 22455 raft_consensus.cc:740] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:15.303843 22455 consensus_queue.cc:260] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:15.313731 22458 ts_tablet_manager.cc:1434] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.084s	user 0.009s	sys 0.000s
I20260709 13:08:15.302513 22181 raft_consensus.cc:2468] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 2.
I20260709 13:08:15.317629 22741 raft_consensus.cc:515] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:15.323113 22184 raft_consensus.cc:2468] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:15.325219 22458 tablet_bootstrap.cc:492] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:15.305274 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:15.328109 22262 raft_consensus.cc:3060] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.337033 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4bb1a452944c59aa3dc6717c52be6a" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:15.338116 22261 raft_consensus.cc:3060] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.339042 22375 leader_election.cc:304] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:15.339957 22956 raft_consensus.cc:2804] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:15.340523 22956 raft_consensus.cc:697] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:15.341688 22956 consensus_queue.cc:237] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:15.337471 22455 ts_tablet_manager.cc:1434] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.217s	user 0.006s	sys 0.003s
W20260709 13:08:15.357221 22300 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 87568.5us
I20260709 13:08:15.360975 22456 tablet_bootstrap.cc:492] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:15.364672 22456 ts_tablet_manager.cc:1403] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.364s	user 0.008s	sys 0.018s
I20260709 13:08:15.366025 22455 tablet_bootstrap.cc:492] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:15.372320 22375 leader_election.cc:304] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:15.374091 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4bb1a452944c59aa3dc6717c52be6a" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
W20260709 13:08:15.375775 22152 leader_election.cc:343] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:15.377229 22815 raft_consensus.cc:2804] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:15.377815 22815 raft_consensus.cc:697] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:15.378736 22815 consensus_queue.cc:237] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:15.379864 22741 leader_election.cc:290] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:15.391250 22261 raft_consensus.cc:2468] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:15.391325 22262 raft_consensus.cc:2468] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:15.398648 22151 leader_election.cc:304] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:15.400358 22953 raft_consensus.cc:2804] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:15.401289 22953 raft_consensus.cc:697] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:15.402264 22953 consensus_queue.cc:237] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:15.404964 22458 tablet_bootstrap.cc:654] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:15.404963 22455 tablet_bootstrap.cc:654] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:15.408576 22151 leader_election.cc:304] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:15.409483 22952 raft_consensus.cc:2804] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:15.410225 22952 raft_consensus.cc:697] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:15.410702 22456 raft_consensus.cc:359] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:15.412878 22952 consensus_queue.cc:237] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:15.411445 22456 raft_consensus.cc:385] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:15.417385 22456 raft_consensus.cc:740] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:15.418082 22456 consensus_queue.cc:260] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:15.427202 22455 tablet_bootstrap.cc:492] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:15.435047 22455 ts_tablet_manager.cc:1403] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.070s	user 0.013s	sys 0.002s
I20260709 13:08:15.427841 22456 ts_tablet_manager.cc:1434] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.000s
I20260709 13:08:15.441411 22456 tablet_bootstrap.cc:492] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:15.446362 22956 consensus_queue.cc:1048] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.458752 22085 catalog_manager.cc:5697] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 2 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:15.463800 22085 catalog_manager.cc:5697] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:15.467212 22082 catalog_manager.cc:5697] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:15.469739 22082 catalog_manager.cc:5697] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:15.474488 22455 raft_consensus.cc:359] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:15.479339 22455 raft_consensus.cc:385] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:15.479611 22455 raft_consensus.cc:740] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:15.480542 22455 consensus_queue.cc:260] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:15.487136 22961 raft_consensus.cc:493] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.487769 22961 raft_consensus.cc:515] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:15.488972 22956 consensus_queue.cc:1048] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:15.491163 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d38314054d8474eb00ca91785ea945d" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:15.496803 22458 tablet_bootstrap.cc:492] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:15.497390 22458 ts_tablet_manager.cc:1403] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.172s	user 0.018s	sys 0.030s
I20260709 13:08:15.498345 22961 leader_election.cc:290] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:15.499653 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d38314054d8474eb00ca91785ea945d" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:15.507592 22455 ts_tablet_manager.cc:1434] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.070s	user 0.002s	sys 0.005s
I20260709 13:08:15.509219 22455 tablet_bootstrap.cc:492] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:15.531117 22300 leader_election.cc:304] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:15.534310 22617 raft_consensus.cc:493] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.538952 22617 raft_consensus.cc:515] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:15.543278 22617 leader_election.cc:290] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:15.550534 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:15.535676 22961 raft_consensus.cc:2749] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:15.556756 22458 raft_consensus.cc:359] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:15.557633 22458 raft_consensus.cc:385] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:15.559860 22456 tablet_bootstrap.cc:654] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:15.557891 22458 raft_consensus.cc:740] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:15.562682 22458 consensus_queue.cc:260] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:15.565385 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:15.565490 22954 consensus_queue.cc:1048] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:08:15.564112 22151 leader_election.cc:343] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:15.574275 22455 tablet_bootstrap.cc:654] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:15.576537 22458 ts_tablet_manager.cc:1434] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.079s	user 0.007s	sys 0.002s
I20260709 13:08:15.568588 22261 raft_consensus.cc:2468] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:15.583340 22956 raft_consensus.cc:493] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.583918 22956 raft_consensus.cc:515] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:15.586184 22956 leader_election.cc:290] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:15.592638 22261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:15.592876 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:15.593561 22261 raft_consensus.cc:2393] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:15.590657 22815 consensus_queue.cc:1048] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:08:15.597651 22375 leader_election.cc:304] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:15.600183 22151 leader_election.cc:304] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:15.604601 22954 raft_consensus.cc:3060] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.605566 22617 raft_consensus.cc:2804] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:15.617559 22458 tablet_bootstrap.cc:492] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:15.626514 22458 tablet_bootstrap.cc:654] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:15.630718 22921 raft_consensus.cc:493] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.631289 22921 raft_consensus.cc:515] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:15.633411 22921 leader_election.cc:290] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:15.616233 22617 raft_consensus.cc:493] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.638092 22617 raft_consensus.cc:3060] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.659480 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96356b5b69fc405cb1678dbfaec2df1f" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:15.661657 22375 leader_election.cc:343] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:15.670454 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96356b5b69fc405cb1678dbfaec2df1f" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:15.671897 22262 raft_consensus.cc:2468] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:15.690409 22374 leader_election.cc:304] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:15.692989 22458 tablet_bootstrap.cc:492] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607: No bootstrap required, opened a new log
I20260709 13:08:15.693640 22458 ts_tablet_manager.cc:1403] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.084s	user 0.020s	sys 0.000s
I20260709 13:08:15.699857 22812 raft_consensus.cc:2804] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:15.700402 22812 raft_consensus.cc:493] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.700766 22812 raft_consensus.cc:3060] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.703917 22456 tablet_bootstrap.cc:492] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:15.704564 22456 ts_tablet_manager.cc:1403] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.263s	user 0.018s	sys 0.011s
I20260709 13:08:15.718695 22333 raft_consensus.cc:3060] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.749706 22954 raft_consensus.cc:2749] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:15.760155 22456 raft_consensus.cc:359] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:15.761039 22456 raft_consensus.cc:385] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:15.761300 22456 raft_consensus.cc:740] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:15.765693 22455 tablet_bootstrap.cc:492] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:15.768855 22455 ts_tablet_manager.cc:1403] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.260s	user 0.034s	sys 0.005s
I20260709 13:08:15.779731 22456 consensus_queue.cc:260] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:15.784471 22458 raft_consensus.cc:359] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:15.785347 22455 raft_consensus.cc:359] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:15.789678 22456 ts_tablet_manager.cc:1434] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.078s	user 0.006s	sys 0.002s
I20260709 13:08:15.798877 22455 raft_consensus.cc:385] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:15.799347 22455 raft_consensus.cc:740] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:15.800040 22458 raft_consensus.cc:385] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:15.801368 22458 raft_consensus.cc:740] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:15.802656 22456 tablet_bootstrap.cc:492] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:15.797819 22741 raft_consensus.cc:493] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.803473 22741 raft_consensus.cc:515] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:15.808251 22455 consensus_queue.cc:260] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:15.808859 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dfb3dc107a469c880393583ee0c218" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:15.805804 22741 leader_election.cc:290] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:15.810801 22458 consensus_queue.cc:260] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:15.835523 22261 raft_consensus.cc:3060] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:15.839598 22152 leader_election.cc:343] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:15.841746 22456 tablet_bootstrap.cc:654] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:15.842484 22455 ts_tablet_manager.cc:1434] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.073s	user 0.010s	sys 0.001s
I20260709 13:08:15.820631 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dfb3dc107a469c880393583ee0c218" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:15.845999 22458 ts_tablet_manager.cc:1434] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.152s	user 0.010s	sys 0.001s
I20260709 13:08:15.843657 22617 raft_consensus.cc:515] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:15.849674 22261 raft_consensus.cc:1275] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Refusing update from remote peer 67fc9e7fc4b34621ae1cd3e1621b89dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.851357 22811 consensus_queue.cc:1048] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:15.857432 22256 raft_consensus.cc:2468] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 0.
I20260709 13:08:15.858646 22184 raft_consensus.cc:1275] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Refusing update from remote peer 67fc9e7fc4b34621ae1cd3e1621b89dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.861725 22151 leader_election.cc:304] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:15.867357 22455 tablet_bootstrap.cc:492] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:15.871412 22617 leader_election.cc:290] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:15.839565 22701 consensus_queue.cc:1048] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 13:08:15.871654 22952 raft_consensus.cc:2804] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:08:15.881711 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:15.888917 22979 consensus_queue.cc:1048] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:15.894269 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
W20260709 13:08:15.897140 22151 leader_election.cc:343] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:15.891141 22952 raft_consensus.cc:493] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.906301 22952 raft_consensus.cc:3060] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.914762 22258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:15.915585 22258 raft_consensus.cc:3060] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:15.919428 22375 consensus_peers.cc:597] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:15.945922 22456 tablet_bootstrap.cc:492] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:15.946612 22456 ts_tablet_manager.cc:1403] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.007s
I20260709 13:08:15.955511 22258 raft_consensus.cc:2468] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:15.956799 22741 consensus_queue.cc:1048] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.962399 22983 consensus_queue.cc:1048] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:15.969419 22741 consensus_queue.cc:1048] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:15.969655 22456 raft_consensus.cc:359] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:15.970450 22456 raft_consensus.cc:385] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:15.970748 22456 raft_consensus.cc:740] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:15.969434 22961 raft_consensus.cc:493] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.972424 22961 raft_consensus.cc:515] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:15.955870 22952 raft_consensus.cc:515] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:15.958850 22812 raft_consensus.cc:515] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:15.975446 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96356b5b69fc405cb1678dbfaec2df1f" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:15.976176 22256 raft_consensus.cc:2468] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 0.
I20260709 13:08:15.977105 22812 leader_election.cc:290] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:15.978432 22961 leader_election.cc:290] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:15.974699 22456 consensus_queue.cc:260] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:15.975091 22701 raft_consensus.cc:493] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.980702 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96356b5b69fc405cb1678dbfaec2df1f" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:15.981084 22701 raft_consensus.cc:515] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:15.981402 22456 ts_tablet_manager.cc:1434] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:08:15.981375 22256 raft_consensus.cc:3060] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.983217 22701 leader_election.cc:290] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:15.979094 22300 leader_election.cc:304] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:15.980568 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96356b5b69fc405cb1678dbfaec2df1f" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:15.984819 22961 raft_consensus.cc:2804] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:15.985160 22407 raft_consensus.cc:2393] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93815010a884438c86fc6479359b69e8 in current term 1: Already voted for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in this term.
I20260709 13:08:15.985260 22961 raft_consensus.cc:493] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.985877 22961 raft_consensus.cc:3060] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.987967 22456 tablet_bootstrap.cc:492] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:15.997752 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96356b5b69fc405cb1678dbfaec2df1f" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:16.000023 22181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42127cae8f494cb081ec04febe577130" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:16.000742 22181 raft_consensus.cc:2468] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:16.002655 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42127cae8f494cb081ec04febe577130" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:16.003355 22456 tablet_bootstrap.cc:654] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:16.015776 22375 leader_election.cc:343] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:16.021377 22986 log.cc:927] Time spent T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:08:16.021508 22456 tablet_bootstrap.cc:492] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:16.038722 22456 ts_tablet_manager.cc:1403] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.051s	user 0.011s	sys 0.003s
I20260709 13:08:16.039217 22961 raft_consensus.cc:515] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:16.040632 22330 raft_consensus.cc:2393] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 1: Already voted for candidate 93815010a884438c86fc6479359b69e8 in this term.
W20260709 13:08:16.030360 22152 consensus_peers.cc:597] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:16.042730 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dfb3dc107a469c880393583ee0c218" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
W20260709 13:08:16.044148 22152 leader_election.cc:343] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:16.045574 22456 raft_consensus.cc:359] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:16.046407 22456 raft_consensus.cc:385] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:16.046667 22456 raft_consensus.cc:740] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:15.960651 22455 tablet_bootstrap.cc:654] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:16.049705 22456 consensus_queue.cc:260] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:16.051399 22456 ts_tablet_manager.cc:1434] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:08:16.052491 22456 tablet_bootstrap.cc:492] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:16.033066 22952 leader_election.cc:290] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:16.032270 22375 leader_election.cc:304] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:16.054251 22701 raft_consensus.cc:2804] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:16.054666 22701 raft_consensus.cc:493] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.055032 22701 raft_consensus.cc:3060] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.032176 22151 leader_election.cc:304] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:16.060810 22683 raft_consensus.cc:2804] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:16.061245 22258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dfb3dc107a469c880393583ee0c218" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:16.062026 22258 raft_consensus.cc:3060] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.068284 22256 raft_consensus.cc:2468] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:16.069826 22374 leader_election.cc:304] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:16.070819 22815 raft_consensus.cc:2804] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:16.071229 22456 tablet_bootstrap.cc:654] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:16.071411 22683 raft_consensus.cc:697] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:16.071360 22815 raft_consensus.cc:697] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:16.072427 22683 consensus_queue.cc:237] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:16.072557 22815 consensus_queue.cc:237] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
W20260709 13:08:16.082211 22300 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 103194us
W20260709 13:08:16.084367 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 52174.5us
I20260709 13:08:16.092478 22456 tablet_bootstrap.cc:492] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:16.093081 22456 ts_tablet_manager.cc:1403] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.003s
I20260709 13:08:16.096155 22456 raft_consensus.cc:359] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:16.102536 22456 raft_consensus.cc:385] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:16.102856 22456 raft_consensus.cc:740] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:16.103567 22456 consensus_queue.cc:260] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:16.112758 22961 leader_election.cc:290] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:16.121719 22456 ts_tablet_manager.cc:1434] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.002s
I20260709 13:08:16.122985 22456 tablet_bootstrap.cc:492] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:16.128144 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96356b5b69fc405cb1678dbfaec2df1f" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:16.071055 22701 raft_consensus.cc:515] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:16.130270 22701 leader_election.cc:290] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:16.127660 22258 raft_consensus.cc:2468] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 1.
I20260709 13:08:16.149523 22151 leader_election.cc:304] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 93815010a884438c86fc6479359b69e8
I20260709 13:08:16.150841 22952 raft_consensus.cc:2804] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:16.151383 22952 raft_consensus.cc:697] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:16.152354 22952 consensus_queue.cc:237] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:16.128185 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96356b5b69fc405cb1678dbfaec2df1f" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:16.154808 22262 raft_consensus.cc:2393] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 93815010a884438c86fc6479359b69e8 in current term 1: Already voted for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in this term.
I20260709 13:08:16.138172 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42127cae8f494cb081ec04febe577130" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:16.138799 22456 tablet_bootstrap.cc:654] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:16.155799 22184 raft_consensus.cc:3060] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.134734 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42127cae8f494cb081ec04febe577130" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:16.159126 22330 raft_consensus.cc:3060] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.157194 22084 catalog_manager.cc:5697] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:16.176257 22330 raft_consensus.cc:2468] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:16.177459 22300 leader_election.cc:304] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:16.179222 22961 raft_consensus.cc:493] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.190987 22455 tablet_bootstrap.cc:492] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:16.191550 22455 ts_tablet_manager.cc:1403] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.325s	user 0.020s	sys 0.032s
I20260709 13:08:16.193938 22962 raft_consensus.cc:2749] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:16.183831 22184 raft_consensus.cc:2468] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:16.199405 22375 leader_election.cc:304] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:16.194870 22961 raft_consensus.cc:515] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:16.202016 22701 raft_consensus.cc:2804] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:16.202605 22701 raft_consensus.cc:697] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:16.199918 22455 raft_consensus.cc:359] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:16.203241 22455 raft_consensus.cc:385] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:16.203678 22701 consensus_queue.cc:237] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:16.203671 22455 raft_consensus.cc:740] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:16.207005 22455 consensus_queue.cc:260] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:16.206740 22961 leader_election.cc:290] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:16.210559 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4bb1a452944c59aa3dc6717c52be6a" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:16.217787 22455 ts_tablet_manager.cc:1434] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.003s
I20260709 13:08:16.215584 22961 raft_consensus.cc:493] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.218492 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4bb1a452944c59aa3dc6717c52be6a" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:16.218724 22961 raft_consensus.cc:515] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:16.219242 22455 tablet_bootstrap.cc:492] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:16.224320 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb4bb4b406fa4f4f9a55ba3b7c005cf0" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:16.226231 22300 leader_election.cc:304] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:16.225540 22961 leader_election.cc:290] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:16.230243 22961 raft_consensus.cc:2749] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:16.231312 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb4bb4b406fa4f4f9a55ba3b7c005cf0" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:16.231886 22683 raft_consensus.cc:493] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.232399 22683 raft_consensus.cc:515] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:16.234485 22683 leader_election.cc:290] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:16.235397 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91b5edf8c9e4a71a195e0f796d439f3" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:16.235467 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91b5edf8c9e4a71a195e0f796d439f3" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:16.236841 22152 leader_election.cc:343] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:16.239611 22151 leader_election.cc:343] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:16.240082 22151 leader_election.cc:304] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 652d70a0b765447792ab9b4383712607; no voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8
I20260709 13:08:16.241768 22952 raft_consensus.cc:2749] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:16.242628 22456 tablet_bootstrap.cc:492] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:16.244670 22456 ts_tablet_manager.cc:1403] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.122s	user 0.020s	sys 0.013s
I20260709 13:08:16.244066 22455 tablet_bootstrap.cc:654] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:16.249112 22300 leader_election.cc:304] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:16.252801 22961 raft_consensus.cc:2749] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:16.253213 22456 raft_consensus.cc:359] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:16.254019 22456 raft_consensus.cc:385] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:16.254379 22456 raft_consensus.cc:740] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:16.255344 22330 raft_consensus.cc:3060] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.293787 22455 tablet_bootstrap.cc:492] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:16.294467 22455 ts_tablet_manager.cc:1403] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.076s	user 0.014s	sys 0.016s
I20260709 13:08:16.297602 22455 raft_consensus.cc:359] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:16.298354 22455 raft_consensus.cc:385] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:16.298624 22455 raft_consensus.cc:740] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:16.299306 22455 consensus_queue.cc:260] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:16.301093 22455 ts_tablet_manager.cc:1434] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:08:16.302023 22455 tablet_bootstrap.cc:492] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Bootstrap starting.
I20260709 13:08:16.307607 22455 tablet_bootstrap.cc:654] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:16.308431 22084 catalog_manager.cc:5697] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:16.255378 22456 consensus_queue.cc:260] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:16.329620 22455 tablet_bootstrap.cc:492] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc: No bootstrap required, opened a new log
I20260709 13:08:16.330209 22455 ts_tablet_manager.cc:1403] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.000s
I20260709 13:08:16.330204 22083 catalog_manager.cc:5697] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:16.332199 22083 catalog_manager.cc:5697] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:16.333125 22456 ts_tablet_manager.cc:1434] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.001s
I20260709 13:08:16.336091 22456 tablet_bootstrap.cc:492] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:16.344604 22455 raft_consensus.cc:359] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:16.345467 22455 raft_consensus.cc:385] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:16.345765 22455 raft_consensus.cc:740] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Initialized, Role: FOLLOWER
I20260709 13:08:16.347325 22455 consensus_queue.cc:260] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:16.352182 22455 ts_tablet_manager.cc:1434] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.002s
I20260709 13:08:16.353425 22456 tablet_bootstrap.cc:654] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:16.388470 22456 tablet_bootstrap.cc:492] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:16.389026 22456 ts_tablet_manager.cc:1403] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.053s	user 0.019s	sys 0.018s
I20260709 13:08:16.393503 22456 raft_consensus.cc:359] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:16.394279 22456 raft_consensus.cc:385] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:16.394780 22456 raft_consensus.cc:740] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:16.395577 22456 consensus_queue.cc:260] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:16.398500 22456 ts_tablet_manager.cc:1434] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:08:16.399602 22456 tablet_bootstrap.cc:492] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8: Bootstrap starting.
I20260709 13:08:16.414681 22456 tablet_bootstrap.cc:654] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:16.425935 22456 tablet_bootstrap.cc:492] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8: No bootstrap required, opened a new log
I20260709 13:08:16.426854 22456 ts_tablet_manager.cc:1403] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.003s
I20260709 13:08:16.430028 22456 raft_consensus.cc:359] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:16.430788 22456 raft_consensus.cc:385] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:16.431093 22456 raft_consensus.cc:740] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93815010a884438c86fc6479359b69e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:16.431806 22456 consensus_queue.cc:260] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:16.434163 22456 ts_tablet_manager.cc:1434] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:08:16.543506 22330 raft_consensus.cc:3060] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.581758 22815 raft_consensus.cc:493] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.582389 22815 raft_consensus.cc:515] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:16.585800 22815 leader_election.cc:290] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:16.587179 22258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:16.587985 22258 raft_consensus.cc:2393] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 1: Already voted for candidate 652d70a0b765447792ab9b4383712607 in this term.
I20260709 13:08:16.595157 22330 raft_consensus.cc:3060] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.595835 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:16.603545 22375 leader_election.cc:304] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:16.604820 22886 raft_consensus.cc:3060] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.611531 22683 consensus_queue.cc:1048] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:16.625218 22815 consensus_queue.cc:1048] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:16.628293 22886 raft_consensus.cc:2749] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:16.636485 22683 consensus_queue.cc:1048] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:16.641039 22812 raft_consensus.cc:493] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.641592 22812 raft_consensus.cc:515] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:16.644650 22812 leader_election.cc:290] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:16.648579 22258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d9ed71058164d51a8aa5b9980536bd8" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:16.649214 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d9ed71058164d51a8aa5b9980536bd8" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:16.649986 22333 raft_consensus.cc:2468] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:16.650893 22701 consensus_queue.cc:1048] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:16.649327 22258 raft_consensus.cc:2468] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:16.652709 22374 leader_election.cc:304] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:16.653880 22741 consensus_queue.cc:1048] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:16.653940 22812 raft_consensus.cc:2804] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:16.656338 22812 raft_consensus.cc:493] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.656755 22812 raft_consensus.cc:3060] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.669229 22701 consensus_queue.cc:1048] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:16.688692 22886 raft_consensus.cc:493] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.689287 22886 raft_consensus.cc:515] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:16.691632 22886 leader_election.cc:290] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:16.696271 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91b5edf8c9e4a71a195e0f796d439f3" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607" is_pre_election: true
I20260709 13:08:16.697129 22184 raft_consensus.cc:2468] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:16.700301 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91b5edf8c9e4a71a195e0f796d439f3" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:16.700986 22330 raft_consensus.cc:2468] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 0.
I20260709 13:08:16.704434 22375 leader_election.cc:304] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:16.708479 22701 raft_consensus.cc:2804] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:16.708899 22701 raft_consensus.cc:493] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.709286 22701 raft_consensus.cc:3060] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.719007 22812 raft_consensus.cc:515] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:16.721603 22812 leader_election.cc:290] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:16.723326 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d9ed71058164d51a8aa5b9980536bd8" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:16.724048 22259 raft_consensus.cc:3060] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.817878 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d9ed71058164d51a8aa5b9980536bd8" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:16.818719 22333 raft_consensus.cc:3060] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.820106 22701 raft_consensus.cc:515] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:16.835850 22701 leader_election.cc:290] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
W20260709 13:08:16.829774 22374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 99108.7us
W20260709 13:08:16.862555 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 82748.4us
I20260709 13:08:16.882423 22333 raft_consensus.cc:2468] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:16.885771 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91b5edf8c9e4a71a195e0f796d439f3" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:16.886703 22333 raft_consensus.cc:3060] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.903285 22184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91b5edf8c9e4a71a195e0f796d439f3" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652d70a0b765447792ab9b4383712607"
I20260709 13:08:16.904235 22184 raft_consensus.cc:3060] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:16.914806 23014 log.cc:927] Time spent T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:08:16.933696 22259 raft_consensus.cc:2468] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:16.952502 22333 raft_consensus.cc:2468] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
W20260709 13:08:16.995621 22152 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 54836.8us
W20260709 13:08:17.006831 22133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:17.053376 22374 leader_election.cc:304] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:17.024794 22923 consensus_queue.cc:1048] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 13:08:17.065567 22375 leader_election.cc:304] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:17.071501 23027 raft_consensus.cc:2804] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:17.079248 23027 raft_consensus.cc:697] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:17.080272 23027 consensus_queue.cc:237] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:17.085136 23034 raft_consensus.cc:2804] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:17.086038 23034 raft_consensus.cc:697] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:17.087316 23034 consensus_queue.cc:237] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
W20260709 13:08:17.103578 22151 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 58427.8us
I20260709 13:08:17.139365 22184 raft_consensus.cc:2468] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
W20260709 13:08:17.158209 22226 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:41261, user_credentials={real_user=slave}} blocked reactor thread for 54051.4us
I20260709 13:08:17.184652 22812 consensus_queue.cc:1048] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:17.252231 22083 catalog_manager.cc:5697] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:17.263196 22083 catalog_manager.cc:5697] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 1 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:17.484097 23020 consensus_queue.cc:1048] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:17.515343 23017 consensus_queue.cc:1048] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:17.685258 22886 consensus_queue.cc:1048] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:17.698309 23017 consensus_queue.cc:1048] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:08:17.846220 22301 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 50479.3us
I20260709 13:08:18.052227 12725 tablet_server.cc:179] TabletServer@127.12.109.65:0 shutting down...
W20260709 13:08:18.077158 22375 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20260709 13:08:18.147596 22227 consensus_peers.cc:597] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.151407 22375 consensus_peers.cc:597] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.156704 22227 consensus_peers.cc:597] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.171236 22301 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 53840.4us
W20260709 13:08:18.175093 22301 consensus_peers.cc:597] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.176239 22375 consensus_peers.cc:597] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.178071 22301 consensus_peers.cc:597] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.183524 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:18.189422 12725 tablet_replica.cc:333] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.190596 12725 raft_consensus.cc:2243] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.191799 12725 raft_consensus.cc:2272] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.194579 12725 tablet_replica.cc:333] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.195261 12725 raft_consensus.cc:2243] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.196087 12725 raft_consensus.cc:2272] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.198539 12725 tablet_replica.cc:333] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.199411 12725 raft_consensus.cc:2243] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.200453 12725 raft_consensus.cc:2272] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:18.202148 22227 consensus_peers.cc:597] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.203274 12725 tablet_replica.cc:333] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.204048 12725 raft_consensus.cc:2243] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.204602 12725 raft_consensus.cc:2272] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.207135 12725 tablet_replica.cc:333] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.208053 12725 raft_consensus.cc:2243] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.208927 12725 raft_consensus.cc:2272] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.211977 12725 tablet_replica.cc:333] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.212826 12725 raft_consensus.cc:2243] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.213395 12725 raft_consensus.cc:2272] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:18.215572 22375 consensus_peers.cc:597] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.215919 12725 tablet_replica.cc:333] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.216712 12725 raft_consensus.cc:2243] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.217211 12725 raft_consensus.cc:2272] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.219570 12725 tablet_replica.cc:333] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.220518 12725 raft_consensus.cc:2243] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.221599 12725 raft_consensus.cc:2272] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.224582 12725 tablet_replica.cc:333] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.225304 12725 raft_consensus.cc:2243] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.225766 12725 raft_consensus.cc:2272] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.227777 12725 tablet_replica.cc:333] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.228536 12725 raft_consensus.cc:2243] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.229377 12725 raft_consensus.cc:2272] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.232031 12725 tablet_replica.cc:333] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.232733 12725 raft_consensus.cc:2243] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.233569 12725 raft_consensus.cc:2272] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.236023 12725 tablet_replica.cc:333] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.236799 12725 raft_consensus.cc:2243] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.237919 12725 raft_consensus.cc:2272] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.240912 12725 tablet_replica.cc:333] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.241657 12725 raft_consensus.cc:2243] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.242286 12725 raft_consensus.cc:2272] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.244436 12725 tablet_replica.cc:333] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.245177 12725 raft_consensus.cc:2243] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.245766 12725 raft_consensus.cc:2272] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.248946 12725 tablet_replica.cc:333] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.249744 12725 raft_consensus.cc:2243] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.250283 12725 raft_consensus.cc:2272] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.252763 12725 tablet_replica.cc:333] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.253420 12725 raft_consensus.cc:2243] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.254448 12725 raft_consensus.cc:2272] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.257181 12725 tablet_replica.cc:333] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.258229 12725 raft_consensus.cc:2243] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.259096 12725 raft_consensus.cc:2272] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.262665 12725 tablet_replica.cc:333] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.263568 12725 raft_consensus.cc:2243] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.264175 12725 raft_consensus.cc:2272] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.268450 12725 tablet_replica.cc:333] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.269325 12725 raft_consensus.cc:2243] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:18.269531 22375 consensus_peers.cc:597] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.269876 12725 raft_consensus.cc:2272] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:18.270324 22375 consensus_peers.cc:597] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.272089 12725 tablet_replica.cc:333] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.272815 12725 raft_consensus.cc:2243] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.273307 12725 raft_consensus.cc:2272] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.276033 12725 tablet_replica.cc:333] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.277011 12725 raft_consensus.cc:2243] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.277529 12725 raft_consensus.cc:2272] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.279872 12725 tablet_replica.cc:333] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.280750 12725 raft_consensus.cc:2243] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.281836 12725 raft_consensus.cc:2272] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.285179 12725 tablet_replica.cc:333] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.286132 12725 raft_consensus.cc:2243] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.286649 12725 raft_consensus.cc:2272] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.289166 12725 tablet_replica.cc:333] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.289930 12725 raft_consensus.cc:2243] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.290531 12725 raft_consensus.cc:2272] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.294548 12725 tablet_replica.cc:333] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.295266 12725 raft_consensus.cc:2243] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.295815 12725 raft_consensus.cc:2272] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.298472 12725 tablet_replica.cc:333] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
W20260709 13:08:18.298458 22227 consensus_peers.cc:597] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.299752 12725 raft_consensus.cc:2243] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.300446 12725 raft_consensus.cc:2272] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.303450 12725 tablet_replica.cc:333] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.304167 12725 raft_consensus.cc:2243] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.305276 12725 raft_consensus.cc:2272] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:18.305387 22301 consensus_peers.cc:597] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.308331 12725 tablet_replica.cc:333] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.309268 12725 raft_consensus.cc:2243] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.319303 12725 raft_consensus.cc:2272] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.322813 12725 tablet_replica.cc:333] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.323808 12725 raft_consensus.cc:2243] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.324324 12725 raft_consensus.cc:2272] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.327118 12725 tablet_replica.cc:333] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.328011 12725 raft_consensus.cc:2243] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.328583 12725 raft_consensus.cc:2272] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:18.331305 22375 consensus_peers.cc:597] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.331521 12725 tablet_replica.cc:333] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.332451 12725 raft_consensus.cc:2243] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:08:18.333194 22227 consensus_peers.cc:597] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.333854 12725 raft_consensus.cc:2272] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.337129 12725 tablet_replica.cc:333] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.338088 12725 raft_consensus.cc:2243] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.339004 12725 raft_consensus.cc:2272] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.341615 12725 tablet_replica.cc:333] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.342319 12725 raft_consensus.cc:2243] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.342844 12725 raft_consensus.cc:2272] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.346290 12725 tablet_replica.cc:333] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.347134 12725 raft_consensus.cc:2243] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.347590 12725 raft_consensus.cc:2272] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.350171 12725 tablet_replica.cc:333] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.350780 12725 raft_consensus.cc:2243] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.351753 12725 raft_consensus.cc:2272] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.353976 12725 tablet_replica.cc:333] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.354621 12725 raft_consensus.cc:2243] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.355295 12725 raft_consensus.cc:2272] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.358417 12725 tablet_replica.cc:333] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.359067 12725 raft_consensus.cc:2243] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.359979 12725 raft_consensus.cc:2272] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.362849 12725 tablet_replica.cc:333] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.363590 12725 raft_consensus.cc:2243] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.364897 12725 raft_consensus.cc:2272] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.367592 12725 tablet_replica.cc:333] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.368144 12725 raft_consensus.cc:2243] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.368614 12725 raft_consensus.cc:2272] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.370994 12725 tablet_replica.cc:333] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.371584 12725 raft_consensus.cc:2243] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:18.372491 12725 raft_consensus.cc:2272] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.374876 12725 tablet_replica.cc:333] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.375548 12725 raft_consensus.cc:2243] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.376256 12725 raft_consensus.cc:2272] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.379415 12725 tablet_replica.cc:333] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.380213 12725 raft_consensus.cc:2243] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.380760 12725 raft_consensus.cc:2272] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:18.381325 22227 consensus_peers.cc:597] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.383752 12725 tablet_replica.cc:333] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.384615 12725 raft_consensus.cc:2243] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:08:18.385577 22375 consensus_peers.cc:597] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.385681 12725 raft_consensus.cc:2272] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.388484 12725 tablet_replica.cc:333] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.389110 12725 raft_consensus.cc:2243] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.389607 12725 raft_consensus.cc:2272] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:18.392305 12725 tablet_replica.cc:333] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607: stopping tablet replica
I20260709 13:08:18.392951 12725 raft_consensus.cc:2243] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:18.393482 12725 raft_consensus.cc:2272] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:18.403100 22301 consensus_peers.cc:597] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.435962 22227 consensus_peers.cc:597] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.478479 22375 connection.cc:769] client connection to 127.12.109.65:46399 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:08:18.479238 22375 consensus_peers.cc:597] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: sendmsg error: Broken pipe (error 32). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.479688 22375 consensus_peers.cc:597] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: sendmsg error: Broken pipe (error 32). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.480108 22375 consensus_peers.cc:597] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: sendmsg error: Broken pipe (error 32). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.492730 22301 consensus_peers.cc:597] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.497272 22375 consensus_peers.cc:597] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.509634 22375 consensus_peers.cc:597] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.562315 22375 consensus_peers.cc:597] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.575700 22375 consensus_peers.cc:597] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:19.269833 23061 raft_consensus.cc:493] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.270569 23061 raft_consensus.cc:515] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:19.274142 23020 raft_consensus.cc:493] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.274869 23020 raft_consensus.cc:515] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
W20260709 13:08:19.279402 22301 leader_election.cc:336] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.280436 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.284049 22300 leader_election.cc:304] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:19.284881 23067 raft_consensus.cc:2749] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:19.285005 23061 leader_election.cc:290] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.286968 23020 leader_election.cc:290] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.287979 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7f96ef51941e39bc97c0b769c7770" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
W20260709 13:08:19.293538 22375 leader_election.cc:336] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.294091 22375 leader_election.cc:304] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:19.295372 23020 raft_consensus.cc:2749] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:19.304196 23020 raft_consensus.cc:493] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.304816 23020 raft_consensus.cc:515] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.307824 23020 leader_election.cc:290] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.308691 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9b737a6d0d24895b749f55ac078ac5f" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
W20260709 13:08:19.311313 22375 leader_election.cc:336] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.311704 22375 leader_election.cc:304] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:19.312304 23020 raft_consensus.cc:2749] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:19.388198 23064 raft_consensus.cc:493] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.388890 23064 raft_consensus.cc:515] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.391285 23064 leader_election.cc:290] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.392268 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9b737a6d0d24895b749f55ac078ac5f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:19.392983 22405 raft_consensus.cc:2468] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:19.394208 22226 leader_election.cc:304] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.395080 23064 raft_consensus.cc:2804] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.395498 23064 raft_consensus.cc:493] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.395900 23064 raft_consensus.cc:3060] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.405228 23064 raft_consensus.cc:515] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.407987 23064 leader_election.cc:290] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.408174 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9b737a6d0d24895b749f55ac078ac5f" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:19.408962 22407 raft_consensus.cc:3060] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.409866 23064 raft_consensus.cc:493] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.410072 23065 raft_consensus.cc:493] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.410499 23064 raft_consensus.cc:515] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.410676 23065 raft_consensus.cc:515] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.413012 23065 leader_election.cc:290] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.413146 23064 leader_election.cc:290] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.414973 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa44244e76604221bcbd08e0f3bbdb82" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:19.415524 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:19.418143 22407 raft_consensus.cc:2468] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 2.
I20260709 13:08:19.419291 22226 leader_election.cc:304] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.420044 23064 raft_consensus.cc:2804] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:19.420475 23064 raft_consensus.cc:697] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:19.421308 23064 consensus_queue.cc:237] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.423799 23020 raft_consensus.cc:493] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.424382 23020 raft_consensus.cc:515] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.428817 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffdd1d227bbe48c8a46c4fefd02eab0b" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.431223 23020 leader_election.cc:290] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
W20260709 13:08:19.435516 22375 leader_election.cc:336] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.436144 22375 leader_election.cc:304] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:19.436954 23020 raft_consensus.cc:2749] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:19.443563 23020 raft_consensus.cc:493] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.444269 23020 raft_consensus.cc:515] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.447686 23020 leader_election.cc:290] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.451098 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa44244e76604221bcbd08e0f3bbdb82" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.452119 22256 raft_consensus.cc:2468] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:19.453550 22374 leader_election.cc:304] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.454733 23020 raft_consensus.cc:2804] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.455260 23020 raft_consensus.cc:493] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.455700 23020 raft_consensus.cc:3060] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:19.461386 22375 leader_election.cc:336] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.468776 22082 catalog_manager.cc:5697] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:19.472189 23020 raft_consensus.cc:515] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.473093 23064 raft_consensus.cc:493] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.474947 23064 raft_consensus.cc:515] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.475970 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa44244e76604221bcbd08e0f3bbdb82" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:19.476711 22256 raft_consensus.cc:3060] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.478964 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:19.481407 23064 leader_election.cc:290] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.486997 23068 raft_consensus.cc:493] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.487303 23065 raft_consensus.cc:493] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.487721 23068 raft_consensus.cc:515] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.487988 23065 raft_consensus.cc:515] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:19.489835 22375 leader_election.cc:336] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.490571 23068 leader_election.cc:290] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.492206 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f204603e14c438bb549d1116eaac788" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.493494 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dfb3dc107a469c880393583ee0c218" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
W20260709 13:08:19.494338 12725 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22208)
I20260709 13:08:19.495939 23065 leader_election.cc:290] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:19.494086 23020 leader_election.cc:290] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.498922 22256 raft_consensus.cc:2468] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 2.
I20260709 13:08:19.500080 22374 leader_election.cc:304] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607
I20260709 13:08:19.501297 23020 raft_consensus.cc:2804] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:19.502843 23020 raft_consensus.cc:697] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:19.503746 23020 consensus_queue.cc:237] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.505031 23065 raft_consensus.cc:493] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.505710 23065 raft_consensus.cc:515] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.508888 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "878ae1fea9154e7088fdba8930c5eb23" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
W20260709 13:08:19.511253 22375 leader_election.cc:336] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.511927 22375 leader_election.cc:304] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:19.512912 23068 raft_consensus.cc:2749] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:19.516338 23065 leader_election.cc:290] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.521534 23061 raft_consensus.cc:493] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.522148 23061 raft_consensus.cc:515] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:19.524286 23061 leader_election.cc:290] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.524966 23061 raft_consensus.cc:493] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.525429 23061 raft_consensus.cc:515] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:19.526530 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dfb3dc107a469c880393583ee0c218" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.527245 22256 raft_consensus.cc:2468] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:19.527726 23061 leader_election.cc:290] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.528640 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:19.530120 23068 raft_consensus.cc:493] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.530645 23068 raft_consensus.cc:515] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.531512 22300 leader_election.cc:304] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.532691 23068 leader_election.cc:290] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.535526 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.535763 23067 raft_consensus.cc:2804] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.536173 23067 raft_consensus.cc:493] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.536566 23067 raft_consensus.cc:3060] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.536175 22256 raft_consensus.cc:2468] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:19.538537 22374 leader_election.cc:304] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
W20260709 13:08:19.539463 22301 leader_election.cc:336] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.541254 23068 raft_consensus.cc:2804] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.544497 23068 raft_consensus.cc:493] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.545013 23068 raft_consensus.cc:3060] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:19.543505 22301 leader_election.cc:336] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.546545 22301 leader_election.cc:304] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:19.550837 23073 raft_consensus.cc:493] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.557863 23073 raft_consensus.cc:515] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:19.558385 22375 leader_election.cc:336] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.552928 23061 raft_consensus.cc:493] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.560276 23073 leader_election.cc:290] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.560830 23073 raft_consensus.cc:493] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.559324 23061 raft_consensus.cc:515] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:19.561314 23073 raft_consensus.cc:515] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:19.563417 23073 leader_election.cc:290] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.569634 23061 leader_election.cc:290] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.570315 23061 raft_consensus.cc:2749] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:19.570636 22083 catalog_manager.cc:5697] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 2 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:19.574218 23067 raft_consensus.cc:515] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:19.576376 23067 leader_election.cc:290] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
W20260709 13:08:19.591499 22301 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 48013.9us
I20260709 13:08:19.595613 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4bb1a452944c59aa3dc6717c52be6a" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.598176 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.600538 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3808206385a0458abc04050aeab1cea7" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.601863 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dfb3dc107a469c880393583ee0c218" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:19.602532 22257 raft_consensus.cc:3060] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.600282 23068 raft_consensus.cc:515] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.606702 23065 raft_consensus.cc:493] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.609632 23065 raft_consensus.cc:515] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:19.598898 23020 raft_consensus.cc:493] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.612535 23065 leader_election.cc:290] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:19.610875 23068 leader_election.cc:290] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.612330 23020 raft_consensus.cc:515] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:19.618037 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:19.618894 22256 raft_consensus.cc:3060] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.619820 23020 leader_election.cc:290] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.619997 23065 raft_consensus.cc:493] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.623162 23065 raft_consensus.cc:515] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.623554 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3808206385a0458abc04050aeab1cea7" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:19.624363 22331 raft_consensus.cc:2468] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:19.625392 23065 leader_election.cc:290] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.620543 23076 raft_consensus.cc:493] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.626072 23076 raft_consensus.cc:515] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:19.628168 23076 leader_election.cc:290] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.629503 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.630834 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffdd1d227bbe48c8a46c4fefd02eab0b" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:19.631524 23065 raft_consensus.cc:493] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.631572 22407 raft_consensus.cc:2468] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:19.632218 23065 raft_consensus.cc:515] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.633262 22226 leader_election.cc:304] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.634930 23065 leader_election.cc:290] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
W20260709 13:08:19.631318 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 46424.9us
I20260709 13:08:19.635864 22256 raft_consensus.cc:2468] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 2.
I20260709 13:08:19.636332 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f204603e14c438bb549d1116eaac788" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:19.637051 22374 leader_election.cc:304] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.637262 22407 raft_consensus.cc:2468] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:19.638077 23078 raft_consensus.cc:2804] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:19.638634 23078 raft_consensus.cc:697] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:19.639480 23068 raft_consensus.cc:493] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.639600 23078 consensus_queue.cc:237] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.640254 23073 raft_consensus.cc:493] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.640285 23068 raft_consensus.cc:515] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.640887 23073 raft_consensus.cc:515] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:19.642828 23068 leader_election.cc:290] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.643064 23073 leader_election.cc:290] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.643846 22226 leader_election.cc:304] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.644789 23065 raft_consensus.cc:2804] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.644753 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "878ae1fea9154e7088fdba8930c5eb23" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.634833 23064 raft_consensus.cc:2804] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.645481 23065 raft_consensus.cc:493] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.645673 22259 raft_consensus.cc:2468] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:19.646489 23064 raft_consensus.cc:493] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.647006 23064 raft_consensus.cc:3060] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.647835 23065 raft_consensus.cc:3060] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.647653 22374 leader_election.cc:304] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.648864 22886 raft_consensus.cc:2804] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.649269 22886 raft_consensus.cc:493] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.649678 22886 raft_consensus.cc:3060] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.660846 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01099e972e9b4d08a64552be3fd52276" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.661080 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8f7472c97049ca98e597c7d4b4eb1c" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:19.668424 23083 raft_consensus.cc:493] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.669035 23083 raft_consensus.cc:515] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:19.627871 22301 leader_election.cc:336] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.671489 23083 leader_election.cc:290] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:19.671692 22301 leader_election.cc:304] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:19.672060 23083 raft_consensus.cc:493] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.672544 23073 raft_consensus.cc:2749] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:19.672725 22301 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 44919.4us
W20260709 13:08:19.673348 22301 leader_election.cc:336] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.677083 22257 raft_consensus.cc:2468] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 2.
I20260709 13:08:19.679108 22300 leader_election.cc:304] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.680044 23073 raft_consensus.cc:2804] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:19.680593 23073 raft_consensus.cc:697] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 2 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
I20260709 13:08:19.673722 22301 leader_election.cc:304] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:19.685969 23075 raft_consensus.cc:2749] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:19.687577 23073 consensus_queue.cc:237] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:19.679316 22375 leader_election.cc:336] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:19.689738 22375 leader_election.cc:336] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.690177 22375 leader_election.cc:304] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:19.666005 22886 raft_consensus.cc:515] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.691758 23079 raft_consensus.cc:2749] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:19.697453 23084 raft_consensus.cc:493] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.698402 23084 raft_consensus.cc:515] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:19.700731 23084 leader_election.cc:290] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:19.700842 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "878ae1fea9154e7088fdba8930c5eb23" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:19.701676 22257 raft_consensus.cc:3060] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.702750 23064 raft_consensus.cc:515] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.706161 23086 raft_consensus.cc:493] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.706264 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffdd1d227bbe48c8a46c4fefd02eab0b" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:19.707453 23065 raft_consensus.cc:515] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.721058 22407 raft_consensus.cc:3060] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:19.720201 22375 leader_election.cc:336] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.722738 23065 leader_election.cc:290] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
W20260709 13:08:19.709322 22301 leader_election.cc:336] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.724493 22301 leader_election.cc:304] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:19.724574 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f204603e14c438bb549d1116eaac788" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:19.725239 22410 raft_consensus.cc:3060] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:19.725569 22301 leader_election.cc:336] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.727576 23065 raft_consensus.cc:493] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.728458 23065 raft_consensus.cc:515] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:19.729857 22407 raft_consensus.cc:2468] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 2.
I20260709 13:08:19.715003 23076 raft_consensus.cc:493] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.714498 23064 leader_election.cc:290] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.672617 23083 raft_consensus.cc:515] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:19.718781 23087 raft_consensus.cc:493] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.711808 22886 leader_election.cc:290] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.730451 23073 raft_consensus.cc:2749] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:19.730742 23065 leader_election.cc:290] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:19.793789 23087 raft_consensus.cc:515] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.776505 23084 raft_consensus.cc:493] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.768005 23079 raft_consensus.cc:493] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.795562 23079 raft_consensus.cc:515] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.795465 23086 raft_consensus.cc:515] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:19.740238 23081 raft_consensus.cc:493] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.797122 23081 raft_consensus.cc:515] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.798206 23020 raft_consensus.cc:493] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.798774 23020 raft_consensus.cc:515] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:19.799635 22226 leader_election.cc:304] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.800741 23020 leader_election.cc:290] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.802353 23081 leader_election.cc:290] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.803589 23076 raft_consensus.cc:515] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:19.804426 22262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.802645 23086 leader_election.cc:290] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:19.805142 22262 raft_consensus.cc:2468] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:19.807207 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:19.807548 23083 leader_election.cc:290] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:19.807883 22333 raft_consensus.cc:2468] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:19.769732 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:19.809017 22329 raft_consensus.cc:2468] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:19.810215 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7f96ef51941e39bc97c0b769c7770" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:19.811231 22407 raft_consensus.cc:2468] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:19.810911 22083 catalog_manager.cc:5697] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:19.812582 22226 leader_election.cc:304] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.813810 23065 raft_consensus.cc:2804] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.814231 23065 raft_consensus.cc:493] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.814484 22374 leader_election.cc:304] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.783179 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:19.815711 22331 raft_consensus.cc:2468] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:19.816648 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:19.820133 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01099e972e9b4d08a64552be3fd52276" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:19.821921 23086 raft_consensus.cc:2804] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:19.805918 23076 leader_election.cc:290] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.821077 22329 raft_consensus.cc:2468] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:19.832796 23087 leader_election.cc:290] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.824013 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8f7472c97049ca98e597c7d4b4eb1c" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:19.838960 22886 raft_consensus.cc:2804] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.839341 22405 raft_consensus.cc:2468] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:19.839617 22886 raft_consensus.cc:493] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.840171 22886 raft_consensus.cc:3060] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.841068 23078 raft_consensus.cc:493] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.841621 23078 raft_consensus.cc:515] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:19.848460 23076 raft_consensus.cc:493] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.849151 23076 raft_consensus.cc:515] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:19.851624 23078 leader_election.cc:290] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
W20260709 13:08:19.853974 22301 leader_election.cc:336] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.854635 22301 leader_election.cc:304] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:19.851831 23076 leader_election.cc:290] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
W20260709 13:08:19.856494 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 49166.3us
I20260709 13:08:19.858115 23076 raft_consensus.cc:2749] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:19.859759 22301 leader_election.cc:336] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.861061 22301 leader_election.cc:304] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:19.863971 23076 raft_consensus.cc:2749] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:19.863965 22375 leader_election.cc:304] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:19.824208 23086 raft_consensus.cc:697] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:19.866137 23086 consensus_queue.cc:237] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.867390 22300 leader_election.cc:304] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:19.832602 22257 raft_consensus.cc:2468] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 2.
I20260709 13:08:19.869427 23076 raft_consensus.cc:2804] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.869899 22374 leader_election.cc:304] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.736852 22227 leader_election.cc:304] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.871232 23064 raft_consensus.cc:2804] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.877667 23078 raft_consensus.cc:2804] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:19.877017 23079 leader_election.cc:290] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.878130 23064 raft_consensus.cc:493] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.878384 23078 raft_consensus.cc:697] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:19.878791 23064 raft_consensus.cc:3060] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.876557 23084 raft_consensus.cc:515] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.879849 22259 raft_consensus.cc:2468] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 1.
I20260709 13:08:19.873270 23068 raft_consensus.cc:2804] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.880475 23068 raft_consensus.cc:493] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.880870 23068 raft_consensus.cc:3060] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.814647 23065 raft_consensus.cc:3060] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.883495 22405 raft_consensus.cc:1275] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer bb26db1bd5a64429848b5d63b157c6f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:19.884435 23084 leader_election.cc:290] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.885390 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:19.886099 22407 raft_consensus.cc:2468] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:19.888039 22226 leader_election.cc:304] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.752650 22410 raft_consensus.cc:2468] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 2.
I20260709 13:08:19.889415 23083 raft_consensus.cc:2804] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.889917 23083 raft_consensus.cc:493] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.879513 23078 consensus_queue.cc:237] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.890376 23083 raft_consensus.cc:3060] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.890472 22226 leader_election.cc:304] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.891515 23087 consensus_queue.cc:1048] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:19.893021 23087 raft_consensus.cc:2804] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:19.893500 23087 raft_consensus.cc:697] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:19.889535 22374 leader_election.cc:304] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.896281 23020 raft_consensus.cc:2804] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.896714 23020 raft_consensus.cc:493] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.897068 23020 raft_consensus.cc:3060] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.902710 22410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96821e05d1924572a73e450bc7dcb265" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" is_pre_election: true
I20260709 13:08:19.903430 22410 raft_consensus.cc:2468] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 1.
I20260709 13:08:19.904896 23087 consensus_queue.cc:237] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
W20260709 13:08:19.908880 22375 leader_election.cc:336] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.912194 22300 leader_election.cc:304] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:19.913374 23073 raft_consensus.cc:2804] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.913805 23073 raft_consensus.cc:493] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.914317 23073 raft_consensus.cc:3060] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:19.871085 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 134354us
I20260709 13:08:19.869885 23076 raft_consensus.cc:493] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.922478 23076 raft_consensus.cc:3060] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.923260 23083 raft_consensus.cc:515] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
W20260709 13:08:19.926124 22301 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.12.109.126:37329, user_credentials={real_user=slave}} blocked reactor thread for 50815.1us
W20260709 13:08:19.927372 22301 leader_election.cc:336] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.931237 22405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:19.874457 22886 raft_consensus.cc:515] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
W20260709 13:08:19.934499 22301 leader_election.cc:336] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:19.925874 23065 raft_consensus.cc:515] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.937844 22886 leader_election.cc:290] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:19.925820 23083 leader_election.cc:290] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.938931 22257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:19.939707 22257 raft_consensus.cc:3060] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.937615 23068 raft_consensus.cc:515] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:19.947428 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4bb1a452944c59aa3dc6717c52be6a" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:19.948166 22331 raft_consensus.cc:2468] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:19.948897 22227 leader_election.cc:304] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.951025 22227 leader_election.cc:304] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.952567 22257 raft_consensus.cc:2468] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 2.
I20260709 13:08:19.953819 22374 leader_election.cc:304] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.955615 22227 leader_election.cc:304] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.960983 23090 raft_consensus.cc:2804] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.961077 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:19.961671 23090 raft_consensus.cc:493] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.962008 22329 raft_consensus.cc:2468] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 1.
I20260709 13:08:19.962680 23092 raft_consensus.cc:2804] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.963094 23092 raft_consensus.cc:493] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.963511 23092 raft_consensus.cc:3060] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.963379 22259 raft_consensus.cc:1275] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Refusing update from remote peer 67fc9e7fc4b34621ae1cd3e1621b89dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:19.963356 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7f96ef51941e39bc97c0b769c7770" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:19.959478 23084 raft_consensus.cc:2804] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:19.964767 23084 raft_consensus.cc:493] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:19.964814 22407 raft_consensus.cc:3060] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.965297 23084 raft_consensus.cc:3060] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.966301 23081 consensus_queue.cc:1048] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:19.962153 23090 raft_consensus.cc:3060] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.925874 23073 raft_consensus.cc:515] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:19.977151 22407 raft_consensus.cc:2468] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 2.
I20260709 13:08:19.959276 23079 raft_consensus.cc:2804] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:19.978257 23079 raft_consensus.cc:697] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:19.961933 23065 leader_election.cc:290] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.978366 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96821e05d1924572a73e450bc7dcb265" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8" is_pre_election: true
I20260709 13:08:19.979316 22329 raft_consensus.cc:2393] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 2: Already voted for candidate 93815010a884438c86fc6479359b69e8 in this term.
I20260709 13:08:19.979224 23079 consensus_queue.cc:237] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:19.979838 23073 leader_election.cc:290] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.981456 23068 leader_election.cc:290] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:19.982115 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96821e05d1924572a73e450bc7dcb265" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:19.982770 22404 raft_consensus.cc:3060] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.990792 22227 leader_election.cc:304] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:20.002218 23065 raft_consensus.cc:2804] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:20.002691 23065 raft_consensus.cc:493] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:20.003067 23065 raft_consensus.cc:3060] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.989918 23076 raft_consensus.cc:515] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:20.008267 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8f7472c97049ca98e597c7d4b4eb1c" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:20.009186 22407 raft_consensus.cc:3060] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.009502 22083 catalog_manager.cc:5697] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 2 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:20.019560 22227 leader_election.cc:304] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:19.992177 22226 leader_election.cc:304] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:20.005757 23076 leader_election.cc:290] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
W20260709 13:08:20.023092 22375 leader_election.cc:336] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.027614 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:20.028450 22331 raft_consensus.cc:3060] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:20.030160 22375 leader_election.cc:336] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.031328 22375 leader_election.cc:336] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.008159 23020 raft_consensus.cc:515] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:20.042846 23084 raft_consensus.cc:515] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:20.044412 22301 leader_election.cc:336] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.045187 23084 leader_election.cc:290] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
W20260709 13:08:20.048789 22301 leader_election.cc:336] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.049222 23086 raft_consensus.cc:2804] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:20.049680 23086 raft_consensus.cc:493] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:20.050150 23086 raft_consensus.cc:3060] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.056797 22404 raft_consensus.cc:2468] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 2.
I20260709 13:08:20.059047 22300 leader_election.cc:304] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607
I20260709 13:08:20.060124 23064 raft_consensus.cc:515] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:20.060567 23090 raft_consensus.cc:515] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:20.061221 23075 raft_consensus.cc:2804] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.061810 23075 raft_consensus.cc:697] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 2 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
W20260709 13:08:20.062639 22375 leader_election.cc:336] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.063460 22375 leader_election.cc:304] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
W20260709 13:08:20.064347 22375 leader_election.cc:336] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.065045 22375 consensus_peers.cc:597] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.065570 22375 leader_election.cc:336] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.065896 22405 raft_consensus.cc:2393] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bb26db1bd5a64429848b5d63b157c6f0 in current term 2: Already voted for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in this term.
I20260709 13:08:20.066314 23081 raft_consensus.cc:2749] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:20.062728 23075 consensus_queue.cc:237] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:20.068936 23020 leader_election.cc:290] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
W20260709 13:08:20.072918 22375 consensus_peers.cc:597] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.075469 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:20.076305 22259 raft_consensus.cc:2393] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in current term 2: Already voted for candidate bb26db1bd5a64429848b5d63b157c6f0 in this term.
I20260709 13:08:20.077134 23064 leader_election.cc:290] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:20.072495 23087 raft_consensus.cc:2804] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.080281 23087 raft_consensus.cc:697] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:20.081418 23087 consensus_queue.cc:237] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:20.049844 23065 raft_consensus.cc:515] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:20.085959 23092 raft_consensus.cc:515] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:20.092326 22262 raft_consensus.cc:1275] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Refusing update from remote peer 67fc9e7fc4b34621ae1cd3e1621b89dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:20.093640 22407 raft_consensus.cc:2468] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 2.
I20260709 13:08:20.094012 23078 consensus_queue.cc:1048] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:20.096735 22300 leader_election.cc:304] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607
I20260709 13:08:20.097958 23061 raft_consensus.cc:2804] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.098537 23061 raft_consensus.cc:697] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 2 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
I20260709 13:08:20.099723 23090 leader_election.cc:290] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:20.099494 23061 consensus_queue.cc:237] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
W20260709 13:08:20.087589 22226 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.68:41261, user_credentials={real_user=slave}} blocked reactor thread for 98694us
W20260709 13:08:20.105786 22375 leader_election.cc:336] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.095016 22331 raft_consensus.cc:2468] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 2.
I20260709 13:08:20.100502 23086 raft_consensus.cc:515] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:20.106488 22375 leader_election.cc:304] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:20.107288 22083 catalog_manager.cc:5697] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 2 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:20.109220 23079 raft_consensus.cc:2749] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:20.109932 22083 catalog_manager.cc:5697] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 2 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:20.114406 22375 leader_election.cc:304] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 652d70a0b765447792ab9b4383712607
I20260709 13:08:20.117988 23086 leader_election.cc:290] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
W20260709 13:08:20.122215 22301 consensus_peers.cc:597] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.125022 23065 leader_election.cc:290] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:20.144182 23092 leader_election.cc:290] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:20.143841 22082 catalog_manager.cc:5697] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:20.162439 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 142935us
W20260709 13:08:20.165426 22227 leader_election.cc:336] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.141198 23020 raft_consensus.cc:2804] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.198311 22262 raft_consensus.cc:1275] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Refusing update from remote peer 93815010a884438c86fc6479359b69e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:20.167768 22083 catalog_manager.cc:5697] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:20.170349 23020 raft_consensus.cc:697] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:20.200835 23020 consensus_queue.cc:237] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:20.188581 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4bb1a452944c59aa3dc6717c52be6a" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:20.202713 22330 raft_consensus.cc:3060] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.201818 23061 consensus_queue.cc:1048] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:08:20.192646 22227 leader_election.cc:336] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.207013 22227 leader_election.cc:336] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.178542 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:20.208477 22329 raft_consensus.cc:3060] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.181988 22334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3808206385a0458abc04050aeab1cea7" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:20.210698 22334 raft_consensus.cc:3060] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.189879 22335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01099e972e9b4d08a64552be3fd52276" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:20.212494 22335 raft_consensus.cc:3060] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.186525 22331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:20.214613 22331 raft_consensus.cc:3060] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.185932 22333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "bb26db1bd5a64429848b5d63b157c6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:20.223012 22333 raft_consensus.cc:3060] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:20.246490 22227 leader_election.cc:336] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.247639 22227 leader_election.cc:336] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.248363 22227 leader_election.cc:336] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.248777 22227 leader_election.cc:304] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:20.253084 23106 raft_consensus.cc:2749] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:20.258543 22330 raft_consensus.cc:2468] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 2.
I20260709 13:08:20.259420 22083 catalog_manager.cc:5697] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:20.251928 22227 leader_election.cc:336] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.262835 22227 leader_election.cc:336] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.265766 22227 leader_election.cc:336] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.270684 22227 leader_election.cc:336] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.271193 22227 leader_election.cc:304] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:20.273108 23106 raft_consensus.cc:2749] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:20.274188 22227 leader_election.cc:336] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.274618 22227 leader_election.cc:304] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:20.276409 22329 raft_consensus.cc:2468] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 2.
I20260709 13:08:20.276816 23106 raft_consensus.cc:2749] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:20.277977 22227 leader_election.cc:336] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.278398 22227 leader_election.cc:304] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
I20260709 13:08:20.269788 22333 raft_consensus.cc:2468] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 2.
I20260709 13:08:20.290091 22085 catalog_manager.cc:5697] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 2 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:20.297470 22227 leader_election.cc:336] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.298107 22227 leader_election.cc:304] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:20.298694 22334 raft_consensus.cc:2468] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 2.
I20260709 13:08:20.389210 23087 raft_consensus.cc:2749] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:20.299247 23106 raft_consensus.cc:2749] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:20.389770 22227 leader_election.cc:336] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.397866 22227 leader_election.cc:304] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc
W20260709 13:08:20.398931 22227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:46399, user_credentials={real_user=slave}} blocked reactor thread for 96029.7us
W20260709 13:08:20.399483 22227 leader_election.cc:336] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.400297 22227 leader_election.cc:336] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.401041 22227 leader_election.cc:336] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.401769 22227 leader_election.cc:336] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.402971 23092 raft_consensus.cc:2749] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:20.406857 22227 leader_election.cc:336] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.410483 22227 leader_election.cc:336] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.411301 22227 leader_election.cc:336] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:20.417089 22227 leader_election.cc:304] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:20.417897 23092 raft_consensus.cc:2804] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.418452 23092 raft_consensus.cc:697] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:20.419329 23092 consensus_queue.cc:237] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:20.423859 22227 leader_election.cc:304] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:20.424896 22227 leader_election.cc:304] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:20.425617 23087 raft_consensus.cc:2804] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.425758 22227 leader_election.cc:304] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:20.425915 22407 raft_consensus.cc:1275] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer bb26db1bd5a64429848b5d63b157c6f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:20.426615 23087 raft_consensus.cc:697] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:20.427875 23086 consensus_queue.cc:1048] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:20.429788 23086 raft_consensus.cc:2804] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.430290 23086 raft_consensus.cc:697] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:20.431103 23086 consensus_queue.cc:237] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:20.433717 23087 consensus_queue.cc:237] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:20.437992 22331 raft_consensus.cc:2468] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 2.
I20260709 13:08:20.442646 22335 raft_consensus.cc:2468] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb26db1bd5a64429848b5d63b157c6f0 in term 2.
I20260709 13:08:20.444427 22262 raft_consensus.cc:1275] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Refusing update from remote peer 67fc9e7fc4b34621ae1cd3e1621b89dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:20.452416 23078 consensus_queue.cc:1048] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:20.454218 22407 raft_consensus.cc:1275] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer bb26db1bd5a64429848b5d63b157c6f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:20.455551 23100 consensus_queue.cc:1048] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:20.425400 23106 raft_consensus.cc:2804] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.460816 23106 raft_consensus.cc:697] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
W20260709 13:08:20.461416 22227 leader_election.cc:336] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.462708 22227 leader_election.cc:336] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.463400 22227 leader_election.cc:336] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.464129 22227 leader_election.cc:336] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.464633 22227 leader_election.cc:336] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.465353 22227 leader_election.cc:336] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:20.465848 22227 consensus_peers.cc:597] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.475958 22227 leader_election.cc:304] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607
I20260709 13:08:20.478219 22227 leader_election.cc:304] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607
I20260709 13:08:20.477242 23086 raft_consensus.cc:2804] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.461817 23106 consensus_queue.cc:237] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:20.481736 23100 raft_consensus.cc:2804] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:08:20.491952 22227 consensus_peers.cc:597] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.494926 12725 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 22208)
I20260709 13:08:20.492856 23086 raft_consensus.cc:697] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:20.501017 23086 consensus_queue.cc:237] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
W20260709 13:08:20.517884 22227 consensus_peers.cc:597] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.518559 23100 raft_consensus.cc:697] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Leader. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:20.519670 23100 consensus_queue.cc:237] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:20.564328 22085 catalog_manager.cc:5697] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } }
I20260709 13:08:20.575142 22257 raft_consensus.cc:1275] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Refusing update from remote peer 67fc9e7fc4b34621ae1cd3e1621b89dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:20.577986 23081 consensus_queue.cc:1048] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:08:20.569715 22375 consensus_peers.cc:597] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.586275 22227 consensus_peers.cc:597] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.595070 22407 raft_consensus.cc:1275] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer bb26db1bd5a64429848b5d63b157c6f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:20.598348 23106 consensus_queue.cc:1048] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:08:20.604689 22375 consensus_peers.cc:597] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.605317 22404 raft_consensus.cc:1275] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer 93815010a884438c86fc6479359b69e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:20.608330 23073 consensus_queue.cc:1048] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:08:20.622367 22301 consensus_peers.cc:597] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.663357 22301 consensus_peers.cc:597] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.677270 22375 consensus_peers.cc:597] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:20.690367 22085 catalog_manager.cc:5697] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:20.692646 22404 raft_consensus.cc:1275] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer 93815010a884438c86fc6479359b69e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:20.694164 23073 consensus_queue.cc:1048] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:20.693465 22085 catalog_manager.cc:5697] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:20.695547 22085 catalog_manager.cc:5697] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:20.697367 22085 catalog_manager.cc:5697] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:20.699127 22085 catalog_manager.cc:5697] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:20.703090 22301 consensus_peers.cc:597] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:20.707094 22329 raft_consensus.cc:1275] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Refusing update from remote peer 67fc9e7fc4b34621ae1cd3e1621b89dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:20.709046 23113 consensus_queue.cc:1048] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:08:20.743049 22375 consensus_peers.cc:597] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.748370 22082 catalog_manager.cc:5697] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:20.756481 22082 catalog_manager.cc:5697] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 reported cstate change: term changed from 1 to 2, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bb26db1bd5a64429848b5d63b157c6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:20.821128 22375 consensus_peers.cc:597] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:20.827292 22375 consensus_peers.cc:597] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:20.860306 22375 consensus_peers.cc:597] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:20.880676 22329 raft_consensus.cc:1275] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Refusing update from remote peer bb26db1bd5a64429848b5d63b157c6f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:20.882258 23086 consensus_queue.cc:1048] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:20.885818 22329 raft_consensus.cc:1275] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Refusing update from remote peer bb26db1bd5a64429848b5d63b157c6f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:08:20.886896 22301 consensus_peers.cc:597] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:20.888245 23086 consensus_queue.cc:1048] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:08:20.907316 22227 consensus_peers.cc:597] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.913872 22227 consensus_peers.cc:597] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.915834 22335 raft_consensus.cc:1275] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Refusing update from remote peer bb26db1bd5a64429848b5d63b157c6f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:20.927471 23106 consensus_queue.cc:1048] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:08:20.936812 22301 consensus_peers.cc:597] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:20.941687 22375 consensus_peers.cc:597] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:20.957512 22227 consensus_peers.cc:597] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.967456 22301 consensus_peers.cc:597] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:20.969960 22227 consensus_peers.cc:597] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.970530 22375 consensus_peers.cc:597] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.024253 22335 raft_consensus.cc:1275] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Refusing update from remote peer bb26db1bd5a64429848b5d63b157c6f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:21.026503 23065 consensus_queue.cc:1048] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:08:21.030453 22375 consensus_peers.cc:597] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.062661 22227 consensus_peers.cc:597] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:21.062805 22329 raft_consensus.cc:1275] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Refusing update from remote peer bb26db1bd5a64429848b5d63b157c6f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:21.065829 23065 consensus_queue.cc:1048] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:08:21.092697 22227 consensus_peers.cc:597] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:21.120198 22335 raft_consensus.cc:1275] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Refusing update from remote peer bb26db1bd5a64429848b5d63b157c6f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:08:21.120980 22375 consensus_peers.cc:597] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.124348 23084 consensus_queue.cc:1048] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:08:21.126581 22375 consensus_peers.cc:597] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.158820 22375 consensus_peers.cc:597] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.176087 22375 consensus_peers.cc:597] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.187362 22375 consensus_peers.cc:597] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.209937 22301 consensus_peers.cc:597] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.237038 22375 consensus_peers.cc:597] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.304925 22375 consensus_peers.cc:597] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.495414 12725 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 22208)
W20260709 13:08:21.644981 22227 consensus_peers.cc:597] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.823662 22227 consensus_peers.cc:597] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.839011 22227 consensus_peers.cc:597] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.998018 22227 consensus_peers.cc:597] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.033106 22227 consensus_peers.cc:597] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.041741 22300 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.68:41261, user_credentials={real_user=slave}} blocked reactor thread for 51572.9us
W20260709 13:08:22.079644 22227 consensus_peers.cc:597] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.250015 22227 consensus_peers.cc:597] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:22.339597 23078 raft_consensus.cc:493] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.340770 23078 raft_consensus.cc:515] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:22.345809 23078 leader_election.cc:290] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:22.351979 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0" is_pre_election: true
I20260709 13:08:22.353117 22259 raft_consensus.cc:2468] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 2.
I20260709 13:08:22.357455 22374 leader_election.cc:304] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:22.358680 23079 raft_consensus.cc:2804] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:22.359071 23079 raft_consensus.cc:493] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.359512 23079 raft_consensus.cc:3060] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:22.364835 22375 leader_election.cc:336] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:22.374289 23079 raft_consensus.cc:515] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:22.377629 23079 leader_election.cc:290] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 election: Requested vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:22.377821 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:22.378614 22259 raft_consensus.cc:3060] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:22.385030 22375 leader_election.cc:336] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:22.387811 22259 raft_consensus.cc:2468] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fc9e7fc4b34621ae1cd3e1621b89dc in term 3.
I20260709 13:08:22.389317 22374 leader_election.cc:304] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, bb26db1bd5a64429848b5d63b157c6f0; no voters: 652d70a0b765447792ab9b4383712607
I20260709 13:08:22.390618 23079 raft_consensus.cc:2804] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:22.391908 23079 raft_consensus.cc:697] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 LEADER]: Becoming Leader. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:22.393292 23079 consensus_queue.cc:237] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:22.413676 22085 catalog_manager.cc:5697] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc reported cstate change: term changed from 1 to 3, leader changed from 652d70a0b765447792ab9b4383712607 (127.12.109.65) to 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68). New cstate: current_term: 3 leader_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:22.496183 12725 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 22208)
W20260709 13:08:22.527621 22227 consensus_peers.cc:597] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.536404 22375 consensus_peers.cc:597] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.572237 22375 consensus_peers.cc:597] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.584877 22301 consensus_peers.cc:597] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.746392 22375 consensus_peers.cc:597] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.760244 22375 consensus_peers.cc:597] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:22.805603 22257 raft_consensus.cc:1275] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Refusing update from remote peer 67fc9e7fc4b34621ae1cd3e1621b89dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:08:22.807976 22301 consensus_peers.cc:597] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:22.807834 23138 consensus_queue.cc:1048] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:08:22.829818 22375 consensus_peers.cc:597] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:22.930647 22227 consensus_peers.cc:597] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.936434 22227 consensus_peers.cc:597] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.007774 22375 consensus_peers.cc:597] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.045210 22227 consensus_peers.cc:597] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.094777 22227 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111) [suppressed 401 similar messages]
W20260709 13:08:23.138691 22301 consensus_peers.cc:597] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.171411 22227 consensus_peers.cc:597] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.198218 22375 consensus_peers.cc:597] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.261698 22301 consensus_peers.cc:597] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.281913 22301 consensus_peers.cc:597] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.282577 22375 consensus_peers.cc:597] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.289407 22375 consensus_peers.cc:597] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.344817 22301 consensus_peers.cc:597] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.348851 22227 consensus_peers.cc:597] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.367578 22227 consensus_peers.cc:597] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.392321 22375 consensus_peers.cc:597] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.435123 22375 consensus_peers.cc:597] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.460654 22227 consensus_peers.cc:597] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.472697 22301 consensus_peers.cc:597] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.481561 22227 consensus_peers.cc:597] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.480654 22375 consensus_peers.cc:597] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.482475 22375 consensus_peers.cc:597] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.487445 22375 consensus_peers.cc:597] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.494500 22227 consensus_peers.cc:597] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.496709 12725 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 22208)
I20260709 13:08:23.557598 12725 tablet_server.cc:196] TabletServer@127.12.109.65:0 shutdown complete.
W20260709 13:08:23.620023 22375 consensus_peers.cc:597] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.697494 22375 consensus_peers.cc:597] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:23.700645 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:23.708735 23149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:23.709015 23150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:23.710325 23152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:23.711773 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:23.713593 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:23.713876 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:23.714103 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602503714086 us; error 0 us; skew 500 ppm
I20260709 13:08:23.714700 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:23.717959 12725 webserver.cc:533] Webserver started at http://127.12.109.65:41687/ using document root <none> and password file <none>
I20260709 13:08:23.718681 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:23.718878 12725 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:08:23.723404 12725 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.001s	sys 0.004s
I20260709 13:08:23.733610 23157 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:23.734686 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:08:23.735044 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-0-root
uuid: "652d70a0b765447792ab9b4383712607"
format_stamp: "Formatted at 2026-07-09 13:08:02 on dist-test-slave-zhdc"
I20260709 13:08:23.735452 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602188851028-12725-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:23.748104 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:23.750080 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:23.754225 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
W20260709 13:08:23.783924 22375 consensus_peers.cc:597] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.816762 22374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 53768us
I20260709 13:08:23.826184 23164 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:08:23.828189 22375 consensus_peers.cc:597] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.828833 22375 consensus_peers.cc:597] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.927054 22375 consensus_peers.cc:597] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.946802 22301 consensus_peers.cc:597] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:24.133296 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:08:24.133926 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.374s	user 0.021s	sys 0.049s
I20260709 13:08:24.134399 12725 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:08:24.140693 23164 tablet_bootstrap.cc:492] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.156085 23164 tablet_bootstrap.cc:492] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.157090 23164 tablet_bootstrap.cc:492] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.157764 23164 ts_tablet_manager.cc:1403] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.002s
I20260709 13:08:24.160041 23164 raft_consensus.cc:359] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.160570 23164 raft_consensus.cc:740] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.161253 23164 consensus_queue.cc:260] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.163499 23164 ts_tablet_manager.cc:1434] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:08:24.164350 23164 tablet_bootstrap.cc:492] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
W20260709 13:08:24.170678 22227 consensus_peers.cc:597] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:24.180856 23164 tablet_bootstrap.cc:492] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.181938 23164 tablet_bootstrap.cc:492] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.182695 23164 ts_tablet_manager.cc:1403] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:08:24.185578 23164 raft_consensus.cc:359] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.186498 23164 raft_consensus.cc:740] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.187536 23164 consensus_queue.cc:260] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.190346 23164 ts_tablet_manager.cc:1434] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:08:24.191090 23164 tablet_bootstrap.cc:492] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
W20260709 13:08:24.206923 22227 consensus_peers.cc:597] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:24.207980 23164 tablet_bootstrap.cc:492] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.208906 23164 tablet_bootstrap.cc:492] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.209466 23164 ts_tablet_manager.cc:1403] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.000s
I20260709 13:08:24.212239 23164 raft_consensus.cc:359] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.212945 23164 raft_consensus.cc:740] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.213632 23164 consensus_queue.cc:260] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.215528 23164 ts_tablet_manager.cc:1434] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 13:08:24.216308 23164 tablet_bootstrap.cc:492] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.257346 23164 tablet_bootstrap.cc:492] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.258404 23164 tablet_bootstrap.cc:492] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.259063 23164 ts_tablet_manager.cc:1403] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.043s	user 0.030s	sys 0.011s
I20260709 13:08:24.261770 23164 raft_consensus.cc:359] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.262503 23164 raft_consensus.cc:740] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.263343 23164 consensus_queue.cc:260] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.267164 23164 ts_tablet_manager.cc:1434] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:08:24.267976 23164 tablet_bootstrap.cc:492] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.289506 23164 tablet_bootstrap.cc:492] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.290524 23164 tablet_bootstrap.cc:492] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.291209 23164 ts_tablet_manager.cc:1403] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.004s
W20260709 13:08:24.293620 22227 consensus_peers.cc:597] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:24.294040 23164 raft_consensus.cc:359] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.294977 23164 raft_consensus.cc:740] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.295857 23164 consensus_queue.cc:260] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.297456 23164 ts_tablet_manager.cc:1434] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:08:24.298282 23164 tablet_bootstrap.cc:492] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.319031 23164 tablet_bootstrap.cc:492] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.320171 23164 tablet_bootstrap.cc:492] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.320957 23164 ts_tablet_manager.cc:1403] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.003s
I20260709 13:08:24.323818 23164 raft_consensus.cc:359] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.324546 23164 raft_consensus.cc:740] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.325277 23164 consensus_queue.cc:260] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.327349 23164 ts_tablet_manager.cc:1434] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:08:24.329262 23164 tablet_bootstrap.cc:492] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.347100 23164 tablet_bootstrap.cc:492] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.348232 23164 tablet_bootstrap.cc:492] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.349007 23164 ts_tablet_manager.cc:1403] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 13:08:24.351855 23164 raft_consensus.cc:359] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.352598 23164 raft_consensus.cc:740] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.353340 23164 consensus_queue.cc:260] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.355471 23164 ts_tablet_manager.cc:1434] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:08:24.356359 23164 tablet_bootstrap.cc:492] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
W20260709 13:08:24.371549 22227 consensus_peers.cc:597] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:24.392496 23164 tablet_bootstrap.cc:492] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.393663 23164 tablet_bootstrap.cc:492] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.394407 23164 ts_tablet_manager.cc:1403] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.016s
I20260709 13:08:24.397293 23164 raft_consensus.cc:359] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.398007 23164 raft_consensus.cc:740] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.398765 23164 consensus_queue.cc:260] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.400830 23164 ts_tablet_manager.cc:1434] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:08:24.401638 23164 tablet_bootstrap.cc:492] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.405792 12725 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:08:24.406257 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.272s	user 0.259s	sys 0.005s
I20260709 13:08:24.428292 23164 tablet_bootstrap.cc:492] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.429531 23164 tablet_bootstrap.cc:492] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.431456 23164 ts_tablet_manager.cc:1403] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.003s
I20260709 13:08:24.434633 23164 raft_consensus.cc:359] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.435542 23164 raft_consensus.cc:740] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.436390 23164 consensus_queue.cc:260] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.439064 23164 ts_tablet_manager.cc:1434] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:08:24.440085 23164 tablet_bootstrap.cc:492] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.462471 23164 tablet_bootstrap.cc:492] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.463810 23164 tablet_bootstrap.cc:492] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.464756 23164 ts_tablet_manager.cc:1403] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.025s	user 0.022s	sys 0.000s
I20260709 13:08:24.468405 23164 raft_consensus.cc:359] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.469436 23164 raft_consensus.cc:740] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.470558 23164 consensus_queue.cc:260] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.473351 23164 ts_tablet_manager.cc:1434] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:08:24.474495 23164 tablet_bootstrap.cc:492] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.494613 23164 tablet_bootstrap.cc:492] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.495822 23164 tablet_bootstrap.cc:492] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.496691 23164 ts_tablet_manager.cc:1403] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.003s
I20260709 13:08:24.499495 23164 raft_consensus.cc:359] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.500176 23164 raft_consensus.cc:740] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.500876 23164 consensus_queue.cc:260] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.502009 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.65:46399
I20260709 13:08:24.502177 23234 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.65:46399 every 8 connection(s)
I20260709 13:08:24.502949 23164 ts_tablet_manager.cc:1434] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:08:24.504227 23164 tablet_bootstrap.cc:492] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.541134 23164 tablet_bootstrap.cc:492] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.542339 23164 tablet_bootstrap.cc:492] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.543200 23164 ts_tablet_manager.cc:1403] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.039s	user 0.017s	sys 0.014s
I20260709 13:08:24.546557 23164 raft_consensus.cc:359] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.547333 23164 raft_consensus.cc:740] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.548272 23164 consensus_queue.cc:260] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.550737 23164 ts_tablet_manager.cc:1434] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:08:24.551616 23164 tablet_bootstrap.cc:492] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.574911 23236 heartbeater.cc:344] Connected to a master server at 127.12.109.126:37329
I20260709 13:08:24.575621 23236 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:24.576786 23236 heartbeater.cc:507] Master 127.12.109.126:37329 requested a full tablet report, sending...
I20260709 13:08:24.581743 23164 tablet_bootstrap.cc:492] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.583271 23164 tablet_bootstrap.cc:492] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.584007 23164 ts_tablet_manager.cc:1403] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.016s
I20260709 13:08:24.587615 23205 raft_consensus.cc:3060] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:24.594806 23164 raft_consensus.cc:359] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.595703 23164 raft_consensus.cc:740] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.596386 23164 consensus_queue.cc:260] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.598853 23164 ts_tablet_manager.cc:1434] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:08:24.599740 23164 tablet_bootstrap.cc:492] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.631157 23164 tablet_bootstrap.cc:492] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.632900 23164 tablet_bootstrap.cc:492] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.633693 23164 ts_tablet_manager.cc:1403] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.006s
I20260709 13:08:24.636246 23164 raft_consensus.cc:359] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.636806 23164 raft_consensus.cc:740] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.637603 23164 consensus_queue.cc:260] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.638675 22085 ts_manager.cc:194] Re-registered known tserver with Master: 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:24.639827 23164 ts_tablet_manager.cc:1434] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:08:24.640793 23164 tablet_bootstrap.cc:492] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.653940 23205 raft_consensus.cc:3060] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:24.663133 22227 consensus_peers.cc:597] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:24.671218 23164 tablet_bootstrap.cc:492] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.672406 23164 tablet_bootstrap.cc:492] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.673190 23164 ts_tablet_manager.cc:1403] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.008s
I20260709 13:08:24.676668 23164 raft_consensus.cc:359] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.677423 23164 raft_consensus.cc:740] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.678234 23164 consensus_queue.cc:260] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.680646 23164 ts_tablet_manager.cc:1434] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:08:24.681483 23164 tablet_bootstrap.cc:492] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.696411 22085 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36860
I20260709 13:08:24.697706 23164 tablet_bootstrap.cc:492] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.698814 23164 tablet_bootstrap.cc:492] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.699442 23164 ts_tablet_manager.cc:1403] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:08:24.701629 23236 heartbeater.cc:499] Master 127.12.109.126:37329 was elected leader, sending a full tablet report...
I20260709 13:08:24.701711 23164 raft_consensus.cc:359] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.702661 23164 raft_consensus.cc:740] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.703418 23164 consensus_queue.cc:260] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.705267 23164 ts_tablet_manager.cc:1434] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:08:24.706061 23164 tablet_bootstrap.cc:492] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.721853 23164 tablet_bootstrap.cc:492] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.722970 23164 tablet_bootstrap.cc:492] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.723703 23164 ts_tablet_manager.cc:1403] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.018s	user 0.004s	sys 0.011s
I20260709 13:08:24.726267 23164 raft_consensus.cc:359] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.726930 23164 raft_consensus.cc:740] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.727571 23164 consensus_queue.cc:260] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.729699 23164 ts_tablet_manager.cc:1434] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 13:08:24.730540 23164 tablet_bootstrap.cc:492] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.745010 23164 tablet_bootstrap.cc:492] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.746102 23164 tablet_bootstrap.cc:492] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.746711 23164 ts_tablet_manager.cc:1403] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.001s
I20260709 13:08:24.749182 23164 raft_consensus.cc:359] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.749723 23164 raft_consensus.cc:740] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.750438 23164 consensus_queue.cc:260] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.752429 23164 ts_tablet_manager.cc:1434] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.003s
I20260709 13:08:24.753146 23164 tablet_bootstrap.cc:492] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.765403 23208 raft_consensus.cc:3060] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:24.778767 23164 tablet_bootstrap.cc:492] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.779989 23164 tablet_bootstrap.cc:492] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.780848 23164 ts_tablet_manager.cc:1403] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.028s	user 0.020s	sys 0.004s
I20260709 13:08:24.783952 23164 raft_consensus.cc:359] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.784737 23164 raft_consensus.cc:740] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.785573 23164 consensus_queue.cc:260] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.787492 23164 ts_tablet_manager.cc:1434] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:08:24.789091 23164 tablet_bootstrap.cc:492] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.800125 23205 raft_consensus.cc:3060] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:24.807015 23164 tablet_bootstrap.cc:492] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.808187 23164 tablet_bootstrap.cc:492] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.808970 23164 ts_tablet_manager.cc:1403] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.001s
I20260709 13:08:24.812954 23164 raft_consensus.cc:359] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.813556 23164 raft_consensus.cc:740] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.814322 23164 consensus_queue.cc:260] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:24.816756 23164 ts_tablet_manager.cc:1434] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:08:24.817422 23164 tablet_bootstrap.cc:492] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.832293 23205 raft_consensus.cc:3060] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:24.834710 23164 tablet_bootstrap.cc:492] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.835839 23164 tablet_bootstrap.cc:492] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.836597 23164 ts_tablet_manager.cc:1403] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 13:08:24.841179 23164 raft_consensus.cc:359] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.841912 23164 raft_consensus.cc:740] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.842917 23164 consensus_queue.cc:260] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.846493 23164 ts_tablet_manager.cc:1434] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:08:24.847456 23164 tablet_bootstrap.cc:492] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.854389 23205 raft_consensus.cc:3060] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:24.890144 23164 tablet_bootstrap.cc:492] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.891322 23164 tablet_bootstrap.cc:492] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.892089 23164 ts_tablet_manager.cc:1403] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.045s	user 0.028s	sys 0.007s
I20260709 13:08:24.896452 23164 raft_consensus.cc:359] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.897178 23164 raft_consensus.cc:740] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.897887 23164 consensus_queue.cc:260] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:24.900040 23164 ts_tablet_manager.cc:1434] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:08:24.900946 23164 tablet_bootstrap.cc:492] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.922639 23205 raft_consensus.cc:3060] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:24.923527 23164 tablet_bootstrap.cc:492] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:24.924705 23164 tablet_bootstrap.cc:492] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:24.925426 23164 ts_tablet_manager.cc:1403] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.008s
I20260709 13:08:24.928259 23164 raft_consensus.cc:359] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.928946 23164 raft_consensus.cc:740] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:24.929800 23164 consensus_queue.cc:260] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:24.931988 23164 ts_tablet_manager.cc:1434] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:08:24.939134 23164 tablet_bootstrap.cc:492] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:24.940759 23208 raft_consensus.cc:3060] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:24.960335 23205 raft_consensus.cc:3060] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:25.015996 23209 raft_consensus.cc:3060] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:25.016518 23164 tablet_bootstrap.cc:492] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.017731 23164 tablet_bootstrap.cc:492] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.018503 23164 ts_tablet_manager.cc:1403] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.080s	user 0.032s	sys 0.040s
I20260709 13:08:25.022215 23164 raft_consensus.cc:359] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.022977 23164 raft_consensus.cc:740] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.023752 23164 consensus_queue.cc:260] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.026118 23164 ts_tablet_manager.cc:1434] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:08:25.027050 23164 tablet_bootstrap.cc:492] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.028811 23205 raft_consensus.cc:3060] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:25.062827 23164 tablet_bootstrap.cc:492] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.064162 23164 tablet_bootstrap.cc:492] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.064957 23164 ts_tablet_manager.cc:1403] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.009s
I20260709 13:08:25.068595 23164 raft_consensus.cc:359] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.069664 23164 raft_consensus.cc:740] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.070722 23164 consensus_queue.cc:260] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.073035 23164 ts_tablet_manager.cc:1434] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:08:25.074967 23164 tablet_bootstrap.cc:492] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.099488 23164 tablet_bootstrap.cc:492] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.100800 23164 tablet_bootstrap.cc:492] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.101615 23164 ts_tablet_manager.cc:1403] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.027s	user 0.018s	sys 0.007s
I20260709 13:08:25.104401 23164 raft_consensus.cc:359] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.105173 23164 raft_consensus.cc:740] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.105968 23164 consensus_queue.cc:260] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.108196 23164 ts_tablet_manager.cc:1434] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:08:25.109097 23164 tablet_bootstrap.cc:492] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.132375 23164 tablet_bootstrap.cc:492] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.133201 23164 tablet_bootstrap.cc:492] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.133816 23164 ts_tablet_manager.cc:1403] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.007s
I20260709 13:08:25.136050 23164 raft_consensus.cc:359] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.136711 23164 raft_consensus.cc:740] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.137523 23164 consensus_queue.cc:260] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.139403 23164 ts_tablet_manager.cc:1434] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:08:25.140960 23164 tablet_bootstrap.cc:492] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
W20260709 13:08:25.154449 22301 consensus_peers.cc:597] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:25.165892 23164 tablet_bootstrap.cc:492] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.167136 23164 tablet_bootstrap.cc:492] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.167979 23164 ts_tablet_manager.cc:1403] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.003s
I20260709 13:08:25.170718 23164 raft_consensus.cc:359] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.171283 23164 raft_consensus.cc:740] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.171931 23164 consensus_queue.cc:260] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.174141 23164 ts_tablet_manager.cc:1434] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:08:25.174954 23164 tablet_bootstrap.cc:492] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.222208 23164 tablet_bootstrap.cc:492] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.223724 23164 tablet_bootstrap.cc:492] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.224591 23164 ts_tablet_manager.cc:1403] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.050s	user 0.030s	sys 0.015s
W20260709 13:08:25.225317 22375 consensus_peers.cc:597] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:25.227445 23164 raft_consensus.cc:359] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.228106 23164 raft_consensus.cc:740] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.228833 23164 consensus_queue.cc:260] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.230624 23164 ts_tablet_manager.cc:1434] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:08:25.231492 23164 tablet_bootstrap.cc:492] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.249507 23164 tablet_bootstrap.cc:492] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.250429 23164 tablet_bootstrap.cc:492] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.251029 23164 ts_tablet_manager.cc:1403] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.007s
I20260709 13:08:25.252916 23164 raft_consensus.cc:359] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.253428 23164 raft_consensus.cc:740] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.254081 23164 consensus_queue.cc:260] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.256124 23164 ts_tablet_manager.cc:1434] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:08:25.257426 23164 tablet_bootstrap.cc:492] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.277570 23164 tablet_bootstrap.cc:492] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.278647 23164 tablet_bootstrap.cc:492] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.279420 23164 ts_tablet_manager.cc:1403] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.000s
I20260709 13:08:25.282145 23164 raft_consensus.cc:359] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.282824 23164 raft_consensus.cc:740] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.283478 23164 consensus_queue.cc:260] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.285655 23164 ts_tablet_manager.cc:1434] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:08:25.286507 23164 tablet_bootstrap.cc:492] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.302378 23164 tablet_bootstrap.cc:492] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.303448 23164 tablet_bootstrap.cc:492] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.304219 23164 ts_tablet_manager.cc:1403] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 13:08:25.306867 23164 raft_consensus.cc:359] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.307544 23164 raft_consensus.cc:740] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.308185 23164 consensus_queue.cc:260] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.310374 23164 ts_tablet_manager.cc:1434] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:08:25.311502 23164 tablet_bootstrap.cc:492] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.328295 23164 tablet_bootstrap.cc:492] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.329335 23164 tablet_bootstrap.cc:492] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.330016 23164 ts_tablet_manager.cc:1403] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.000s
I20260709 13:08:25.335335 23164 raft_consensus.cc:359] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.336189 23164 raft_consensus.cc:740] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.336987 23164 consensus_queue.cc:260] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.339231 23164 ts_tablet_manager.cc:1434] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.001s
I20260709 13:08:25.340211 23164 tablet_bootstrap.cc:492] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.356536 23164 tablet_bootstrap.cc:492] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.357597 23164 tablet_bootstrap.cc:492] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.358390 23164 ts_tablet_manager.cc:1403] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 13:08:25.365622 23164 raft_consensus.cc:359] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.366415 23164 raft_consensus.cc:740] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.367132 23164 consensus_queue.cc:260] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.370147 23164 ts_tablet_manager.cc:1434] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:08:25.371004 23164 tablet_bootstrap.cc:492] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.394642 23164 tablet_bootstrap.cc:492] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.395733 23164 tablet_bootstrap.cc:492] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.396512 23164 ts_tablet_manager.cc:1403] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.003s
I20260709 13:08:25.398831 23164 raft_consensus.cc:359] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.399367 23164 raft_consensus.cc:740] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.400049 23164 consensus_queue.cc:260] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.402297 23164 ts_tablet_manager.cc:1434] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:08:25.402215 23206 raft_consensus.cc:3060] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:25.403469 23164 tablet_bootstrap.cc:492] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.425689 23164 tablet_bootstrap.cc:492] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.426712 23164 tablet_bootstrap.cc:492] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.427397 23164 ts_tablet_manager.cc:1403] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.024s	user 0.020s	sys 0.000s
I20260709 13:08:25.429359 23164 raft_consensus.cc:359] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.429893 23164 raft_consensus.cc:740] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.430545 23164 consensus_queue.cc:260] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.432811 23164 ts_tablet_manager.cc:1434] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:08:25.433529 23164 tablet_bootstrap.cc:492] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.448160 23164 tablet_bootstrap.cc:492] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.449163 23164 tablet_bootstrap.cc:492] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.449776 23164 ts_tablet_manager.cc:1403] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 13:08:25.451906 23164 raft_consensus.cc:359] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.452546 23164 raft_consensus.cc:740] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.453130 23164 consensus_queue.cc:260] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.454654 23164 ts_tablet_manager.cc:1434] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:08:25.455729 23164 tablet_bootstrap.cc:492] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.473186 23206 raft_consensus.cc:3060] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:25.475883 23164 tablet_bootstrap.cc:492] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.477200 23164 tablet_bootstrap.cc:492] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.477936 23164 ts_tablet_manager.cc:1403] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 13:08:25.480031 23209 raft_consensus.cc:3060] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:25.481355 23164 raft_consensus.cc:359] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.482499 23164 raft_consensus.cc:740] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.490204 23164 consensus_queue.cc:260] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.495879 23164 ts_tablet_manager.cc:1434] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:08:25.504714 23164 tablet_bootstrap.cc:492] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.577534 23164 tablet_bootstrap.cc:492] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.578713 23164 tablet_bootstrap.cc:492] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.579525 23164 ts_tablet_manager.cc:1403] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.075s	user 0.020s	sys 0.004s
I20260709 13:08:25.582757 23164 raft_consensus.cc:359] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.586804 23164 raft_consensus.cc:740] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.587698 23164 consensus_queue.cc:260] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.593716 23164 ts_tablet_manager.cc:1434] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:08:25.594626 23164 tablet_bootstrap.cc:492] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.636965 23164 tablet_bootstrap.cc:492] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.638087 23164 tablet_bootstrap.cc:492] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.638820 23164 ts_tablet_manager.cc:1403] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.023s
I20260709 13:08:25.641644 23164 raft_consensus.cc:359] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.642416 23164 raft_consensus.cc:740] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.643069 23164 consensus_queue.cc:260] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:25.644999 23164 ts_tablet_manager.cc:1434] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:08:25.645776 23164 tablet_bootstrap.cc:492] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.662516 23164 tablet_bootstrap.cc:492] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.663534 23164 tablet_bootstrap.cc:492] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.664271 23164 ts_tablet_manager.cc:1403] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.003s
I20260709 13:08:25.667071 23164 raft_consensus.cc:359] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.667757 23164 raft_consensus.cc:740] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.668447 23164 consensus_queue.cc:260] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.670408 23164 ts_tablet_manager.cc:1434] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.008s
I20260709 13:08:25.671420 23164 tablet_bootstrap.cc:492] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.686614 23164 tablet_bootstrap.cc:492] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.687739 23164 tablet_bootstrap.cc:492] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.688511 23164 ts_tablet_manager.cc:1403] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
W20260709 13:08:25.688408 22375 consensus_peers.cc:597] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:25.691246 23164 raft_consensus.cc:359] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.691960 23164 raft_consensus.cc:740] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.692607 23164 consensus_queue.cc:260] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.692998 23206 raft_consensus.cc:3060] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:25.694645 23164 ts_tablet_manager.cc:1434] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:08:25.695470 23164 tablet_bootstrap.cc:492] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.732777 23164 tablet_bootstrap.cc:492] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.733685 23164 tablet_bootstrap.cc:492] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.734508 23164 ts_tablet_manager.cc:1403] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.019s
I20260709 13:08:25.737811 23164 raft_consensus.cc:359] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.738588 23164 raft_consensus.cc:740] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.738816 23205 raft_consensus.cc:3060] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:25.739334 23164 consensus_queue.cc:260] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.741832 23164 ts_tablet_manager.cc:1434] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:08:25.742723 23164 tablet_bootstrap.cc:492] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.793924 23164 tablet_bootstrap.cc:492] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.794998 23164 tablet_bootstrap.cc:492] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.795589 23164 ts_tablet_manager.cc:1403] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.053s	user 0.026s	sys 0.023s
I20260709 13:08:25.798290 23164 raft_consensus.cc:359] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.798995 23164 raft_consensus.cc:740] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.799660 23164 consensus_queue.cc:260] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:25.801965 23164 ts_tablet_manager.cc:1434] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:08:25.803079 23164 tablet_bootstrap.cc:492] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607: Bootstrap starting.
I20260709 13:08:25.839771 23164 tablet_bootstrap.cc:492] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:25.840888 23164 tablet_bootstrap.cc:492] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607: Bootstrap complete.
I20260709 13:08:25.841620 23164 ts_tablet_manager.cc:1403] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.021s
I20260709 13:08:25.844327 23164 raft_consensus.cc:359] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.845016 23164 raft_consensus.cc:740] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652d70a0b765447792ab9b4383712607, State: Initialized, Role: FOLLOWER
I20260709 13:08:25.845763 23164 consensus_queue.cc:260] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:25.847780 23164 ts_tablet_manager.cc:1434] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.006s
I20260709 13:08:25.903824 23205 raft_consensus.cc:3060] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:26.019903 23205 raft_consensus.cc:3060] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:26.086009 23205 raft_consensus.cc:3060] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:08:32.140431 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 44942.9us
W20260709 13:08:32.228858 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 57455.2us
W20260709 13:08:32.234370 22374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:37225, user_credentials={real_user=slave}} blocked reactor thread for 62683.6us
I20260709 13:08:34.535948 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:34.586146 22262 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "214d3c5d47d848af997da7647c556b85"
dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:48640
I20260709 13:08:34.587306 22262 raft_consensus.cc:606] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:34.606438 22407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f757dd6e4b94a8a84bfaddd602685e4"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "93815010a884438c86fc6479359b69e8"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:34.606943 22407 raft_consensus.cc:606] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Received request to transfer leadership to TS 93815010a884438c86fc6479359b69e8
I20260709 13:08:34.609508 22407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:34.609943 22407 raft_consensus.cc:606] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:34.612447 22407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:34.612993 22407 raft_consensus.cc:606] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:34.615448 22407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "93815010a884438c86fc6479359b69e8"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:34.615881 22407 raft_consensus.cc:606] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Received request to transfer leadership to TS 93815010a884438c86fc6479359b69e8
I20260709 13:08:34.618119 22407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "93815010a884438c86fc6479359b69e8"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:34.618685 22407 raft_consensus.cc:606] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 LEADER]: Received request to transfer leadership to TS 93815010a884438c86fc6479359b69e8
I20260709 13:08:34.619766 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:08:34.621681 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:34.639958 23329 raft_consensus.cc:993] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 93815010a884438c86fc6479359b69e8 to start an election
I20260709 13:08:34.640681 23329 raft_consensus.cc:1081] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Signalling peer 93815010a884438c86fc6479359b69e8 to start an election
I20260709 13:08:34.642617 22329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b"
dest_uuid: "93815010a884438c86fc6479359b69e8"
 from {username='slave'} at 127.0.0.1:33160
I20260709 13:08:34.643313 22329 raft_consensus.cc:493] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:34.643783 22329 raft_consensus.cc:3060] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:34.658540 22329 raft_consensus.cc:515] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:34.661705 22329 leader_election.cc:290] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:34.663348 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:34.664237 22407 raft_consensus.cc:3055] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:34.663476 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a06b42cc294ea4aa15d4ecb7ce088b" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:34.664562 22407 raft_consensus.cc:740] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:34.664866 22259 raft_consensus.cc:3060] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:34.665513 22407 consensus_queue.cc:260] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:34.666980 22407 raft_consensus.cc:3060] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:34.690569 22407 raft_consensus.cc:2468] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 2.
I20260709 13:08:34.691989 22300 leader_election.cc:304] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:34.692900 23309 raft_consensus.cc:2804] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:34.693470 23309 raft_consensus.cc:697] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 2 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
I20260709 13:08:34.694568 23309 consensus_queue.cc:237] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:34.695148 23329 raft_consensus.cc:993] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:34.696971 23329 raft_consensus.cc:1081] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:34.702608 22259 raft_consensus.cc:2468] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 2.
I20260709 13:08:34.707402 23206 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36748
I20260709 13:08:34.707937 23206 raft_consensus.cc:493] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:34.708308 23206 raft_consensus.cc:3060] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.716779 22082 catalog_manager.cc:5697] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 1 to 2, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:34.718017 23206 raft_consensus.cc:515] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:34.720819 23206 leader_election.cc:290] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:34.760651 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:34.761385 22407 raft_consensus.cc:3055] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:34.761698 22407 raft_consensus.cc:740] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:34.762678 22407 consensus_queue.cc:260] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:34.764276 22407 raft_consensus.cc:3060] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.769156 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a39127728d4e48a9aeb4e71e4c7e1518" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:34.769879 22259 raft_consensus.cc:3060] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.772444 22407 raft_consensus.cc:2468] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:34.773977 23159 leader_election.cc:304] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:34.778825 22259 raft_consensus.cc:2468] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:34.787107 23344 raft_consensus.cc:2804] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:34.788676 23344 raft_consensus.cc:697] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:34.789624 23344 consensus_queue.cc:237] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:34.802197 22082 catalog_manager.cc:5697] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 2 to 3, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:34.836596 23322 raft_consensus.cc:993] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:34.837260 23329 raft_consensus.cc:1081] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:34.839092 23209 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36748
I20260709 13:08:34.839650 23209 raft_consensus.cc:493] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:34.840015 23209 raft_consensus.cc:3060] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.846868 23320 raft_consensus.cc:993] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:34.847563 23312 raft_consensus.cc:1081] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:34.848431 23209 raft_consensus.cc:515] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:34.849226 23205 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "214d3c5d47d848af997da7647c556b85"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36756
I20260709 13:08:34.849790 23205 raft_consensus.cc:493] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:34.850171 23205 raft_consensus.cc:3060] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.850557 23209 leader_election.cc:290] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:34.851540 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:34.852164 22407 raft_consensus.cc:3055] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:34.852892 22407 raft_consensus.cc:740] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:34.854354 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51f4b4253994202856577a87dd8d61c" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:34.854460 22407 consensus_queue.cc:260] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:34.855145 22259 raft_consensus.cc:3060] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.856122 22407 raft_consensus.cc:3060] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.859936 23205 raft_consensus.cc:515] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:34.863339 22259 raft_consensus.cc:2468] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:34.864209 23205 leader_election.cc:290] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:34.864583 23159 leader_election.cc:304] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:34.864271 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:34.865613 22256 raft_consensus.cc:3055] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:34.865736 23344 raft_consensus.cc:2804] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:34.865965 22256 raft_consensus.cc:740] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:34.866366 23344 raft_consensus.cc:697] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:34.866950 22256 consensus_queue.cc:260] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:34.867333 23344 consensus_queue.cc:237] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:34.868625 22256 raft_consensus.cc:3060] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.869694 22407 raft_consensus.cc:2468] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:34.878269 23326 raft_consensus.cc:993] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 93815010a884438c86fc6479359b69e8 to start an election
I20260709 13:08:34.878814 23322 raft_consensus.cc:1081] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 LEADER]: Signalling peer 93815010a884438c86fc6479359b69e8 to start an election
I20260709 13:08:34.880939 22329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6"
dest_uuid: "93815010a884438c86fc6479359b69e8"
 from {username='slave'} at 127.0.0.1:33160
I20260709 13:08:34.881543 22329 raft_consensus.cc:493] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:34.881984 22329 raft_consensus.cc:3060] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:34.883072 22256 raft_consensus.cc:2468] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:34.886046 23159 leader_election.cc:304] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:34.887022 23344 raft_consensus.cc:2804] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:34.887936 23344 raft_consensus.cc:697] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:34.888866 23344 consensus_queue.cc:237] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:34.893055 22329 raft_consensus.cc:515] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:34.895502 22329 leader_election.cc:290] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 4 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:34.892450 22082 catalog_manager.cc:5697] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 2 to 3, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:34.897178 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:34.897843 22259 raft_consensus.cc:3060] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:34.902168 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9cc1b69991141439a4ee9459859b4a6" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:34.902878 22407 raft_consensus.cc:3055] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:34.903172 22407 raft_consensus.cc:740] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:34.904474 22407 consensus_queue.cc:260] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:34.905833 22407 raft_consensus.cc:3060] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:34.908054 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214d3c5d47d848af997da7647c556b85" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:34.908722 22329 raft_consensus.cc:3060] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.915148 22407 raft_consensus.cc:2468] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 4.
I20260709 13:08:34.916540 22300 leader_election.cc:304] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:34.917511 23309 raft_consensus.cc:2804] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:34.917976 23309 raft_consensus.cc:697] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 4 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
I20260709 13:08:34.918902 23309 consensus_queue.cc:237] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:34.921721 22259 raft_consensus.cc:2468] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 4.
I20260709 13:08:34.926478 23326 raft_consensus.cc:993] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 93815010a884438c86fc6479359b69e8 to start an election
I20260709 13:08:34.927017 23322 raft_consensus.cc:1081] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Signalling peer 93815010a884438c86fc6479359b69e8 to start an election
I20260709 13:08:34.929708 22330 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f757dd6e4b94a8a84bfaddd602685e4"
dest_uuid: "93815010a884438c86fc6479359b69e8"
 from {username='slave'} at 127.0.0.1:33160
I20260709 13:08:34.930253 22330 raft_consensus.cc:493] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:34.930684 22330 raft_consensus.cc:3060] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:34.938951 22329 raft_consensus.cc:2468] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:34.942809 22085 catalog_manager.cc:5697] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 3 to 4, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 4 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:34.947181 22082 catalog_manager.cc:5697] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 2 to 3, leader changed from bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:34.959661 22330 raft_consensus.cc:515] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:34.963600 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f757dd6e4b94a8a84bfaddd602685e4" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:34.963917 22330 leader_election.cc:290] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:34.964255 22259 raft_consensus.cc:3060] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:34.966244 22407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f757dd6e4b94a8a84bfaddd602685e4" candidate_uuid: "93815010a884438c86fc6479359b69e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:34.966893 22407 raft_consensus.cc:3055] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:34.967307 22407 raft_consensus.cc:740] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:34.968104 22407 consensus_queue.cc:260] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:34.969244 22407 raft_consensus.cc:3060] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:34.972393 22259 raft_consensus.cc:2468] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 2.
I20260709 13:08:34.973903 22300 leader_election.cc:304] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93815010a884438c86fc6479359b69e8, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:34.974902 23309 raft_consensus.cc:2804] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:34.975589 23309 raft_consensus.cc:697] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [term 2 LEADER]: Becoming Leader. State: Replica: 93815010a884438c86fc6479359b69e8, State: Running, Role: LEADER
I20260709 13:08:34.976532 22407 raft_consensus.cc:2468] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93815010a884438c86fc6479359b69e8 in term 2.
I20260709 13:08:34.976478 23309 consensus_queue.cc:237] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } }
I20260709 13:08:34.987390 22085 catalog_manager.cc:5697] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 reported cstate change: term changed from 1 to 2, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 93815010a884438c86fc6479359b69e8 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "93815010a884438c86fc6479359b69e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.155814 22407 raft_consensus.cc:1275] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer 93815010a884438c86fc6479359b69e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:35.156980 23309 consensus_queue.cc:1048] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:35.170764 22259 raft_consensus.cc:1275] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Refusing update from remote peer 93815010a884438c86fc6479359b69e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:35.172237 23309 consensus_queue.cc:1048] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:35.248910 22259 raft_consensus.cc:1275] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:35.250275 23344 consensus_queue.cc:1048] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:35.287115 22407 raft_consensus.cc:1275] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:35.288727 23345 consensus_queue.cc:1048] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:35.325958 22407 raft_consensus.cc:1275] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:35.327472 23345 consensus_queue.cc:1048] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:35.328677 22407 raft_consensus.cc:1275] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 4 FOLLOWER]: Refusing update from remote peer 93815010a884438c86fc6479359b69e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:35.330202 23309 consensus_queue.cc:1048] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:35.339411 22256 raft_consensus.cc:1275] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:35.343626 22256 raft_consensus.cc:1275] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 4 FOLLOWER]: Refusing update from remote peer 93815010a884438c86fc6479359b69e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:35.344811 23345 consensus_queue.cc:1048] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:35.346573 23339 consensus_queue.cc:1048] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:35.454807 22330 raft_consensus.cc:1275] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:35.456161 23363 consensus_queue.cc:1048] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:35.467128 22259 raft_consensus.cc:1275] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:35.470803 23368 consensus_queue.cc:1048] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:35.583958 22404 raft_consensus.cc:1275] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer 93815010a884438c86fc6479359b69e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:35.585166 23309 consensus_queue.cc:1048] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:35.595944 22259 raft_consensus.cc:1275] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Refusing update from remote peer 93815010a884438c86fc6479359b69e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:35.597582 23338 consensus_queue.cc:1048] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:35.622879 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:35.666026 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:35.706817 22404 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:35.707480 22404 raft_consensus.cc:606] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:35.709970 22404 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17e65d47217943239d60b88b4b70ba68"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:35.710526 22404 raft_consensus.cc:606] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:35.712936 22404 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:35.713397 22404 raft_consensus.cc:606] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:35.715726 22404 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62cc2afc26f14239af465ce79a044163"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:35.716316 22404 raft_consensus.cc:606] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:35.719437 22259 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb"
dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:48640
I20260709 13:08:35.719995 22259 raft_consensus.cc:606] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:35.722507 22404 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:35.723011 22404 raft_consensus.cc:606] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:35.724241 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:08:35.728355 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:35.748178 23322 raft_consensus.cc:993] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:35.748903 23326 raft_consensus.cc:1081] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:35.750713 23209 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36748
I20260709 13:08:35.751359 23209 raft_consensus.cc:493] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:35.751686 23209 raft_consensus.cc:3060] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:35.760336 23209 raft_consensus.cc:515] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:35.763418 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:35.764060 22404 raft_consensus.cc:3055] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:35.764412 22404 raft_consensus.cc:740] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:35.765250 22404 consensus_queue.cc:260] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:35.766404 22404 raft_consensus.cc:3060] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:35.780347 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d86dab181da42f79842cbdbd4fa5b05" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:35.781100 22259 raft_consensus.cc:3060] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:35.787753 23209 leader_election.cc:290] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:35.788362 22404 raft_consensus.cc:2468] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 2.
I20260709 13:08:35.790287 23159 leader_election.cc:304] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:35.791213 23345 raft_consensus.cc:2804] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:35.791742 23345 raft_consensus.cc:697] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 2 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:35.792723 23345 consensus_queue.cc:237] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:35.794011 22259 raft_consensus.cc:2468] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 2.
I20260709 13:08:35.808889 22085 catalog_manager.cc:5697] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 1 to 2, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.823078 23326 raft_consensus.cc:993] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:35.823648 23326 raft_consensus.cc:1081] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:35.825659 23205 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17e65d47217943239d60b88b4b70ba68"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36748
I20260709 13:08:35.826272 23205 raft_consensus.cc:493] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:35.826552 23205 raft_consensus.cc:3060] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:35.833581 23205 raft_consensus.cc:515] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:35.835536 23205 leader_election.cc:290] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:35.835783 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e65d47217943239d60b88b4b70ba68" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:35.836165 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e65d47217943239d60b88b4b70ba68" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:35.836925 22330 raft_consensus.cc:3060] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:35.836995 22404 raft_consensus.cc:3055] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:35.837448 22404 raft_consensus.cc:740] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:35.838331 22404 consensus_queue.cc:260] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:35.839552 22404 raft_consensus.cc:3060] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:35.844640 22330 raft_consensus.cc:2468] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 2.
I20260709 13:08:35.846004 23160 leader_election.cc:304] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:35.846217 22404 raft_consensus.cc:2468] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 2.
I20260709 13:08:35.846957 23345 raft_consensus.cc:2804] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:35.847486 23345 raft_consensus.cc:697] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 2 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:35.848445 23345 consensus_queue.cc:237] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:35.860289 22085 catalog_manager.cc:5697] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 1 to 2, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:35.900852 23322 raft_consensus.cc:993] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:35.901486 23326 raft_consensus.cc:1081] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:35.903154 23209 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36748
I20260709 13:08:35.903774 23209 raft_consensus.cc:493] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:35.904053 23209 raft_consensus.cc:3060] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:35.910543 23209 raft_consensus.cc:515] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:35.912338 23209 leader_election.cc:290] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:35.913130 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:35.913400 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad7895d69404083ba56a16062a8ed20" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:35.913748 22404 raft_consensus.cc:3055] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:35.914038 22259 raft_consensus.cc:3060] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:35.914198 22404 raft_consensus.cc:740] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:35.915058 22404 consensus_queue.cc:260] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:35.916445 22404 raft_consensus.cc:3060] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:35.920593 22259 raft_consensus.cc:2468] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 2.
I20260709 13:08:35.921694 23159 leader_election.cc:304] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:35.922713 23345 raft_consensus.cc:2804] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:35.923219 23345 raft_consensus.cc:697] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 2 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:35.924185 23345 consensus_queue.cc:237] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:35.929294 22404 raft_consensus.cc:2468] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 2.
I20260709 13:08:35.937285 22085 catalog_manager.cc:5697] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 1 to 2, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.975564 23329 raft_consensus.cc:993] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:35.976150 23326 raft_consensus.cc:1081] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:35.977650 23209 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62cc2afc26f14239af465ce79a044163"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36748
I20260709 13:08:35.978237 23209 raft_consensus.cc:493] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:35.978560 23209 raft_consensus.cc:3060] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.984702 23209 raft_consensus.cc:515] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:35.986229 23209 leader_election.cc:290] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:35.987056 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:35.987383 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cc2afc26f14239af465ce79a044163" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:35.987663 22404 raft_consensus.cc:3055] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:35.987958 22259 raft_consensus.cc:3060] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.988049 22404 raft_consensus.cc:740] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:35.989002 22404 consensus_queue.cc:260] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:35.990367 22404 raft_consensus.cc:3060] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.996302 22259 raft_consensus.cc:2468] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:35.997653 23159 leader_election.cc:304] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:35.998577 23345 raft_consensus.cc:2804] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:35.999090 23345 raft_consensus.cc:697] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:35.999984 23345 consensus_queue.cc:237] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:36.003619 22404 raft_consensus.cc:2468] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:36.011260 22085 catalog_manager.cc:5697] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 2 to 3, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:36.080135 23322 raft_consensus.cc:993] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:36.080705 23322 raft_consensus.cc:1081] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:36.082252 23209 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36748
I20260709 13:08:36.082847 23209 raft_consensus.cc:493] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:36.083199 23209 raft_consensus.cc:3060] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:36.090575 23209 raft_consensus.cc:515] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:36.092244 23209 leader_election.cc:290] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 4 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:36.093421 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:36.093516 22259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd464ed1f28f42ccad10b0df649c7ce7" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:36.094332 22404 raft_consensus.cc:3055] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:36.094365 22259 raft_consensus.cc:3060] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:36.094779 22404 raft_consensus.cc:740] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:36.095595 22404 consensus_queue.cc:260] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:36.096915 22404 raft_consensus.cc:3060] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:36.102041 22259 raft_consensus.cc:2468] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 4.
I20260709 13:08:36.103358 23159 leader_election.cc:304] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:36.104208 23345 raft_consensus.cc:2804] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:36.105376 23345 raft_consensus.cc:697] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 4 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:36.106277 22404 raft_consensus.cc:2468] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 4.
I20260709 13:08:36.106271 23345 consensus_queue.cc:237] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:36.115845 22085 catalog_manager.cc:5697] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 3 to 4, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 4 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:36.197669 23312 raft_consensus.cc:993] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:36.198377 23352 raft_consensus.cc:1081] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:36.199971 23209 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36756
I20260709 13:08:36.200507 23209 raft_consensus.cc:493] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:36.200809 23209 raft_consensus.cc:3060] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:36.208159 23209 raft_consensus.cc:515] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:36.210922 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:36.211639 22256 raft_consensus.cc:3055] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:36.211665 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd676d7998314b3b9964705206bd4bcb" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:36.212056 22256 raft_consensus.cc:740] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:36.212342 22329 raft_consensus.cc:3060] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:36.212951 22256 consensus_queue.cc:260] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:36.210309 23209 leader_election.cc:290] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:36.214180 22256 raft_consensus.cc:3060] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:36.220008 22329 raft_consensus.cc:2468] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:36.221370 22256 raft_consensus.cc:2468] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:36.221338 23160 leader_election.cc:304] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:36.222409 23345 raft_consensus.cc:2804] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:36.223165 23345 raft_consensus.cc:697] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:36.224114 23345 consensus_queue.cc:237] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:36.236388 22085 catalog_manager.cc:5697] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 2 to 3, leader changed from bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:36.241222 22404 raft_consensus.cc:1275] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:36.242764 23345 consensus_queue.cc:1048] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:36.257805 22329 raft_consensus.cc:1275] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:36.261336 23368 consensus_queue.cc:1048] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:36.346897 22259 raft_consensus.cc:1275] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:36.348294 23368 consensus_queue.cc:1048] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:36.352356 22404 raft_consensus.cc:1275] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:36.353803 23368 consensus_queue.cc:1048] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:36.474196 22404 raft_consensus.cc:1275] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:36.475476 23368 consensus_queue.cc:1048] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:36.485822 22259 raft_consensus.cc:1275] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:36.488039 23363 consensus_queue.cc:1048] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:36.539896 22404 raft_consensus.cc:1275] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 4 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:08:36.540210 22375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:34889, user_credentials={real_user=slave}} blocked reactor thread for 43979.8us
I20260709 13:08:36.542351 23363 consensus_queue.cc:1048] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:36.562882 22256 raft_consensus.cc:1275] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 4 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:36.565157 23345 consensus_queue.cc:1048] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:36.593782 22407 raft_consensus.cc:1275] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:36.595494 23363 consensus_queue.cc:1048] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:36.600126 22262 raft_consensus.cc:1275] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:36.615514 23345 consensus_queue.cc:1048] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:36.710317 22330 raft_consensus.cc:1275] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:36.711627 23345 consensus_queue.cc:1048] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:36.725744 22256 raft_consensus.cc:1275] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:36.729413 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:36.729359 23344 consensus_queue.cc:1048] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:36.782642 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:36.835464 22404 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:36.836264 22404 raft_consensus.cc:606] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:36.839833 22407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "691ad2369e00484995e87d12f574dfdc"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:36.840427 22407 raft_consensus.cc:606] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:36.842990 22407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5e8f1fb740446b8b42c60afd2e7c3eb"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:36.843448 22407 raft_consensus.cc:606] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:36.846359 22407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:36.846971 22407 raft_consensus.cc:606] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:36.849762 22404 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb4bb4b406fa4f4f9a55ba3b7c005cf0"
dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:54388
I20260709 13:08:36.850337 22404 raft_consensus.cc:606] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:36.854427 22256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5"
dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
mode: GRACEFUL
new_leader_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:48640
I20260709 13:08:36.854997 22256 raft_consensus.cc:606] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Received request to transfer leadership to TS 652d70a0b765447792ab9b4383712607
I20260709 13:08:36.856248 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:08:36.858697 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:37.062117 23329 raft_consensus.cc:993] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:37.062681 23329 raft_consensus.cc:1081] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:37.064189 23206 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36748
I20260709 13:08:37.064630 23206 raft_consensus.cc:493] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:37.064875 23206 raft_consensus.cc:3060] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:37.071594 23206 raft_consensus.cc:515] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:37.073180 23206 leader_election.cc:290] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:37.073885 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:37.074193 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf086b080a8c46ff809e2b9a14bdacae" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:37.074628 22330 raft_consensus.cc:3060] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:37.074872 22404 raft_consensus.cc:3055] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:37.075253 22404 raft_consensus.cc:740] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:37.076052 22404 consensus_queue.cc:260] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:37.077914 22404 raft_consensus.cc:3060] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:37.081115 22330 raft_consensus.cc:2468] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:37.082376 23160 leader_election.cc:304] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:37.083153 23363 raft_consensus.cc:2804] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:37.083570 23363 raft_consensus.cc:697] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:37.083659 22404 raft_consensus.cc:2468] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:37.084518 23363 consensus_queue.cc:237] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:37.095170 22083 catalog_manager.cc:5697] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 2 to 3, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.120054 23404 raft_consensus.cc:993] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:37.120633 23391 raft_consensus.cc:1081] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:37.122205 23206 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb4bb4b406fa4f4f9a55ba3b7c005cf0"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36748
I20260709 13:08:37.122828 23206 raft_consensus.cc:493] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:37.123203 23206 raft_consensus.cc:3060] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:37.130888 23206 raft_consensus.cc:515] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:37.133882 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb4bb4b406fa4f4f9a55ba3b7c005cf0" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:37.134549 22404 raft_consensus.cc:3055] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:37.134860 22404 raft_consensus.cc:740] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:37.135622 22404 consensus_queue.cc:260] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:37.135833 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb4bb4b406fa4f4f9a55ba3b7c005cf0" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:37.136709 22330 raft_consensus.cc:3060] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:37.137359 23206 leader_election.cc:290] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:37.136745 22404 raft_consensus.cc:3060] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:37.146209 22404 raft_consensus.cc:2468] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 2.
I20260709 13:08:37.147665 23159 leader_election.cc:304] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 
I20260709 13:08:37.148636 23363 raft_consensus.cc:2804] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:37.149152 23363 raft_consensus.cc:697] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 2 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:37.151079 23363 consensus_queue.cc:237] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:37.155059 22330 raft_consensus.cc:2468] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 2.
I20260709 13:08:37.172266 23404 raft_consensus.cc:993] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:37.174341 23391 raft_consensus.cc:1081] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:37.176383 23205 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5e8f1fb740446b8b42c60afd2e7c3eb"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36748
I20260709 13:08:37.176951 23205 raft_consensus.cc:493] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:37.177335 23205 raft_consensus.cc:3060] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:37.184549 22083 catalog_manager.cc:5697] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 1 to 2, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.186630 23205 raft_consensus.cc:515] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:37.189260 23205 leader_election.cc:290] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:37.189461 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e8f1fb740446b8b42c60afd2e7c3eb" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:37.190176 22330 raft_consensus.cc:3060] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:37.189967 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e8f1fb740446b8b42c60afd2e7c3eb" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:37.190842 22404 raft_consensus.cc:3055] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:37.191175 22404 raft_consensus.cc:740] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:37.194633 22404 consensus_queue.cc:260] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:37.195952 22404 raft_consensus.cc:3060] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:37.200922 22330 raft_consensus.cc:2468] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 2.
I20260709 13:08:37.202196 23160 leader_election.cc:304] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:37.203090 23345 raft_consensus.cc:2804] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:37.203580 23345 raft_consensus.cc:697] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 2 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:37.203881 22404 raft_consensus.cc:2468] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 2.
I20260709 13:08:37.204550 23345 consensus_queue.cc:237] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:37.218176 23391 raft_consensus.cc:993] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:37.218816 23404 raft_consensus.cc:1081] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:37.221268 23205 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "691ad2369e00484995e87d12f574dfdc"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36748
I20260709 13:08:37.221858 23205 raft_consensus.cc:493] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:37.223618 23205 raft_consensus.cc:3060] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:37.221381 22083 catalog_manager.cc:5697] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 1 to 2, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.233935 23205 raft_consensus.cc:515] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:37.236351 23205 leader_election.cc:290] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 election: Requested vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261)
I20260709 13:08:37.237898 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ad2369e00484995e87d12f574dfdc" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:37.238505 22404 raft_consensus.cc:3055] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:37.238788 22404 raft_consensus.cc:740] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:37.239722 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ad2369e00484995e87d12f574dfdc" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:37.239544 22404 consensus_queue.cc:260] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:37.240355 22330 raft_consensus.cc:3060] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:37.240952 22404 raft_consensus.cc:3060] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:37.248412 22330 raft_consensus.cc:2468] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 2.
I20260709 13:08:37.248487 22404 raft_consensus.cc:2468] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 2.
I20260709 13:08:37.252425 23160 leader_election.cc:304] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:37.253273 23345 raft_consensus.cc:2804] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:37.253793 23345 raft_consensus.cc:697] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 2 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:37.254776 23345 consensus_queue.cc:237] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:37.267058 22083 catalog_manager.cc:5697] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 1 to 2, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.272037 23406 raft_consensus.cc:993] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:37.272665 23391 raft_consensus.cc:1081] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:37.274407 23205 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36748
I20260709 13:08:37.274962 23205 raft_consensus.cc:493] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:37.275293 23205 raft_consensus.cc:3060] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:37.281950 23205 raft_consensus.cc:515] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:37.283552 23205 leader_election.cc:290] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Requested vote from peers 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68:41261), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:37.284390 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc"
I20260709 13:08:37.284701 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a08df7f7be4e808ba69787809be0d3" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:37.285070 22404 raft_consensus.cc:3055] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:37.285406 22256 raft_consensus.cc:3060] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:37.285492 22404 raft_consensus.cc:740] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 67fc9e7fc4b34621ae1cd3e1621b89dc, State: Running, Role: LEADER
I20260709 13:08:37.286432 22404 consensus_queue.cc:260] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:37.287883 22404 raft_consensus.cc:3060] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:37.291641 22256 raft_consensus.cc:2468] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:37.292774 23159 leader_election.cc:304] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0; no voters: 
I20260709 13:08:37.293589 23345 raft_consensus.cc:2804] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:37.294037 23345 raft_consensus.cc:697] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:37.294088 22404 raft_consensus.cc:2468] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:37.295094 23345 consensus_queue.cc:237] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:37.304171 23392 raft_consensus.cc:993] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0: : Instructing follower 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:37.305158 23312 raft_consensus.cc:1081] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Signalling peer 652d70a0b765447792ab9b4383712607 to start an election
I20260709 13:08:37.306753 23205 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5"
dest_uuid: "652d70a0b765447792ab9b4383712607"
 from {username='slave'} at 127.0.0.1:36756
I20260709 13:08:37.307317 23205 raft_consensus.cc:493] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:37.307662 23205 raft_consensus.cc:3060] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:37.307932 22083 catalog_manager.cc:5697] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 2 to 3, leader changed from 67fc9e7fc4b34621ae1cd3e1621b89dc (127.12.109.68) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:37.316879 23205 raft_consensus.cc:515] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:37.320134 22256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb26db1bd5a64429848b5d63b157c6f0"
I20260709 13:08:37.320279 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb774f42e99437f8180b3d9e89a9cf5" candidate_uuid: "652d70a0b765447792ab9b4383712607" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "93815010a884438c86fc6479359b69e8"
I20260709 13:08:37.320791 22256 raft_consensus.cc:3055] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:37.321096 22330 raft_consensus.cc:3060] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:37.321182 22256 raft_consensus.cc:740] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bb26db1bd5a64429848b5d63b157c6f0, State: Running, Role: LEADER
I20260709 13:08:37.322249 23205 leader_election.cc:290] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Requested vote from peers bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225), 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889)
I20260709 13:08:37.322209 22256 consensus_queue.cc:260] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:37.323552 22256 raft_consensus.cc:3060] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:37.329380 22330 raft_consensus.cc:2468] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:37.330574 23160 leader_election.cc:304] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8; no voters: 
I20260709 13:08:37.331676 22256 raft_consensus.cc:2468] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652d70a0b765447792ab9b4383712607 in term 3.
I20260709 13:08:37.332587 23345 raft_consensus.cc:2804] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:37.333159 23345 raft_consensus.cc:697] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Becoming Leader. State: Replica: 652d70a0b765447792ab9b4383712607, State: Running, Role: LEADER
I20260709 13:08:37.334187 23345 consensus_queue.cc:237] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } }
I20260709 13:08:37.347630 22083 catalog_manager.cc:5697] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 reported cstate change: term changed from 2 to 3, leader changed from bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66) to 652d70a0b765447792ab9b4383712607 (127.12.109.65). New cstate: current_term: 3 leader_uuid: "652d70a0b765447792ab9b4383712607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:37.483248 22329 raft_consensus.cc:1275] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:37.484586 23345 consensus_queue.cc:1048] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:37.495178 22404 raft_consensus.cc:1275] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:37.499281 23368 consensus_queue.cc:1048] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:37.647058 22404 raft_consensus.cc:1275] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:37.648597 23345 consensus_queue.cc:1048] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:37.667435 22330 raft_consensus.cc:1275] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:37.669417 23344 consensus_queue.cc:1048] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:37.703006 22329 raft_consensus.cc:1275] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:37.704787 23345 consensus_queue.cc:1048] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:37.726419 22404 raft_consensus.cc:1275] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:37.728225 23363 consensus_queue.cc:1048] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:37.740635 22408 raft_consensus.cc:1275] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:37.743201 23363 consensus_queue.cc:1048] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:37.757740 22329 raft_consensus.cc:1275] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:37.759481 23363 consensus_queue.cc:1048] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:37.774639 22262 raft_consensus.cc:1275] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:37.776479 23344 consensus_queue.cc:1048] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:37.796066 22256 raft_consensus.cc:1275] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:37.797859 23363 consensus_queue.cc:1048] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:37.801396 22329 raft_consensus.cc:1275] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:37.818331 23363 consensus_queue.cc:1048] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:37.830238 22404 raft_consensus.cc:1275] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Refusing update from remote peer 652d70a0b765447792ab9b4383712607: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:37.832460 23363 consensus_queue.cc:1048] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:37.859709 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:37.932626 12725 tablet_server.cc:179] TabletServer@127.12.109.65:46399 shutting down...
W20260709 13:08:37.975255 22375 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 124 similar messages]
W20260709 13:08:37.988628 22375 consensus_peers.cc:597] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:37.990329 22375 consensus_peers.cc:597] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.002022 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:38.003326 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.004555 12725 raft_consensus.cc:2243] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.005815 12725 raft_consensus.cc:2272] T 691ad2369e00484995e87d12f574dfdc P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.010536 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.011735 12725 raft_consensus.cc:2243] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.012447 12725 raft_consensus.cc:2272] T a91b5edf8c9e4a71a195e0f796d439f3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.012657 22227 consensus_peers.cc:597] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.015123 22227 consensus_peers.cc:597] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.016597 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.018304 12725 raft_consensus.cc:2243] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.019310 12725 raft_consensus.cc:2272] T fd464ed1f28f42ccad10b0df649c7ce7 P 652d70a0b765447792ab9b4383712607 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.020493 22301 consensus_peers.cc:597] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.022166 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.023052 12725 raft_consensus.cc:2243] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.023496 12725 raft_consensus.cc:2272] T 814c209a7aff4b11bdc62505cc29c286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.026471 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.027139 12725 raft_consensus.cc:2243] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.027661 12725 raft_consensus.cc:2272] T ade9dda3178e42c783bc93e5deec21b7 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.030494 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.031076 12725 raft_consensus.cc:2243] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.031832 12725 raft_consensus.cc:2272] T 214d3c5d47d848af997da7647c556b85 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.034211 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.034863 12725 raft_consensus.cc:2243] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.035305 12725 raft_consensus.cc:2272] T 3808206385a0458abc04050aeab1cea7 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.037729 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.038416 12725 raft_consensus.cc:2243] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.039280 12725 raft_consensus.cc:2272] T a5e8f1fb740446b8b42c60afd2e7c3eb P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.042012 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.042883 12725 raft_consensus.cc:2243] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.043828 12725 raft_consensus.cc:2272] T 4ad7895d69404083ba56a16062a8ed20 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.046317 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.047274 12725 raft_consensus.cc:2243] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.047832 12725 raft_consensus.cc:2272] T 59f7f96ef51941e39bc97c0b769c7770 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.049290 22227 consensus_peers.cc:597] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.050678 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.051434 12725 raft_consensus.cc:2243] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.052330 12725 raft_consensus.cc:2272] T 16a08df7f7be4e808ba69787809be0d3 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.055712 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.056643 12725 raft_consensus.cc:2243] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.057273 12725 raft_consensus.cc:2272] T dc4bb1a452944c59aa3dc6717c52be6a P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.060421 12725 tablet_replica.cc:333] stopping tablet replica
W20260709 13:08:38.061020 22227 consensus_peers.cc:597] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.061223 12725 raft_consensus.cc:2243] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.062409 12725 raft_consensus.cc:2272] T cd676d7998314b3b9964705206bd4bcb P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.064764 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.065397 12725 raft_consensus.cc:2243] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.066295 12725 raft_consensus.cc:2272] T a39127728d4e48a9aeb4e71e4c7e1518 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.068955 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.069782 12725 raft_consensus.cc:2243] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.070780 12725 raft_consensus.cc:2272] T 0d86dab181da42f79842cbdbd4fa5b05 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.073375 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.074026 12725 raft_consensus.cc:2243] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.074548 12725 raft_consensus.cc:2272] T 878ae1fea9154e7088fdba8930c5eb23 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.077401 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.078150 12725 raft_consensus.cc:2243] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.078676 12725 raft_consensus.cc:2272] T 42127cae8f494cb081ec04febe577130 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.081250 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.081826 12725 raft_consensus.cc:2243] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.082329 12725 raft_consensus.cc:2272] T c0b51a3b1dc444319751dd92054d5dc3 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.085182 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.085934 12725 raft_consensus.cc:2243] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.086483 12725 raft_consensus.cc:2272] T 3a4c4a9cd1164b7997f7188f9da9f70b P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.088982 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.089802 12725 raft_consensus.cc:2243] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.090406 12725 raft_consensus.cc:2272] T 01099e972e9b4d08a64552be3fd52276 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.093372 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.094245 12725 raft_consensus.cc:2243] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.095340 12725 raft_consensus.cc:2272] T ffb774f42e99437f8180b3d9e89a9cf5 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.098042 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.098708 12725 raft_consensus.cc:2243] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.099102 12725 raft_consensus.cc:2272] T 57dfb3dc107a469c880393583ee0c218 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.101426 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.101954 12725 raft_consensus.cc:2243] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.102422 12725 raft_consensus.cc:2272] T f9b737a6d0d24895b749f55ac078ac5f P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.104430 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.105059 12725 raft_consensus.cc:2243] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.105832 12725 raft_consensus.cc:2272] T 17e65d47217943239d60b88b4b70ba68 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.108287 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.109117 12725 raft_consensus.cc:2243] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.110380 12725 raft_consensus.cc:2272] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.112949 22301 consensus_peers.cc:597] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.113474 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.114241 12725 raft_consensus.cc:2243] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.114744 12725 raft_consensus.cc:2272] T ffdd1d227bbe48c8a46c4fefd02eab0b P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.117054 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.117689 12725 raft_consensus.cc:2243] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.118182 12725 raft_consensus.cc:2272] T f0de2f3edc854918bb42aa68cdc2c9fa P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.120407 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.121053 12725 raft_consensus.cc:2243] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.121562 12725 raft_consensus.cc:2272] T 2144e27a177e41b0b22e474d9f209102 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.124846 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.125610 12725 raft_consensus.cc:2243] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.126157 12725 raft_consensus.cc:2272] T d3a61ae21bab486cbfa407942fdeac35 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.128784 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.129669 12725 raft_consensus.cc:2243] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.130775 12725 raft_consensus.cc:2272] T cf086b080a8c46ff809e2b9a14bdacae P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.133705 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.134439 12725 raft_consensus.cc:2243] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.134908 12725 raft_consensus.cc:2272] T fa44244e76604221bcbd08e0f3bbdb82 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.137554 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.138324 12725 raft_consensus.cc:2243] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.138811 12725 raft_consensus.cc:2272] T 86f9732bc4d24ee3a8e261358eba3a21 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.141578 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.142261 12725 raft_consensus.cc:2243] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.142716 12725 raft_consensus.cc:2272] T 1cd9f843a01847a4b475b6dbcb704a6f P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.144959 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.145560 12725 raft_consensus.cc:2243] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.145967 12725 raft_consensus.cc:2272] T 40cf2d6d73fe4593b96ead779d12207e P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.148648 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.149451 12725 raft_consensus.cc:2243] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.150019 12725 raft_consensus.cc:2272] T f7f82cbc4e8544bbac74ba96797d687c P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.151671 22227 consensus_peers.cc:597] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.152822 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.153523 12725 raft_consensus.cc:2243] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.154443 12725 raft_consensus.cc:2272] T 62cc2afc26f14239af465ce79a044163 P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.156682 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.157292 12725 raft_consensus.cc:2243] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.157696 12725 raft_consensus.cc:2272] T 7d38314054d8474eb00ca91785ea945d P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.159937 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.160529 12725 raft_consensus.cc:2243] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.160972 12725 raft_consensus.cc:2272] T 96821e05d1924572a73e450bc7dcb265 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.163394 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.163939 12725 raft_consensus.cc:2243] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.164347 12725 raft_consensus.cc:2272] T c995e1a8ce9c48aab8235f258f77d080 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.166503 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.167062 12725 raft_consensus.cc:2243] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.167539 12725 raft_consensus.cc:2272] T 4f204603e14c438bb549d1116eaac788 P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.169826 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.170419 12725 raft_consensus.cc:2243] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.170861 12725 raft_consensus.cc:2272] T ec8f7472c97049ca98e597c7d4b4eb1c P 652d70a0b765447792ab9b4383712607 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.173053 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.173632 12725 raft_consensus.cc:2243] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.174616 12725 raft_consensus.cc:2272] T a51f4b4253994202856577a87dd8d61c P 652d70a0b765447792ab9b4383712607 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.177459 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.178159 12725 raft_consensus.cc:2243] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.178584 12725 raft_consensus.cc:2272] T 30e6db323fdb4762a2e92b729b051111 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.180744 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.181267 12725 raft_consensus.cc:2243] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.181701 12725 raft_consensus.cc:2272] T 5e95f83f20904ff692e70a1872c3e77a P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.184166 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:38.184810 12725 raft_consensus.cc:2243] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.185396 12725 raft_consensus.cc:2272] T fcf4b5402e45470c81a3389d815a9286 P 652d70a0b765447792ab9b4383712607 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.202987 22301 consensus_peers.cc:597] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.212606 22375 consensus_peers.cc:597] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.221637 22375 consensus_peers.cc:597] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.228596 22301 consensus_peers.cc:597] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.237998 22227 consensus_peers.cc:597] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.276974 22227 consensus_peers.cc:597] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.279450 22375 consensus_peers.cc:597] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.279759 22227 consensus_peers.cc:597] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.280356 22227 consensus_peers.cc:597] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.282621 12725 tablet_server.cc:196] TabletServer@127.12.109.65:46399 shutdown complete.
W20260709 13:08:38.283712 22301 consensus_peers.cc:597] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.284313 22301 consensus_peers.cc:597] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.286651 22227 consensus_peers.cc:597] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.308591 22375 consensus_peers.cc:597] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.311077 22375 consensus_peers.cc:597] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.322606 22301 consensus_peers.cc:597] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.325125 22301 consensus_peers.cc:597] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.359575 22227 consensus_peers.cc:597] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.381281 12725 tablet_server.cc:179] TabletServer@127.12.109.66:0 shutting down...
W20260709 13:08:38.418645 22227 consensus_peers.cc:597] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.420420 22227 consensus_peers.cc:597] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.421766 22374 consensus_peers.cc:597] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.447338 22375 consensus_peers.cc:597] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.451870 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:38.453056 12725 tablet_replica.cc:333] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.455528 12725 raft_consensus.cc:2243] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.456178 12725 raft_consensus.cc:2272] T 57dfb3dc107a469c880393583ee0c218 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.459645 12725 tablet_replica.cc:333] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.460464 12725 raft_consensus.cc:2243] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.461026 12725 raft_consensus.cc:2272] T a51f4b4253994202856577a87dd8d61c P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.463878 22300 consensus_peers.cc:597] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.463907 12725 tablet_replica.cc:333] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.465015 12725 raft_consensus.cc:2243] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.466044 12725 raft_consensus.cc:2272] T dc4bb1a452944c59aa3dc6717c52be6a P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.468434 12725 tablet_replica.cc:333] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.469074 12725 raft_consensus.cc:2243] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.469537 12725 raft_consensus.cc:2272] T 96356b5b69fc405cb1678dbfaec2df1f P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.471726 12725 tablet_replica.cc:333] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.472386 12725 raft_consensus.cc:2243] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.473553 12725 raft_consensus.cc:2272] T 7d38314054d8474eb00ca91785ea945d P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.476716 12725 tablet_replica.cc:333] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.477569 12725 raft_consensus.cc:2243] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:08:38.477927 22227 consensus_peers.cc:597] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 -> Peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Couldn't send request to peer 652d70a0b765447792ab9b4383712607. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.478582 12725 raft_consensus.cc:2272] T f9b737a6d0d24895b749f55ac078ac5f P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.481031 12725 tablet_replica.cc:333] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.481738 12725 raft_consensus.cc:2243] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.482246 12725 raft_consensus.cc:2272] T fdbf4608aa9f48d68cb77a4907dff6a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.484951 12725 tablet_replica.cc:333] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.485920 12725 raft_consensus.cc:2243] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.487016 12725 raft_consensus.cc:2272] T 3808206385a0458abc04050aeab1cea7 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.490136 12725 tablet_replica.cc:333] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.490724 12725 raft_consensus.cc:2243] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.491108 12725 raft_consensus.cc:2272] T 2d9ed71058164d51a8aa5b9980536bd8 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.493258 12725 tablet_replica.cc:333] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.493830 12725 raft_consensus.cc:2243] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.494374 12725 raft_consensus.cc:2272] T 16a08df7f7be4e808ba69787809be0d3 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.497117 12725 tablet_replica.cc:333] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.497931 12725 raft_consensus.cc:2243] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.498903 12725 raft_consensus.cc:2272] T 59f7f96ef51941e39bc97c0b769c7770 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.500823 22300 consensus_peers.cc:597] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.501092 12725 tablet_replica.cc:333] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.502041 12725 raft_consensus.cc:2243] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.503293 12725 raft_consensus.cc:2272] T ffdd1d227bbe48c8a46c4fefd02eab0b P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.505798 12725 tablet_replica.cc:333] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.506462 12725 raft_consensus.cc:2243] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.506901 12725 raft_consensus.cc:2272] T 3e49aee5d99347029a0f5e51fe01b896 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.508894 12725 tablet_replica.cc:333] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.509477 12725 raft_consensus.cc:2243] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.509888 12725 raft_consensus.cc:2272] T 214d3c5d47d848af997da7647c556b85 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.512173 12725 tablet_replica.cc:333] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.513011 12725 raft_consensus.cc:2243] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.513518 12725 raft_consensus.cc:2272] T fa44244e76604221bcbd08e0f3bbdb82 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.516294 12725 tablet_replica.cc:333] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.517160 12725 raft_consensus.cc:2243] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.517697 12725 raft_consensus.cc:2272] T 0418b53b8c2e4786b445d55f0ea5a723 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.520535 12725 tablet_replica.cc:333] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.521407 12725 raft_consensus.cc:2243] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.522589 12725 raft_consensus.cc:2272] T 86f9732bc4d24ee3a8e261358eba3a21 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.522715 22300 consensus_peers.cc:597] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.525418 12725 tablet_replica.cc:333] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.526222 12725 raft_consensus.cc:2243] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.527257 12725 raft_consensus.cc:2272] T 1cd9f843a01847a4b475b6dbcb704a6f P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.530246 12725 tablet_replica.cc:333] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
W20260709 13:08:38.530820 22374 consensus_peers.cc:597] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.531011 12725 raft_consensus.cc:2243] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.531695 12725 raft_consensus.cc:2272] T 2076c24c8d1d47e6b3e4ece63fae1597 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.534690 12725 tablet_replica.cc:333] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.535535 12725 raft_consensus.cc:2243] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.536662 12725 raft_consensus.cc:2272] T 4f204603e14c438bb549d1116eaac788 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.539754 12725 tablet_replica.cc:333] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.540553 12725 raft_consensus.cc:2243] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.541108 12725 raft_consensus.cc:2272] T fd464ed1f28f42ccad10b0df649c7ce7 P bb26db1bd5a64429848b5d63b157c6f0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.543584 12725 tablet_replica.cc:333] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.544209 12725 raft_consensus.cc:2243] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.544719 12725 raft_consensus.cc:2272] T 4ad7895d69404083ba56a16062a8ed20 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.546343 22300 consensus_peers.cc:597] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.547277 12725 tablet_replica.cc:333] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.547912 12725 raft_consensus.cc:2243] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.548380 12725 raft_consensus.cc:2272] T c0b51a3b1dc444319751dd92054d5dc3 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.551067 12725 tablet_replica.cc:333] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.551729 12725 raft_consensus.cc:2243] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.552145 12725 raft_consensus.cc:2272] T cd676d7998314b3b9964705206bd4bcb P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.554165 12725 tablet_replica.cc:333] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.554908 12725 raft_consensus.cc:2243] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.555397 12725 raft_consensus.cc:2272] T 808e97b948be44ee895c060c4329964e P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.558182 12725 tablet_replica.cc:333] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.558943 12725 raft_consensus.cc:2243] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.559473 12725 raft_consensus.cc:2272] T 0d86dab181da42f79842cbdbd4fa5b05 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.561998 12725 tablet_replica.cc:333] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.562789 12725 raft_consensus.cc:2243] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.563385 12725 raft_consensus.cc:2272] T a39127728d4e48a9aeb4e71e4c7e1518 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.566430 12725 tablet_replica.cc:333] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.567191 12725 raft_consensus.cc:2243] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.567718 12725 raft_consensus.cc:2272] T c9cc1b69991141439a4ee9459859b4a6 P bb26db1bd5a64429848b5d63b157c6f0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.570161 12725 tablet_replica.cc:333] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
W20260709 13:08:38.571043 22374 consensus_peers.cc:597] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.571177 12725 raft_consensus.cc:2243] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.571960 12725 raft_consensus.cc:2272] T 62cc2afc26f14239af465ce79a044163 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.574777 12725 tablet_replica.cc:333] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.575596 12725 raft_consensus.cc:2243] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.576150 12725 raft_consensus.cc:2272] T b5a06b42cc294ea4aa15d4ecb7ce088b P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.578792 12725 tablet_replica.cc:333] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.579909 12725 raft_consensus.cc:2243] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.580518 12725 raft_consensus.cc:2272] T 7f757dd6e4b94a8a84bfaddd602685e4 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.577319 22374 consensus_peers.cc:597] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.583446 12725 tablet_replica.cc:333] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.584306 12725 raft_consensus.cc:2243] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.584800 12725 raft_consensus.cc:2272] T f7f82cbc4e8544bbac74ba96797d687c P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.586990 12725 tablet_replica.cc:333] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.587602 12725 raft_consensus.cc:2243] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.588038 12725 raft_consensus.cc:2272] T 01b83067f9fa498487aeed4f60e8bfde P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.589984 12725 tablet_replica.cc:333] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.590727 12725 raft_consensus.cc:2243] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.591612 12725 raft_consensus.cc:2272] T 2144e27a177e41b0b22e474d9f209102 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.593755 12725 tablet_replica.cc:333] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.594398 12725 raft_consensus.cc:2243] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.594816 12725 raft_consensus.cc:2272] T ffb774f42e99437f8180b3d9e89a9cf5 P bb26db1bd5a64429848b5d63b157c6f0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.597201 12725 tablet_replica.cc:333] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.597780 12725 raft_consensus.cc:2243] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.598636 12725 raft_consensus.cc:2272] T 01099e972e9b4d08a64552be3fd52276 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.600752 12725 tablet_replica.cc:333] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.601311 12725 raft_consensus.cc:2243] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.601728 12725 raft_consensus.cc:2272] T 6cadc140203246b088e6e2883bbe1977 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.603789 12725 tablet_replica.cc:333] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.604339 12725 raft_consensus.cc:2243] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.605172 12725 raft_consensus.cc:2272] T 3a4c4a9cd1164b7997f7188f9da9f70b P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.607223 12725 tablet_replica.cc:333] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.607744 12725 raft_consensus.cc:2243] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.608565 12725 raft_consensus.cc:2272] T ade9dda3178e42c783bc93e5deec21b7 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.610632 12725 tablet_replica.cc:333] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.611133 12725 raft_consensus.cc:2243] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.611575 12725 raft_consensus.cc:2272] T 878ae1fea9154e7088fdba8930c5eb23 P bb26db1bd5a64429848b5d63b157c6f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.613652 12725 tablet_replica.cc:333] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.614274 12725 raft_consensus.cc:2243] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.614677 12725 raft_consensus.cc:2272] T c87912ee2e4f46f1b0eb4e574255f31e P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.615319 22300 consensus_peers.cc:597] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.616775 12725 tablet_replica.cc:333] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.617403 12725 raft_consensus.cc:2243] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.618315 12725 raft_consensus.cc:2272] T d3a61ae21bab486cbfa407942fdeac35 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.619237 22300 consensus_peers.cc:597] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.620913 12725 tablet_replica.cc:333] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.621533 12725 raft_consensus.cc:2243] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.622618 12725 raft_consensus.cc:2272] T 5106df2a56c74793aad5538c83a79f61 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.625547 12725 tablet_replica.cc:333] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0: stopping tablet replica
I20260709 13:08:38.626256 12725 raft_consensus.cc:2243] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:38.626224 22300 consensus_peers.cc:597] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.626904 12725 raft_consensus.cc:2272] T fcf4b5402e45470c81a3389d815a9286 P bb26db1bd5a64429848b5d63b157c6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.627568 22374 consensus_peers.cc:597] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.648626 22374 consensus_peers.cc:597] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.656316 22374 consensus_peers.cc:597] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.687480 22300 consensus_peers.cc:597] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.715215 12725 tablet_server.cc:196] TabletServer@127.12.109.66:0 shutdown complete.
W20260709 13:08:38.760313 22300 consensus_peers.cc:597] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:37225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.763589 22374 consensus_peers.cc:597] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:37225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.811399 22374 consensus_peers.cc:597] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:37225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.822918 12725 tablet_server.cc:179] TabletServer@127.12.109.67:0 shutting down...
W20260709 13:08:38.828555 22375 consensus_peers.cc:597] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:38.869655 22375 consensus_peers.cc:597] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.871236 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:38.872292 12725 tablet_replica.cc:333] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.873422 12725 raft_consensus.cc:2243] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.874154 12725 raft_consensus.cc:2272] T a91b5edf8c9e4a71a195e0f796d439f3 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.876940 12725 tablet_replica.cc:333] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.877725 12725 raft_consensus.cc:2243] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.878362 12725 raft_consensus.cc:2272] T 2d9ed71058164d51a8aa5b9980536bd8 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.881009 12725 tablet_replica.cc:333] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.881616 12725 raft_consensus.cc:2243] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.882520 12725 raft_consensus.cc:2272] T 57dfb3dc107a469c880393583ee0c218 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.884667 12725 tablet_replica.cc:333] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.885268 12725 raft_consensus.cc:2243] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.885694 12725 raft_consensus.cc:2272] T 42127cae8f494cb081ec04febe577130 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.887869 12725 tablet_replica.cc:333] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.888482 12725 raft_consensus.cc:2243] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.888934 12725 raft_consensus.cc:2272] T dc4bb1a452944c59aa3dc6717c52be6a P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.891019 12725 tablet_replica.cc:333] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.891602 12725 raft_consensus.cc:2243] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.892021 12725 raft_consensus.cc:2272] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.895022 12725 tablet_replica.cc:333] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.895908 12725 raft_consensus.cc:2243] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.896466 12725 raft_consensus.cc:2272] T 7d38314054d8474eb00ca91785ea945d P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.899294 12725 tablet_replica.cc:333] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.900096 12725 raft_consensus.cc:2243] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.900573 12725 raft_consensus.cc:2272] T 17e65d47217943239d60b88b4b70ba68 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.903124 12725 tablet_replica.cc:333] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.903908 12725 raft_consensus.cc:2243] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.904413 12725 raft_consensus.cc:2272] T 3e49aee5d99347029a0f5e51fe01b896 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.906828 12725 tablet_replica.cc:333] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.907531 12725 raft_consensus.cc:2243] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.908097 12725 raft_consensus.cc:2272] T 214d3c5d47d848af997da7647c556b85 P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.908861 22374 consensus_peers.cc:597] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Couldn't send request to peer bb26db1bd5a64429848b5d63b157c6f0. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:37225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.910533 12725 tablet_replica.cc:333] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.911191 12725 raft_consensus.cc:2243] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.911566 12725 raft_consensus.cc:2272] T 86f9732bc4d24ee3a8e261358eba3a21 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.913661 12725 tablet_replica.cc:333] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.914269 12725 raft_consensus.cc:2243] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.914644 12725 raft_consensus.cc:2272] T 1cd9f843a01847a4b475b6dbcb704a6f P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.916712 12725 tablet_replica.cc:333] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.917316 12725 raft_consensus.cc:2243] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.918391 12725 raft_consensus.cc:2272] T 40cf2d6d73fe4593b96ead779d12207e P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.921283 12725 tablet_replica.cc:333] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.922212 12725 raft_consensus.cc:2243] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.923146 12725 raft_consensus.cc:2272] T 0418b53b8c2e4786b445d55f0ea5a723 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.925235 12725 tablet_replica.cc:333] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.925818 12725 raft_consensus.cc:2243] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.926316 12725 raft_consensus.cc:2272] T 808e97b948be44ee895c060c4329964e P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.929047 12725 tablet_replica.cc:333] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.929913 12725 raft_consensus.cc:2243] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.930480 12725 raft_consensus.cc:2272] T cd676d7998314b3b9964705206bd4bcb P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.932469 12725 tablet_replica.cc:333] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.933076 12725 raft_consensus.cc:2243] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.933809 12725 raft_consensus.cc:2272] T c9cc1b69991141439a4ee9459859b4a6 P 93815010a884438c86fc6479359b69e8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.935724 12725 tablet_replica.cc:333] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.936236 12725 raft_consensus.cc:2243] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.936964 12725 raft_consensus.cc:2272] T b5a06b42cc294ea4aa15d4ecb7ce088b P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.938989 12725 tablet_replica.cc:333] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.939731 12725 raft_consensus.cc:2243] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.940781 12725 raft_consensus.cc:2272] T 7f757dd6e4b94a8a84bfaddd602685e4 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.943552 12725 tablet_replica.cc:333] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.944386 12725 raft_consensus.cc:2243] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.945451 12725 raft_consensus.cc:2272] T f7f82cbc4e8544bbac74ba96797d687c P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.947892 22375 consensus_peers.cc:597] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.948295 12725 tablet_replica.cc:333] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.949232 12725 raft_consensus.cc:2243] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.950400 12725 raft_consensus.cc:2272] T 01b83067f9fa498487aeed4f60e8bfde P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.952682 12725 tablet_replica.cc:333] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.953202 12725 raft_consensus.cc:2243] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.953584 12725 raft_consensus.cc:2272] T f0de2f3edc854918bb42aa68cdc2c9fa P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.955466 12725 tablet_replica.cc:333] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.955999 12725 raft_consensus.cc:2243] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.956408 12725 raft_consensus.cc:2272] T 2144e27a177e41b0b22e474d9f209102 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.958531 12725 tablet_replica.cc:333] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.959033 12725 raft_consensus.cc:2243] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.959470 12725 raft_consensus.cc:2272] T 96356b5b69fc405cb1678dbfaec2df1f P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.961728 12725 tablet_replica.cc:333] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.962335 12725 raft_consensus.cc:2243] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.962716 12725 raft_consensus.cc:2272] T 691ad2369e00484995e87d12f574dfdc P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.964483 12725 tablet_replica.cc:333] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.965019 12725 raft_consensus.cc:2243] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.965416 12725 raft_consensus.cc:2272] T d3a61ae21bab486cbfa407942fdeac35 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.967669 12725 tablet_replica.cc:333] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.968268 12725 raft_consensus.cc:2243] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.968804 12725 raft_consensus.cc:2272] T cf086b080a8c46ff809e2b9a14bdacae P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.971271 12725 tablet_replica.cc:333] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.971863 12725 raft_consensus.cc:2243] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.972337 12725 raft_consensus.cc:2272] T ffb774f42e99437f8180b3d9e89a9cf5 P 93815010a884438c86fc6479359b69e8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.974706 12725 tablet_replica.cc:333] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.975327 12725 raft_consensus.cc:2243] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.975816 12725 raft_consensus.cc:2272] T 3a4c4a9cd1164b7997f7188f9da9f70b P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.978188 12725 tablet_replica.cc:333] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.978772 12725 raft_consensus.cc:2243] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.979238 12725 raft_consensus.cc:2272] T 01099e972e9b4d08a64552be3fd52276 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.981418 12725 tablet_replica.cc:333] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.982026 12725 raft_consensus.cc:2243] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.982702 12725 raft_consensus.cc:2272] T 6cadc140203246b088e6e2883bbe1977 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.985406 12725 tablet_replica.cc:333] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.986191 12725 raft_consensus.cc:2243] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.987360 12725 raft_consensus.cc:2272] T 814c209a7aff4b11bdc62505cc29c286 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.989840 12725 tablet_replica.cc:333] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.990502 12725 raft_consensus.cc:2243] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.991006 12725 raft_consensus.cc:2272] T 5106df2a56c74793aad5538c83a79f61 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.993891 12725 tablet_replica.cc:333] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.994500 12725 raft_consensus.cc:2243] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.995385 12725 raft_consensus.cc:2272] T fcf4b5402e45470c81a3389d815a9286 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.997825 12725 tablet_replica.cc:333] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:38.998435 12725 raft_consensus.cc:2243] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.999233 12725 raft_consensus.cc:2272] T ec8f7472c97049ca98e597c7d4b4eb1c P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.001266 12725 tablet_replica.cc:333] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:39.001793 12725 raft_consensus.cc:2243] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.002733 12725 raft_consensus.cc:2272] T 96821e05d1924572a73e450bc7dcb265 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.004824 12725 tablet_replica.cc:333] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:39.005360 12725 raft_consensus.cc:2243] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.006212 12725 raft_consensus.cc:2272] T c995e1a8ce9c48aab8235f258f77d080 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.008365 12725 tablet_replica.cc:333] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:39.008991 12725 raft_consensus.cc:2243] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.009544 12725 raft_consensus.cc:2272] T 30e6db323fdb4762a2e92b729b051111 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.012138 12725 tablet_replica.cc:333] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:39.012872 12725 raft_consensus.cc:2243] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.013434 12725 raft_consensus.cc:2272] T 5e95f83f20904ff692e70a1872c3e77a P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.016211 12725 tablet_replica.cc:333] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:39.016896 12725 raft_consensus.cc:2243] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.017416 12725 raft_consensus.cc:2272] T 2076c24c8d1d47e6b3e4ece63fae1597 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.020025 12725 tablet_replica.cc:333] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:39.020761 12725 raft_consensus.cc:2243] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.021615 12725 raft_consensus.cc:2272] T c87912ee2e4f46f1b0eb4e574255f31e P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.023931 12725 tablet_replica.cc:333] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:39.024578 12725 raft_consensus.cc:2243] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.025089 12725 raft_consensus.cc:2272] T fdbf4608aa9f48d68cb77a4907dff6a6 P 93815010a884438c86fc6479359b69e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:39.027288 22375 consensus_peers.cc:597] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.027797 12725 tablet_replica.cc:333] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:39.028483 12725 raft_consensus.cc:2243] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.028942 12725 raft_consensus.cc:2272] T 3808206385a0458abc04050aeab1cea7 P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.031021 12725 tablet_replica.cc:333] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8: stopping tablet replica
I20260709 13:08:39.031517 12725 raft_consensus.cc:2243] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.031936 12725 raft_consensus.cc:2272] T a5e8f1fb740446b8b42c60afd2e7c3eb P 93815010a884438c86fc6479359b69e8 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:39.079263 22375 consensus_peers.cc:597] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.111023 12725 tablet_server.cc:196] TabletServer@127.12.109.67:0 shutdown complete.
W20260709 13:08:39.128981 22375 consensus_peers.cc:597] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.67:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.167698 23404 raft_consensus.cc:493] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:39.168340 23404 raft_consensus.cc:515] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
W20260709 13:08:39.168632 22375 consensus_peers.cc:597] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.67:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:39.169224 22375 consensus_peers.cc:597] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.67:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.170759 23404 leader_election.cc:290] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
W20260709 13:08:39.176784 22375 leader_election.cc:336] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:39.177405 22375 leader_election.cc:336] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Network error: Client connection negotiation failed: client connection to 127.12.109.67:34889: connect: Connection refused (error 111)
I20260709 13:08:39.177804 22375 leader_election.cc:304] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:39.178471 23404 raft_consensus.cc:2749] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:39.184046 23404 raft_consensus.cc:493] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:39.184588 23404 raft_consensus.cc:515] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:39.186689 23404 leader_election.cc:290] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
W20260709 13:08:39.191481 22374 leader_election.cc:336] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Network error: Client connection negotiation failed: client connection to 127.12.109.66:37225: connect: Connection refused (error 111)
W20260709 13:08:39.196372 22375 leader_election.cc:336] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:39.196754 22375 leader_election.cc:304] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:39.197392 23404 raft_consensus.cc:2749] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:08:39.198283 22375 consensus_peers.cc:597] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.67:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:39.201318 22375 consensus_peers.cc:597] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.67:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.203156 23404 raft_consensus.cc:493] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:39.203709 23404 raft_consensus.cc:515] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:39.205897 23404 leader_election.cc:290] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
W20260709 13:08:39.210156 22375 leader_election.cc:336] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
W20260709 13:08:39.210757 22374 leader_election.cc:336] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Network error: Client connection negotiation failed: client connection to 127.12.109.66:37225: connect: Connection refused (error 111)
I20260709 13:08:39.211262 22374 leader_election.cc:304] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:39.212133 23406 raft_consensus.cc:2749] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:39.218631 12725 tablet_server.cc:179] TabletServer@127.12.109.68:0 shutting down...
I20260709 13:08:39.232667 23404 raft_consensus.cc:493] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:39.233505 23404 raft_consensus.cc:515] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:39.238613 23404 leader_election.cc:290] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
W20260709 13:08:39.246658 22375 leader_election.cc:336] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Network error: Client connection negotiation failed: client connection to 127.12.109.67:34889: connect: Connection refused (error 111)
W20260709 13:08:39.247366 22375 leader_election.cc:336] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:39.247828 22375 leader_election.cc:304] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:39.248903 23440 raft_consensus.cc:2749] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:08:39.263906 22375 consensus_peers.cc:597] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.67:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.277347 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:39.278439 12725 tablet_replica.cc:333] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.278476 23404 raft_consensus.cc:493] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:39.279255 12725 raft_consensus.cc:2243] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.279242 23404 raft_consensus.cc:515] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93815010a884438c86fc6479359b69e8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 34889 } } peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } }
I20260709 13:08:39.279960 12725 raft_consensus.cc:2272] T a51f4b4253994202856577a87dd8d61c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.281325 23404 leader_election.cc:290] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889), 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399)
I20260709 13:08:39.283305 12725 tablet_replica.cc:333] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.284359 12725 raft_consensus.cc:2243] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.285502 12725 raft_consensus.cc:2272] T f0de2f3edc854918bb42aa68cdc2c9fa P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:39.286330 22375 leader_election.cc:336] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Network error: Client connection negotiation failed: client connection to 127.12.109.67:34889: connect: Connection refused (error 111)
W20260709 13:08:39.287364 22375 leader_election.cc:336] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:39.287765 22375 leader_election.cc:304] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, 93815010a884438c86fc6479359b69e8
I20260709 13:08:39.288445 12725 tablet_replica.cc:333] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.288520 23404 raft_consensus.cc:2749] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:39.289417 12725 raft_consensus.cc:2243] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.289912 12725 raft_consensus.cc:2272] T 17e65d47217943239d60b88b4b70ba68 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:39.290457 22375 consensus_peers.cc:597] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc -> Peer 93815010a884438c86fc6479359b69e8 (127.12.109.67:34889): Couldn't send request to peer 93815010a884438c86fc6479359b69e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.67:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.293114 12725 tablet_replica.cc:333] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.294044 12725 raft_consensus.cc:2243] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.294626 12725 raft_consensus.cc:2272] T f9b737a6d0d24895b749f55ac078ac5f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.296756 12725 tablet_replica.cc:333] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.297317 12725 raft_consensus.cc:2243] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.298156 12725 raft_consensus.cc:2272] T 2d9ed71058164d51a8aa5b9980536bd8 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.300225 12725 tablet_replica.cc:333] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.300798 12725 raft_consensus.cc:2243] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.301170 12725 raft_consensus.cc:2272] T 16a08df7f7be4e808ba69787809be0d3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.303169 12725 tablet_replica.cc:333] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.303738 12725 raft_consensus.cc:2243] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.304137 12725 raft_consensus.cc:2272] T 59f7f96ef51941e39bc97c0b769c7770 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.306310 12725 tablet_replica.cc:333] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.306893 12725 raft_consensus.cc:2243] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.307644 12725 raft_consensus.cc:2272] T 3e49aee5d99347029a0f5e51fe01b896 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.309727 12725 tablet_replica.cc:333] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.310374 12725 raft_consensus.cc:2243] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.310766 12725 raft_consensus.cc:2272] T fb4bb4b406fa4f4f9a55ba3b7c005cf0 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.312759 12725 tablet_replica.cc:333] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.313309 12725 raft_consensus.cc:2243] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.313706 12725 raft_consensus.cc:2272] T ffdd1d227bbe48c8a46c4fefd02eab0b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.316143 12725 tablet_replica.cc:333] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.316751 12725 raft_consensus.cc:2243] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.317806 12725 raft_consensus.cc:2272] T fa44244e76604221bcbd08e0f3bbdb82 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.320820 12725 tablet_replica.cc:333] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.321507 12725 raft_consensus.cc:2243] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.322034 12725 raft_consensus.cc:2272] T 40cf2d6d73fe4593b96ead779d12207e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.324968 12725 tablet_replica.cc:333] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.325631 12725 raft_consensus.cc:2243] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.326234 12725 raft_consensus.cc:2272] T 0418b53b8c2e4786b445d55f0ea5a723 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.328583 12725 tablet_replica.cc:333] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.329100 12725 raft_consensus.cc:2243] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.329476 12725 raft_consensus.cc:2272] T c995e1a8ce9c48aab8235f258f77d080 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.331553 12725 tablet_replica.cc:333] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.332094 12725 raft_consensus.cc:2243] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.332569 12725 raft_consensus.cc:2272] T 96821e05d1924572a73e450bc7dcb265 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.334650 12725 tablet_replica.cc:333] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.335153 12725 raft_consensus.cc:2243] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.335661 12725 raft_consensus.cc:2272] T 01b83067f9fa498487aeed4f60e8bfde P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.337790 12725 tablet_replica.cc:333] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.338382 12725 raft_consensus.cc:2243] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.339183 12725 raft_consensus.cc:2272] T 808e97b948be44ee895c060c4329964e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.341109 12725 tablet_replica.cc:333] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.341692 12725 raft_consensus.cc:2243] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.342175 12725 raft_consensus.cc:2272] T a39127728d4e48a9aeb4e71e4c7e1518 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.344105 12725 tablet_replica.cc:333] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.344700 12725 raft_consensus.cc:2243] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.345114 12725 raft_consensus.cc:2272] T 0d86dab181da42f79842cbdbd4fa5b05 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.347067 12725 tablet_replica.cc:333] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.347575 12725 raft_consensus.cc:2243] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.348024 12725 raft_consensus.cc:2272] T c9cc1b69991141439a4ee9459859b4a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.350395 12725 tablet_replica.cc:333] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.351094 12725 raft_consensus.cc:2243] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.351953 12725 raft_consensus.cc:2272] T c0b51a3b1dc444319751dd92054d5dc3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.354033 12725 tablet_replica.cc:333] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.354745 12725 raft_consensus.cc:2243] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.355561 12725 raft_consensus.cc:2272] T 96356b5b69fc405cb1678dbfaec2df1f P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.357632 12725 tablet_replica.cc:333] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.358258 12725 raft_consensus.cc:2243] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.358695 12725 raft_consensus.cc:2272] T 691ad2369e00484995e87d12f574dfdc P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.360540 12725 tablet_replica.cc:333] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.361119 12725 raft_consensus.cc:2243] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.361562 12725 raft_consensus.cc:2272] T cf086b080a8c46ff809e2b9a14bdacae P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.363644 12725 tablet_replica.cc:333] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.364257 12725 raft_consensus.cc:2243] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.365128 12725 raft_consensus.cc:2272] T 6cadc140203246b088e6e2883bbe1977 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.367267 12725 tablet_replica.cc:333] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.367877 12725 raft_consensus.cc:2243] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.368906 12725 raft_consensus.cc:2272] T 30e6db323fdb4762a2e92b729b051111 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.368944 23404 raft_consensus.cc:493] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 652d70a0b765447792ab9b4383712607)
I20260709 13:08:39.369663 23404 raft_consensus.cc:515] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fc9e7fc4b34621ae1cd3e1621b89dc" member_type: VOTER last_known_addr { host: "127.12.109.68" port: 41261 } } peers { permanent_uuid: "652d70a0b765447792ab9b4383712607" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 46399 } } peers { permanent_uuid: "bb26db1bd5a64429848b5d63b157c6f0" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37225 } }
I20260709 13:08:39.371629 12725 tablet_replica.cc:333] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.371882 23404 leader_election.cc:290] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399), bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225)
I20260709 13:08:39.372622 12725 raft_consensus.cc:2243] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.373799 12725 raft_consensus.cc:2272] T 5e95f83f20904ff692e70a1872c3e77a P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:39.376507 22375 leader_election.cc:336] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 652d70a0b765447792ab9b4383712607 (127.12.109.65:46399): Network error: Client connection negotiation failed: client connection to 127.12.109.65:46399: connect: Connection refused (error 111)
I20260709 13:08:39.376881 12725 tablet_replica.cc:333] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
W20260709 13:08:39.377044 22374 leader_election.cc:336] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bb26db1bd5a64429848b5d63b157c6f0 (127.12.109.66:37225): Network error: Client connection negotiation failed: client connection to 127.12.109.66:37225: connect: Connection refused (error 111)
I20260709 13:08:39.377503 22374 leader_election.cc:304] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67fc9e7fc4b34621ae1cd3e1621b89dc; no voters: 652d70a0b765447792ab9b4383712607, bb26db1bd5a64429848b5d63b157c6f0
I20260709 13:08:39.377861 12725 raft_consensus.cc:2243] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.378333 23404 raft_consensus.cc:2749] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:39.378545 12725 raft_consensus.cc:2272] T ade9dda3178e42c783bc93e5deec21b7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.380932 12725 tablet_replica.cc:333] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.381498 12725 raft_consensus.cc:2243] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.382460 12725 raft_consensus.cc:2272] T a91b5edf8c9e4a71a195e0f796d439f3 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.384596 12725 tablet_replica.cc:333] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.385113 12725 raft_consensus.cc:2243] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.385496 12725 raft_consensus.cc:2272] T fd464ed1f28f42ccad10b0df649c7ce7 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.387537 12725 tablet_replica.cc:333] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.388032 12725 raft_consensus.cc:2243] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.388818 12725 raft_consensus.cc:2272] T 2076c24c8d1d47e6b3e4ece63fae1597 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.390965 12725 tablet_replica.cc:333] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.391479 12725 raft_consensus.cc:2243] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.391889 12725 raft_consensus.cc:2272] T 4f204603e14c438bb549d1116eaac788 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.393894 12725 tablet_replica.cc:333] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.394505 12725 raft_consensus.cc:2243] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.395030 12725 raft_consensus.cc:2272] T ec8f7472c97049ca98e597c7d4b4eb1c P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.397131 12725 tablet_replica.cc:333] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.397697 12725 raft_consensus.cc:2243] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.398232 12725 raft_consensus.cc:2272] T 4ad7895d69404083ba56a16062a8ed20 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.400770 12725 tablet_replica.cc:333] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.401597 12725 raft_consensus.cc:2243] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.402578 12725 raft_consensus.cc:2272] T fdbf4608aa9f48d68cb77a4907dff6a6 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.404729 12725 tablet_replica.cc:333] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.405296 12725 raft_consensus.cc:2243] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.405720 12725 raft_consensus.cc:2272] T a5e8f1fb740446b8b42c60afd2e7c3eb P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.407706 12725 tablet_replica.cc:333] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.408324 12725 raft_consensus.cc:2243] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.408783 12725 raft_consensus.cc:2272] T 814c209a7aff4b11bdc62505cc29c286 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.410858 12725 tablet_replica.cc:333] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.411410 12725 raft_consensus.cc:2243] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.411821 12725 raft_consensus.cc:2272] T 5106df2a56c74793aad5538c83a79f61 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.413787 12725 tablet_replica.cc:333] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.414373 12725 raft_consensus.cc:2243] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.414808 12725 raft_consensus.cc:2272] T c87912ee2e4f46f1b0eb4e574255f31e P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.416826 12725 tablet_replica.cc:333] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.417361 12725 raft_consensus.cc:2243] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.417780 12725 raft_consensus.cc:2272] T 62cc2afc26f14239af465ce79a044163 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.419902 12725 tablet_replica.cc:333] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.420469 12725 raft_consensus.cc:2243] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.420915 12725 raft_consensus.cc:2272] T b5a06b42cc294ea4aa15d4ecb7ce088b P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.422894 12725 tablet_replica.cc:333] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.423404 12725 raft_consensus.cc:2243] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.423857 12725 raft_consensus.cc:2272] T 7f757dd6e4b94a8a84bfaddd602685e4 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.425782 12725 tablet_replica.cc:333] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.426304 12725 raft_consensus.cc:2243] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.427201 12725 raft_consensus.cc:2272] T 42127cae8f494cb081ec04febe577130 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.429347 12725 tablet_replica.cc:333] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc: stopping tablet replica
I20260709 13:08:39.429829 12725 raft_consensus.cc:2243] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.430610 12725 raft_consensus.cc:2272] T 878ae1fea9154e7088fdba8930c5eb23 P 67fc9e7fc4b34621ae1cd3e1621b89dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.503696 12725 tablet_server.cc:196] TabletServer@127.12.109.68:0 shutdown complete.
I20260709 13:08:39.578270 12725 master.cc:562] Master@127.12.109.126:37329 shutting down...
I20260709 13:08:39.606963 12725 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.607637 12725 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.608021 12725 tablet_replica.cc:333] T 00000000000000000000000000000000 P c0557a4af02b4c25b02595b71c60436b: stopping tablet replica
I20260709 13:08:39.631383 12725 master.cc:584] Master@127.12.109.126:37329 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (37628 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:08:39.736783 12725 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.109.126:40143
I20260709 13:08:39.738396 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:39.745167 23441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:39.745455 23444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:39.745661 23442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:39.746763 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:39.748047 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:39.748246 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:39.748422 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602519748404 us; error 0 us; skew 500 ppm
I20260709 13:08:39.748997 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:39.751933 12725 webserver.cc:533] Webserver started at http://127.12.109.126:46537/ using document root <none> and password file <none>
I20260709 13:08:39.752467 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:39.752676 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:39.752959 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:39.754287 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/master-0-root/instance:
uuid: "acbfdf33d795451fbe6e1122a6868f8d"
format_stamp: "Formatted at 2026-07-09 13:08:39 on dist-test-slave-zhdc"
I20260709 13:08:39.759087 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:08:39.762806 23449 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:39.763725 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:08:39.764128 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/master-0-root
uuid: "acbfdf33d795451fbe6e1122a6868f8d"
format_stamp: "Formatted at 2026-07-09 13:08:39 on dist-test-slave-zhdc"
I20260709 13:08:39.764492 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:39.782972 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:39.784392 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:39.828677 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.126:40143
I20260709 13:08:39.828787 23500 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.126:40143 every 8 connection(s)
I20260709 13:08:39.833123 23501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:39.845449 23501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d: Bootstrap starting.
I20260709 13:08:39.851162 23501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:39.856163 23501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d: No bootstrap required, opened a new log
I20260709 13:08:39.858686 23501 raft_consensus.cc:359] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acbfdf33d795451fbe6e1122a6868f8d" member_type: VOTER }
I20260709 13:08:39.859114 23501 raft_consensus.cc:385] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:39.859325 23501 raft_consensus.cc:740] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acbfdf33d795451fbe6e1122a6868f8d, State: Initialized, Role: FOLLOWER
I20260709 13:08:39.859961 23501 consensus_queue.cc:260] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acbfdf33d795451fbe6e1122a6868f8d" member_type: VOTER }
I20260709 13:08:39.860425 23501 raft_consensus.cc:399] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:08:39.860631 23501 raft_consensus.cc:493] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:08:39.860844 23501 raft_consensus.cc:3060] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:39.866587 23501 raft_consensus.cc:515] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acbfdf33d795451fbe6e1122a6868f8d" member_type: VOTER }
I20260709 13:08:39.867175 23501 leader_election.cc:304] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acbfdf33d795451fbe6e1122a6868f8d; no voters: 
I20260709 13:08:39.868430 23501 leader_election.cc:290] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:08:39.868862 23504 raft_consensus.cc:2804] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:39.870355 23504 raft_consensus.cc:697] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [term 1 LEADER]: Becoming Leader. State: Replica: acbfdf33d795451fbe6e1122a6868f8d, State: Running, Role: LEADER
I20260709 13:08:39.871091 23504 consensus_queue.cc:237] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acbfdf33d795451fbe6e1122a6868f8d" member_type: VOTER }
I20260709 13:08:39.871827 23501 sys_catalog.cc:565] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:08:39.876665 23506 sys_catalog.cc:455] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader acbfdf33d795451fbe6e1122a6868f8d. Latest consensus state: current_term: 1 leader_uuid: "acbfdf33d795451fbe6e1122a6868f8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acbfdf33d795451fbe6e1122a6868f8d" member_type: VOTER } }
I20260709 13:08:39.878214 23506 sys_catalog.cc:458] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:39.879359 23505 sys_catalog.cc:455] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acbfdf33d795451fbe6e1122a6868f8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acbfdf33d795451fbe6e1122a6868f8d" member_type: VOTER } }
I20260709 13:08:39.880048 23505 sys_catalog.cc:458] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:39.880380 23512 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:08:39.885716 23512 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:08:39.889756 12725 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:08:39.895605 23512 catalog_manager.cc:1383] Generated new cluster ID: 227b90aefe0b42d092684146806dea41
I20260709 13:08:39.895944 23512 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:08:39.911252 23512 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:08:39.912638 23512 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:08:39.924973 23512 catalog_manager.cc:6070] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d: Generated new TSK 0
I20260709 13:08:39.925663 23512 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:08:39.956974 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:39.963896 23522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:39.966106 23523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:39.968611 12725 server_base.cc:1061] running on GCE node
W20260709 13:08:39.968858 23525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:39.970201 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:39.970412 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:39.970541 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602519970529 us; error 0 us; skew 500 ppm
I20260709 13:08:39.971328 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:39.975103 12725 webserver.cc:533] Webserver started at http://127.12.109.65:41111/ using document root <none> and password file <none>
I20260709 13:08:39.975958 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:39.976388 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:39.976697 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:39.978237 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-0-root/instance:
uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
format_stamp: "Formatted at 2026-07-09 13:08:39 on dist-test-slave-zhdc"
I20260709 13:08:39.987321 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:08:39.992411 23530 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:39.993502 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:08:39.993934 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-0-root
uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
format_stamp: "Formatted at 2026-07-09 13:08:39 on dist-test-slave-zhdc"
I20260709 13:08:39.994401 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:40.032927 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:40.034456 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:40.036096 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:40.038796 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:40.039026 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.039297 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:40.039489 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.087164 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.65:45637
I20260709 13:08:40.087347 23592 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.65:45637 every 8 connection(s)
I20260709 13:08:40.092839 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:40.101827 23597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:40.103690 23598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:40.110316 23593 heartbeater.cc:344] Connected to a master server at 127.12.109.126:40143
I20260709 13:08:40.110329 12725 server_base.cc:1061] running on GCE node
W20260709 13:08:40.110368 23600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:40.111094 23593 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:40.112349 23593 heartbeater.cc:507] Master 127.12.109.126:40143 requested a full tablet report, sending...
I20260709 13:08:40.116303 23466 ts_manager.cc:194] Registered new tserver with Master: 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637)
I20260709 13:08:40.117851 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:40.118283 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:40.118479 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602520118456 us; error 0 us; skew 500 ppm
I20260709 13:08:40.118969 23466 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50174
I20260709 13:08:40.119139 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:40.122272 12725 webserver.cc:533] Webserver started at http://127.12.109.66:40781/ using document root <none> and password file <none>
I20260709 13:08:40.122800 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:40.123032 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:40.123337 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:40.124658 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-1-root/instance:
uuid: "bcb961e4f2944eb585f6bad2a758fc79"
format_stamp: "Formatted at 2026-07-09 13:08:40 on dist-test-slave-zhdc"
I20260709 13:08:40.129662 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:08:40.133713 23605 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.134763 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:08:40.135040 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-1-root
uuid: "bcb961e4f2944eb585f6bad2a758fc79"
format_stamp: "Formatted at 2026-07-09 13:08:40 on dist-test-slave-zhdc"
I20260709 13:08:40.135315 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:40.153627 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:40.155110 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:40.156729 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:40.159359 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:40.159614 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.159878 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:40.160055 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.207150 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.66:41483
I20260709 13:08:40.207267 23667 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.66:41483 every 8 connection(s)
I20260709 13:08:40.212829 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:40.221848 23671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:40.221851 23672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:40.226164 23668 heartbeater.cc:344] Connected to a master server at 127.12.109.126:40143
I20260709 13:08:40.226730 23668 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:40.227306 12725 server_base.cc:1061] running on GCE node
W20260709 13:08:40.227603 23675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:40.227835 23668 heartbeater.cc:507] Master 127.12.109.126:40143 requested a full tablet report, sending...
I20260709 13:08:40.228888 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:40.229190 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:40.229372 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602520229356 us; error 0 us; skew 500 ppm
I20260709 13:08:40.230122 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:40.230669 23466 ts_manager.cc:194] Registered new tserver with Master: bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:40.232811 23466 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50186
I20260709 13:08:40.233059 12725 webserver.cc:533] Webserver started at http://127.12.109.67:33371/ using document root <none> and password file <none>
I20260709 13:08:40.233834 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:40.234115 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:40.234422 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:40.235737 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-2-root/instance:
uuid: "1f00e9da593145e68587c3779ae8acae"
format_stamp: "Formatted at 2026-07-09 13:08:40 on dist-test-slave-zhdc"
I20260709 13:08:40.240593 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:08:40.244349 23679 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.245546 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:08:40.245918 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-2-root
uuid: "1f00e9da593145e68587c3779ae8acae"
format_stamp: "Formatted at 2026-07-09 13:08:40 on dist-test-slave-zhdc"
I20260709 13:08:40.246331 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:40.268158 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:40.269501 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:40.270889 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:40.273423 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:40.273664 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.273974 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:40.274233 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:08:40.319869 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.67:41983
I20260709 13:08:40.319977 23741 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.67:41983 every 8 connection(s)
I20260709 13:08:40.335073 23742 heartbeater.cc:344] Connected to a master server at 127.12.109.126:40143
I20260709 13:08:40.335631 23742 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:40.336519 23742 heartbeater.cc:507] Master 127.12.109.126:40143 requested a full tablet report, sending...
I20260709 13:08:40.338825 23466 ts_manager.cc:194] Registered new tserver with Master: 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:40.339334 12725 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015892404s
I20260709 13:08:40.340741 23466 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50196
I20260709 13:08:40.368973 23466 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50202:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:08:40.372184 23466 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:08:40.467061 23707 tablet_service.cc:1511] Processing CreateTablet for tablet 0fc469ae092d40f5aca5bef046fa8cce (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:08:40.468820 23707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fc469ae092d40f5aca5bef046fa8cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.468874 23705 tablet_service.cc:1511] Processing CreateTablet for tablet b96b8ed446fa4b66930a590bda784ff1 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:08:40.470503 23705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b96b8ed446fa4b66930a590bda784ff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.473886 23704 tablet_service.cc:1511] Processing CreateTablet for tablet 7a0c51f78c384031b9ffa8d79085d7e4 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:08:40.475593 23704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a0c51f78c384031b9ffa8d79085d7e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.475286 23706 tablet_service.cc:1511] Processing CreateTablet for tablet 540c29b91f164ba881d3bd49a33653b6 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:08:40.476758 23706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 540c29b91f164ba881d3bd49a33653b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.487470 23633 tablet_service.cc:1511] Processing CreateTablet for tablet 0fc469ae092d40f5aca5bef046fa8cce (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:08:40.489167 23633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fc469ae092d40f5aca5bef046fa8cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.491395 23632 tablet_service.cc:1511] Processing CreateTablet for tablet 540c29b91f164ba881d3bd49a33653b6 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:08:40.493016 23632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 540c29b91f164ba881d3bd49a33653b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.493726 23703 tablet_service.cc:1511] Processing CreateTablet for tablet c31d9cae460a432bb196bf822834bb01 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:08:40.495448 23703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c31d9cae460a432bb196bf822834bb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.495324 23631 tablet_service.cc:1511] Processing CreateTablet for tablet b96b8ed446fa4b66930a590bda784ff1 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:08:40.496881 23631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b96b8ed446fa4b66930a590bda784ff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.499343 23630 tablet_service.cc:1511] Processing CreateTablet for tablet 7a0c51f78c384031b9ffa8d79085d7e4 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:08:40.500816 23630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a0c51f78c384031b9ffa8d79085d7e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.503191 23629 tablet_service.cc:1511] Processing CreateTablet for tablet c31d9cae460a432bb196bf822834bb01 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:08:40.504654 23629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c31d9cae460a432bb196bf822834bb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.507483 23702 tablet_service.cc:1511] Processing CreateTablet for tablet d07a9927140e49fc8273c94c155ad1cb (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:08:40.509002 23702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d07a9927140e49fc8273c94c155ad1cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.507360 23628 tablet_service.cc:1511] Processing CreateTablet for tablet d07a9927140e49fc8273c94c155ad1cb (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:08:40.510977 23628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d07a9927140e49fc8273c94c155ad1cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.515717 23558 tablet_service.cc:1511] Processing CreateTablet for tablet 0fc469ae092d40f5aca5bef046fa8cce (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:08:40.517302 23558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fc469ae092d40f5aca5bef046fa8cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.519392 23557 tablet_service.cc:1511] Processing CreateTablet for tablet 540c29b91f164ba881d3bd49a33653b6 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:08:40.522264 23556 tablet_service.cc:1511] Processing CreateTablet for tablet b96b8ed446fa4b66930a590bda784ff1 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:08:40.524537 23555 tablet_service.cc:1511] Processing CreateTablet for tablet 7a0c51f78c384031b9ffa8d79085d7e4 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:08:40.526986 23554 tablet_service.cc:1511] Processing CreateTablet for tablet c31d9cae460a432bb196bf822834bb01 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:08:40.529428 23553 tablet_service.cc:1511] Processing CreateTablet for tablet d07a9927140e49fc8273c94c155ad1cb (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:08:40.532716 23552 tablet_service.cc:1511] Processing CreateTablet for tablet 9eda7f9aaa60469d9c3deed0977252b6 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:08:40.535023 23627 tablet_service.cc:1511] Processing CreateTablet for tablet 9eda7f9aaa60469d9c3deed0977252b6 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:08:40.536566 23627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eda7f9aaa60469d9c3deed0977252b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.544332 23701 tablet_service.cc:1511] Processing CreateTablet for tablet 9eda7f9aaa60469d9c3deed0977252b6 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:08:40.545576 23557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 540c29b91f164ba881d3bd49a33653b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.545989 23701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eda7f9aaa60469d9c3deed0977252b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.546314 23551 tablet_service.cc:1511] Processing CreateTablet for tablet 38a1a8c29abb4f4089bbd7881a879ad6 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:08:40.549342 23707 tablet_service.cc:1511] Processing CreateTablet for tablet 38a1a8c29abb4f4089bbd7881a879ad6 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:08:40.551029 23552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eda7f9aaa60469d9c3deed0977252b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.552091 23707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38a1a8c29abb4f4089bbd7881a879ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.553402 23626 tablet_service.cc:1511] Processing CreateTablet for tablet 38a1a8c29abb4f4089bbd7881a879ad6 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:08:40.554885 23626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38a1a8c29abb4f4089bbd7881a879ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.558712 23551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38a1a8c29abb4f4089bbd7881a879ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.560330 23555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a0c51f78c384031b9ffa8d79085d7e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.562682 23553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d07a9927140e49fc8273c94c155ad1cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.564181 23550 tablet_service.cc:1511] Processing CreateTablet for tablet c2797488d20e4732a4e65c083418840b (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:08:40.565672 23550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2797488d20e4732a4e65c083418840b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.566788 23554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c31d9cae460a432bb196bf822834bb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.570088 23556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b96b8ed446fa4b66930a590bda784ff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.576992 23625 tablet_service.cc:1511] Processing CreateTablet for tablet c2797488d20e4732a4e65c083418840b (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:08:40.580130 23624 tablet_service.cc:1511] Processing CreateTablet for tablet 9eb01d6595f3431faf29c07dcc7c17d8 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:08:40.581909 23625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2797488d20e4732a4e65c083418840b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.589493 23624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eb01d6595f3431faf29c07dcc7c17d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.617358 23549 tablet_service.cc:1511] Processing CreateTablet for tablet 9eb01d6595f3431faf29c07dcc7c17d8 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:08:40.618992 23549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eb01d6595f3431faf29c07dcc7c17d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.619395 23703 tablet_service.cc:1511] Processing CreateTablet for tablet 9eb01d6595f3431faf29c07dcc7c17d8 (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:08:40.620859 23703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eb01d6595f3431faf29c07dcc7c17d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.627153 23707 tablet_service.cc:1511] Processing CreateTablet for tablet c2797488d20e4732a4e65c083418840b (DEFAULT_TABLE table=test-workload [id=20511818cdcc408faae2223d841ef60b]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:08:40.628829 23707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2797488d20e4732a4e65c083418840b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.617269 23758 tablet_bootstrap.cc:492] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:40.651988 23758 tablet_bootstrap.cc:654] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.666316 23758 tablet_bootstrap.cc:492] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8: No bootstrap required, opened a new log
I20260709 13:08:40.668879 23758 ts_tablet_manager.cc:1403] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.004s
I20260709 13:08:40.672557 23758 raft_consensus.cc:359] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.675700 23759 tablet_bootstrap.cc:492] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79: Bootstrap starting.
I20260709 13:08:40.673382 23758 raft_consensus.cc:385] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.676513 23758 raft_consensus.cc:740] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:40.677591 23758 consensus_queue.cc:260] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.680109 23757 tablet_bootstrap.cc:492] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae: Bootstrap starting.
I20260709 13:08:40.753773 23593 heartbeater.cc:499] Master 127.12.109.126:40143 was elected leader, sending a full tablet report...
I20260709 13:08:40.753873 23761 raft_consensus.cc:493] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.756199 23761 raft_consensus.cc:515] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.763118 23758 ts_tablet_manager.cc:1434] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.094s	user 0.024s	sys 0.035s
I20260709 13:08:40.777300 23761 leader_election.cc:290] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:40.779667 23758 tablet_bootstrap.cc:492] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:40.786397 23757 tablet_bootstrap.cc:654] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.788551 23759 tablet_bootstrap.cc:654] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.797441 23758 tablet_bootstrap.cc:654] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.804842 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a0c51f78c384031b9ffa8d79085d7e4" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
W20260709 13:08:40.806685 23532 leader_election.cc:343] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:40.818091 23643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a0c51f78c384031b9ffa8d79085d7e4" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
W20260709 13:08:40.820582 23533 leader_election.cc:343] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:40.821290 23533 leader_election.cc:304] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:40.822674 23762 raft_consensus.cc:2749] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:40.833096 23759 tablet_bootstrap.cc:492] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79: No bootstrap required, opened a new log
I20260709 13:08:40.834481 23759 ts_tablet_manager.cc:1403] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79: Time spent bootstrapping tablet: real 0.159s	user 0.013s	sys 0.023s
I20260709 13:08:40.838019 23759 raft_consensus.cc:359] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:40.838733 23759 raft_consensus.cc:385] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.839001 23759 raft_consensus.cc:740] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Initialized, Role: FOLLOWER
I20260709 13:08:40.839732 23759 consensus_queue.cc:260] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:40.839962 23758 tablet_bootstrap.cc:492] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8: No bootstrap required, opened a new log
I20260709 13:08:40.840906 23758 ts_tablet_manager.cc:1403] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.061s	user 0.025s	sys 0.016s
W20260709 13:08:40.842211 23594 tablet.cc:2406] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:08:40.842697 23759 ts_tablet_manager.cc:1434] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.006s
I20260709 13:08:40.842330 23668 heartbeater.cc:499] Master 127.12.109.126:40143 was elected leader, sending a full tablet report...
I20260709 13:08:40.843878 23759 tablet_bootstrap.cc:492] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79: Bootstrap starting.
I20260709 13:08:40.844883 23757 tablet_bootstrap.cc:492] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae: No bootstrap required, opened a new log
I20260709 13:08:40.845501 23757 ts_tablet_manager.cc:1403] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae: Time spent bootstrapping tablet: real 0.166s	user 0.017s	sys 0.031s
I20260709 13:08:40.848979 23758 raft_consensus.cc:359] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.849326 23757 raft_consensus.cc:359] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.849882 23758 raft_consensus.cc:385] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.850275 23757 raft_consensus.cc:385] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.850342 23758 raft_consensus.cc:740] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:40.850711 23757 raft_consensus.cc:740] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Initialized, Role: FOLLOWER
I20260709 13:08:40.851223 23759 tablet_bootstrap.cc:654] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.851642 23758 consensus_queue.cc:260] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.851843 23757 consensus_queue.cc:260] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.854367 23758 ts_tablet_manager.cc:1434] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.001s
I20260709 13:08:40.855566 23758 tablet_bootstrap.cc:492] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:40.865829 23758 tablet_bootstrap.cc:654] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.867527 23757 ts_tablet_manager.cc:1434] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae: Time spent starting tablet: real 0.022s	user 0.017s	sys 0.003s
I20260709 13:08:40.867499 23742 heartbeater.cc:499] Master 127.12.109.126:40143 was elected leader, sending a full tablet report...
I20260709 13:08:40.869267 23757 tablet_bootstrap.cc:492] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae: Bootstrap starting.
I20260709 13:08:40.877504 23757 tablet_bootstrap.cc:654] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.878723 23758 tablet_bootstrap.cc:492] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8: No bootstrap required, opened a new log
I20260709 13:08:40.879482 23758 ts_tablet_manager.cc:1403] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.011s
W20260709 13:08:40.884256 23514 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:40.885555 23758 raft_consensus.cc:359] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.886539 23758 raft_consensus.cc:385] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.886929 23758 raft_consensus.cc:740] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:40.887820 23758 consensus_queue.cc:260] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.891664 23758 ts_tablet_manager.cc:1434] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:08:40.892930 23758 tablet_bootstrap.cc:492] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:40.891693 23759 tablet_bootstrap.cc:492] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79: No bootstrap required, opened a new log
I20260709 13:08:40.895135 23759 ts_tablet_manager.cc:1403] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.010s
I20260709 13:08:40.896989 23757 tablet_bootstrap.cc:492] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae: No bootstrap required, opened a new log
I20260709 13:08:40.897783 23757 ts_tablet_manager.cc:1403] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.008s
I20260709 13:08:40.898478 23759 raft_consensus.cc:359] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.899205 23759 raft_consensus.cc:385] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.899485 23759 raft_consensus.cc:740] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Initialized, Role: FOLLOWER
I20260709 13:08:40.900265 23759 consensus_queue.cc:260] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.900928 23758 tablet_bootstrap.cc:654] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.901451 23757 raft_consensus.cc:359] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.902400 23759 ts_tablet_manager.cc:1434] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:08:40.903641 23759 tablet_bootstrap.cc:492] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79: Bootstrap starting.
I20260709 13:08:40.902417 23757 raft_consensus.cc:385] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.906438 23757 raft_consensus.cc:740] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Initialized, Role: FOLLOWER
I20260709 13:08:40.907464 23757 consensus_queue.cc:260] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.910143 23757 ts_tablet_manager.cc:1434] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:08:40.911198 23757 tablet_bootstrap.cc:492] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae: Bootstrap starting.
I20260709 13:08:40.912659 23759 tablet_bootstrap.cc:654] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.916160 23758 tablet_bootstrap.cc:492] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8: No bootstrap required, opened a new log
I20260709 13:08:40.916915 23758 ts_tablet_manager.cc:1403] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.004s
I20260709 13:08:40.918141 23762 raft_consensus.cc:493] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.918641 23757 tablet_bootstrap.cc:654] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.921264 23758 raft_consensus.cc:359] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.922433 23758 raft_consensus.cc:385] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.922854 23758 raft_consensus.cc:740] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:40.924162 23758 consensus_queue.cc:260] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.927625 23758 ts_tablet_manager.cc:1434] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:08:40.929263 23758 tablet_bootstrap.cc:492] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:40.931520 23757 tablet_bootstrap.cc:492] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae: No bootstrap required, opened a new log
I20260709 13:08:40.932183 23757 ts_tablet_manager.cc:1403] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.011s
I20260709 13:08:40.935544 23757 raft_consensus.cc:359] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.936326 23757 raft_consensus.cc:385] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.918727 23762 raft_consensus.cc:515] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.936815 23757 raft_consensus.cc:740] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Initialized, Role: FOLLOWER
I20260709 13:08:40.938191 23757 consensus_queue.cc:260] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.939980 23758 tablet_bootstrap.cc:654] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.941433 23757 ts_tablet_manager.cc:1434] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:08:40.941936 23643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07a9927140e49fc8273c94c155ad1cb" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:40.942682 23757 tablet_bootstrap.cc:492] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae: Bootstrap starting.
W20260709 13:08:40.944267 23533 leader_election.cc:343] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:40.946645 23768 raft_consensus.cc:493] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.947204 23768 raft_consensus.cc:515] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:40.950004 23768 leader_election.cc:290] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983), 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637)
I20260709 13:08:40.954980 23757 tablet_bootstrap.cc:654] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.957669 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07a9927140e49fc8273c94c155ad1cb" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
I20260709 13:08:40.955046 23762 leader_election.cc:290] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
W20260709 13:08:40.960672 23532 leader_election.cc:343] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:40.961167 23532 leader_election.cc:304] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:40.963581 23762 raft_consensus.cc:2749] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:40.972543 23758 tablet_bootstrap.cc:492] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8: No bootstrap required, opened a new log
I20260709 13:08:40.973300 23758 ts_tablet_manager.cc:1403] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.015s
I20260709 13:08:40.974364 23768 raft_consensus.cc:493] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.974972 23768 raft_consensus.cc:515] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.976014 23759 tablet_bootstrap.cc:492] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79: No bootstrap required, opened a new log
I20260709 13:08:40.976696 23759 ts_tablet_manager.cc:1403] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79: Time spent bootstrapping tablet: real 0.073s	user 0.026s	sys 0.034s
I20260709 13:08:40.977437 23768 leader_election.cc:290] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:40.977041 23758 raft_consensus.cc:359] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.978286 23758 raft_consensus.cc:385] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.978628 23758 raft_consensus.cc:740] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:40.979524 23758 consensus_queue.cc:260] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.980202 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
I20260709 13:08:40.982424 23758 ts_tablet_manager.cc:1434] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:08:40.982551 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540c29b91f164ba881d3bd49a33653b6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
I20260709 13:08:40.983439 23717 raft_consensus.cc:2468] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 0.
I20260709 13:08:40.985474 23758 tablet_bootstrap.cc:492] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:40.985564 23759 raft_consensus.cc:359] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.987116 23759 raft_consensus.cc:385] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.987578 23759 raft_consensus.cc:740] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Initialized, Role: FOLLOWER
W20260709 13:08:40.989051 23607 leader_election.cc:343] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:40.989967 23607 leader_election.cc:304] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:40.990018 23759 consensus_queue.cc:260] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:40.993685 23758 tablet_bootstrap.cc:654] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.997769 23757 tablet_bootstrap.cc:492] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae: No bootstrap required, opened a new log
I20260709 13:08:40.998421 23757 ts_tablet_manager.cc:1403] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae: Time spent bootstrapping tablet: real 0.056s	user 0.020s	sys 0.008s
I20260709 13:08:40.999969 23759 ts_tablet_manager.cc:1434] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79: Time spent starting tablet: real 0.023s	user 0.014s	sys 0.001s
I20260709 13:08:41.001031 23568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" is_pre_election: true
I20260709 13:08:41.020709 23759 tablet_bootstrap.cc:492] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79: Bootstrap starting.
W20260709 13:08:41.021741 23607 leader_election.cc:343] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.022332 23757 raft_consensus.cc:359] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:41.022625 23607 leader_election.cc:304] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcb961e4f2944eb585f6bad2a758fc79; no voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8
I20260709 13:08:41.023267 23757 raft_consensus.cc:385] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.023662 23757 raft_consensus.cc:740] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.023968 23780 raft_consensus.cc:2749] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.024636 23757 consensus_queue.cc:260] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:40.998502 23768 raft_consensus.cc:2804] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.001801 23567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540c29b91f164ba881d3bd49a33653b6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" is_pre_election: true
I20260709 13:08:41.030750 23759 tablet_bootstrap.cc:654] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.030231 23757 ts_tablet_manager.cc:1434] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
W20260709 13:08:41.032516 23607 leader_election.cc:343] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.030675 23768 raft_consensus.cc:493] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.034346 23768 raft_consensus.cc:3060] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.034591 23757 tablet_bootstrap.cc:492] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae: Bootstrap starting.
I20260709 13:08:41.044646 23757 tablet_bootstrap.cc:654] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.047675 23768 raft_consensus.cc:515] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.056253 23568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540c29b91f164ba881d3bd49a33653b6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:41.056756 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540c29b91f164ba881d3bd49a33653b6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:41.057708 23717 raft_consensus.cc:3060] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:41.058239 23607 leader_election.cc:343] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.056529 23768 leader_election.cc:290] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:41.065281 23759 tablet_bootstrap.cc:492] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79: No bootstrap required, opened a new log
I20260709 13:08:41.065346 23757 tablet_bootstrap.cc:492] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae: No bootstrap required, opened a new log
I20260709 13:08:41.065598 23758 tablet_bootstrap.cc:492] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8: No bootstrap required, opened a new log
I20260709 13:08:41.066218 23759 ts_tablet_manager.cc:1403] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79: Time spent bootstrapping tablet: real 0.046s	user 0.018s	sys 0.009s
I20260709 13:08:41.066392 23757 ts_tablet_manager.cc:1403] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.007s
I20260709 13:08:41.066437 23758 ts_tablet_manager.cc:1403] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.081s	user 0.031s	sys 0.042s
I20260709 13:08:41.068346 23717 raft_consensus.cc:2468] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 1.
I20260709 13:08:41.069895 23607 leader_election.cc:304] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 9bc10389fc774e4eb3ef7127b3fe39e8
I20260709 13:08:41.069832 23759 raft_consensus.cc:359] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.070106 23757 raft_consensus.cc:359] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.070909 23759 raft_consensus.cc:385] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.070375 23758 raft_consensus.cc:359] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.071168 23757 raft_consensus.cc:385] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.071265 23759 raft_consensus.cc:740] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.071456 23758 raft_consensus.cc:385] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.071539 23768 raft_consensus.cc:2804] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.071631 23757 raft_consensus.cc:740] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.072263 23758 raft_consensus.cc:740] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.072546 23759 consensus_queue.cc:260] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.072896 23757 consensus_queue.cc:260] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.073357 23758 consensus_queue.cc:260] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.073427 23768 raft_consensus.cc:697] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 LEADER]: Becoming Leader. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:41.075416 23759 ts_tablet_manager.cc:1434] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:08:41.075255 23768 consensus_queue.cc:237] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.076601 23759 tablet_bootstrap.cc:492] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79: Bootstrap starting.
I20260709 13:08:41.078006 23757 ts_tablet_manager.cc:1434] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:08:41.078704 23758 ts_tablet_manager.cc:1434] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:08:41.079277 23757 tablet_bootstrap.cc:492] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae: Bootstrap starting.
I20260709 13:08:41.079984 23758 tablet_bootstrap.cc:492] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:41.087774 23759 tablet_bootstrap.cc:654] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.087777 23758 tablet_bootstrap.cc:654] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.087777 23757 tablet_bootstrap.cc:654] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.098474 23759 tablet_bootstrap.cc:492] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79: No bootstrap required, opened a new log
I20260709 13:08:41.099318 23759 ts_tablet_manager.cc:1403] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.000s
I20260709 13:08:41.103298 23759 raft_consensus.cc:359] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.104318 23759 raft_consensus.cc:385] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.104652 23759 raft_consensus.cc:740] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.104995 23758 tablet_bootstrap.cc:492] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: No bootstrap required, opened a new log
I20260709 13:08:41.105597 23758 ts_tablet_manager.cc:1403] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.004s
I20260709 13:08:41.105753 23759 consensus_queue.cc:260] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.107825 23757 tablet_bootstrap.cc:492] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae: No bootstrap required, opened a new log
I20260709 13:08:41.108426 23757 ts_tablet_manager.cc:1403] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.002s
I20260709 13:08:41.108690 23759 ts_tablet_manager.cc:1434] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:08:41.108160 23465 catalog_manager.cc:5697] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.108987 23758 raft_consensus.cc:359] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.109727 23758 raft_consensus.cc:385] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.109846 23759 tablet_bootstrap.cc:492] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79: Bootstrap starting.
I20260709 13:08:41.110072 23758 raft_consensus.cc:740] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.110859 23758 consensus_queue.cc:260] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.112288 23757 raft_consensus.cc:359] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.113320 23758 ts_tablet_manager.cc:1434] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:08:41.113441 23757 raft_consensus.cc:385] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.114293 23757 raft_consensus.cc:740] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.114537 23758 tablet_bootstrap.cc:492] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:41.115401 23757 consensus_queue.cc:260] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.117998 23757 ts_tablet_manager.cc:1434] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 13:08:41.118556 23762 raft_consensus.cc:493] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.119132 23757 tablet_bootstrap.cc:492] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae: Bootstrap starting.
I20260709 13:08:41.119189 23762 raft_consensus.cc:515] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.123198 23758 tablet_bootstrap.cc:654] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.124204 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
I20260709 13:08:41.125710 23643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:41.126945 23757 tablet_bootstrap.cc:654] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.127519 23769 raft_consensus.cc:493] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.128244 23769 raft_consensus.cc:515] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
W20260709 13:08:41.139065 23532 leader_election.cc:343] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:41.140363 23533 leader_election.cc:343] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.140933 23533 leader_election.cc:304] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:41.141942 23761 raft_consensus.cc:2749] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.143889 23762 leader_election.cc:290] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:41.144917 23769 leader_election.cc:290] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.156231 23757 tablet_bootstrap.cc:492] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae: No bootstrap required, opened a new log
I20260709 13:08:41.157003 23757 ts_tablet_manager.cc:1403] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.020s
I20260709 13:08:41.161062 23757 raft_consensus.cc:359] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.161903 23757 raft_consensus.cc:385] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.162274 23757 raft_consensus.cc:740] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.163908 23757 consensus_queue.cc:260] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.164953 23759 tablet_bootstrap.cc:654] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.165030 23643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:41.166870 23757 ts_tablet_manager.cc:1434] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
W20260709 13:08:41.167368 23682 leader_election.cc:343] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.168511 23757 tablet_bootstrap.cc:492] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae: Bootstrap starting.
I20260709 13:08:41.175730 23568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" is_pre_election: true
I20260709 13:08:41.176635 23568 raft_consensus.cc:2468] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 0.
I20260709 13:08:41.177999 23681 leader_election.cc:304] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:41.179418 23769 raft_consensus.cc:2804] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.179797 23769 raft_consensus.cc:493] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.180481 23769 raft_consensus.cc:3060] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.189232 23757 tablet_bootstrap.cc:654] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.195712 23758 tablet_bootstrap.cc:492] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: No bootstrap required, opened a new log
I20260709 13:08:41.196465 23758 ts_tablet_manager.cc:1403] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.082s	user 0.029s	sys 0.020s
I20260709 13:08:41.197439 23769 raft_consensus.cc:515] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.200816 23758 raft_consensus.cc:359] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.202006 23758 raft_consensus.cc:385] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.202419 23758 raft_consensus.cc:740] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.203599 23758 consensus_queue.cc:260] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.206086 23758 ts_tablet_manager.cc:1434] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:08:41.207361 23758 tablet_bootstrap.cc:492] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:41.208691 23796 raft_consensus.cc:493] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.209481 23568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:41.210279 23568 raft_consensus.cc:3060] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.208348 23643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
W20260709 13:08:41.219897 23682 leader_election.cc:343] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.207624 23769 leader_election.cc:290] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.215240 23758 tablet_bootstrap.cc:654] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.223596 23759 tablet_bootstrap.cc:492] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79: No bootstrap required, opened a new log
I20260709 13:08:41.223289 23568 raft_consensus.cc:2468] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 1.
I20260709 13:08:41.224414 23759 ts_tablet_manager.cc:1403] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79: Time spent bootstrapping tablet: real 0.115s	user 0.037s	sys 0.012s
I20260709 13:08:41.225469 23681 leader_election.cc:304] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:41.226857 23769 raft_consensus.cc:2804] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.209285 23796 raft_consensus.cc:515] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.230847 23759 raft_consensus.cc:359] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.232782 23759 raft_consensus.cc:385] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.232110 23796 leader_election.cc:290] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.233034 23568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" is_pre_election: true
I20260709 13:08:41.234045 23757 tablet_bootstrap.cc:492] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae: No bootstrap required, opened a new log
I20260709 13:08:41.234236 23643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:41.234355 23796 raft_consensus.cc:493] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.234803 23757 ts_tablet_manager.cc:1403] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae: Time spent bootstrapping tablet: real 0.067s	user 0.010s	sys 0.024s
I20260709 13:08:41.233847 23568 raft_consensus.cc:2468] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 0.
I20260709 13:08:41.235121 23796 raft_consensus.cc:515] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
W20260709 13:08:41.235889 23682 leader_election.cc:343] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.236657 23681 leader_election.cc:304] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:41.233114 23759 raft_consensus.cc:740] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.238612 23759 consensus_queue.cc:260] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.238930 23567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" is_pre_election: true
I20260709 13:08:41.238754 23757 raft_consensus.cc:359] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.239657 23757 raft_consensus.cc:385] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.239670 23567 raft_consensus.cc:2468] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 0.
I20260709 13:08:41.240099 23757 raft_consensus.cc:740] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.241475 23759 ts_tablet_manager.cc:1434] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:08:41.241293 23681 leader_election.cc:304] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 
I20260709 13:08:41.241117 23757 consensus_queue.cc:260] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.242878 23759 tablet_bootstrap.cc:492] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79: Bootstrap starting.
I20260709 13:08:41.241112 23796 leader_election.cc:290] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.245373 23800 raft_consensus.cc:2804] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.245251 23796 raft_consensus.cc:2804] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.246006 23796 raft_consensus.cc:493] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.246526 23796 raft_consensus.cc:3060] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.245899 23800 raft_consensus.cc:493] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.248072 23800 raft_consensus.cc:3060] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.250962 23643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:41.252058 23643 raft_consensus.cc:2468] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 0.
I20260709 13:08:41.261915 23801 raft_consensus.cc:493] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.239208 23758 tablet_bootstrap.cc:492] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: No bootstrap required, opened a new log
I20260709 13:08:41.264328 23758 ts_tablet_manager.cc:1403] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.010s
I20260709 13:08:41.262404 23757 ts_tablet_manager.cc:1434] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.016s
I20260709 13:08:41.265731 23757 tablet_bootstrap.cc:492] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae: Bootstrap starting.
I20260709 13:08:41.267484 23758 raft_consensus.cc:359] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.268216 23758 raft_consensus.cc:385] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.268421 23759 tablet_bootstrap.cc:654] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.268510 23758 raft_consensus.cc:740] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.270182 23769 raft_consensus.cc:697] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:41.270783 23757 tablet_bootstrap.cc:654] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.271275 23769 consensus_queue.cc:237] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.262444 23762 raft_consensus.cc:493] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.263337 23796 raft_consensus.cc:515] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.272797 23762 raft_consensus.cc:515] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.275431 23801 raft_consensus.cc:515] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.275668 23568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:41.276412 23568 raft_consensus.cc:3060] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.277945 23801 leader_election.cc:290] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.278193 23768 raft_consensus.cc:493] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.279012 23768 raft_consensus.cc:515] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.279846 23800 raft_consensus.cc:515] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.282104 23800 leader_election.cc:290] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.269529 23758 consensus_queue.cc:260] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.283196 23567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a0c51f78c384031b9ffa8d79085d7e4" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" is_pre_election: true
I20260709 13:08:41.283571 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a0c51f78c384031b9ffa8d79085d7e4" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
I20260709 13:08:41.283970 23567 raft_consensus.cc:2468] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 0.
I20260709 13:08:41.284278 23717 raft_consensus.cc:2468] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 0.
I20260709 13:08:41.284814 23758 ts_tablet_manager.cc:1434] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:08:41.285248 23568 raft_consensus.cc:2468] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 1.
I20260709 13:08:41.285979 23758 tablet_bootstrap.cc:492] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:41.286586 23796 leader_election.cc:290] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.287992 23641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:41.288607 23607 leader_election.cc:304] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:41.289625 23642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96b8ed446fa4b66930a590bda784ff1" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:41.290536 23642 raft_consensus.cc:2468] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 0.
I20260709 13:08:41.289626 23780 raft_consensus.cc:2804] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.292012 23780 raft_consensus.cc:493] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.292565 23780 raft_consensus.cc:3060] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.294633 23643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:41.295607 23643 raft_consensus.cc:2468] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in term 0.
I20260709 13:08:41.298398 23567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:41.299119 23567 raft_consensus.cc:3060] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.302815 23758 tablet_bootstrap.cc:654] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.304255 23533 leader_election.cc:304] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:41.305027 23681 leader_election.cc:304] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 
I20260709 13:08:41.305550 23761 raft_consensus.cc:2804] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.305923 23796 raft_consensus.cc:2804] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.306414 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
I20260709 13:08:41.306836 23796 raft_consensus.cc:697] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
W20260709 13:08:41.307861 23682 leader_election.cc:343] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.308092 23796 consensus_queue.cc:237] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.308754 23682 leader_election.cc:304] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:41.305991 23761 raft_consensus.cc:493] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.309653 23800 raft_consensus.cc:2804] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.310001 23761 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.310107 23800 raft_consensus.cc:493] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.310745 23800 raft_consensus.cc:3060] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:41.311198 23532 leader_election.cc:343] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.310089 23568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96b8ed446fa4b66930a590bda784ff1" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" is_pre_election: true
I20260709 13:08:41.312599 23568 raft_consensus.cc:2468] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 0.
I20260709 13:08:41.314299 23643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:41.314946 23643 raft_consensus.cc:3060] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.319667 23762 leader_election.cc:290] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:41.320766 23762 raft_consensus.cc:493] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.321275 23762 raft_consensus.cc:515] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.281898 23768 leader_election.cc:290] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:41.328791 23761 raft_consensus.cc:515] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.321975 23800 raft_consensus.cc:515] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.335471 23761 leader_election.cc:290] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 election: Requested vote from peers bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:41.325891 23567 raft_consensus.cc:2468] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 1.
I20260709 13:08:41.338709 23640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96b8ed446fa4b66930a590bda784ff1" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:41.339808 23640 raft_consensus.cc:3060] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.335958 23800 leader_election.cc:290] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.342861 23641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:41.343966 23641 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.330734 23758 tablet_bootstrap.cc:492] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8: No bootstrap required, opened a new log
I20260709 13:08:41.328779 23642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96b8ed446fa4b66930a590bda784ff1" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:41.347858 23758 ts_tablet_manager.cc:1403] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.062s	user 0.015s	sys 0.000s
I20260709 13:08:41.351284 23567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96b8ed446fa4b66930a590bda784ff1" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:41.352272 23567 raft_consensus.cc:3060] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.360942 23758 raft_consensus.cc:359] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:41.361896 23758 raft_consensus.cc:385] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.362223 23758 raft_consensus.cc:740] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.363210 23758 consensus_queue.cc:260] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:41.349690 23681 leader_election.cc:304] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:41.365777 23800 raft_consensus.cc:2804] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.366539 23800 raft_consensus.cc:697] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:41.365789 23758 ts_tablet_manager.cc:1434] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.003s
I20260709 13:08:41.367209 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:41.367707 23800 consensus_queue.cc:237] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
W20260709 13:08:41.369248 23532 leader_election.cc:343] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.378652 23762 leader_election.cc:290] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:41.381467 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96b8ed446fa4b66930a590bda784ff1" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
I20260709 13:08:41.383194 23717 raft_consensus.cc:2393] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in current term 1: Already voted for candidate 1f00e9da593145e68587c3779ae8acae in this term.
I20260709 13:08:41.388927 23532 leader_election.cc:304] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:41.402812 23641 raft_consensus.cc:2468] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in term 1.
I20260709 13:08:41.404645 23533 leader_election.cc:304] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 1f00e9da593145e68587c3779ae8acae
I20260709 13:08:41.406008 23761 raft_consensus.cc:2804] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.406270 23567 raft_consensus.cc:2468] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 1.
I20260709 13:08:41.406533 23761 raft_consensus.cc:697] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 LEADER]: Becoming Leader. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Running, Role: LEADER
I20260709 13:08:41.406920 23762 raft_consensus.cc:2749] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.407356 23643 raft_consensus.cc:2468] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 1.
I20260709 13:08:41.407670 23761 consensus_queue.cc:237] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.393901 23780 raft_consensus.cc:515] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.412040 23757 tablet_bootstrap.cc:492] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae: No bootstrap required, opened a new log
I20260709 13:08:41.411311 23464 catalog_manager.cc:5697] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 1 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:41.412459 23759 tablet_bootstrap.cc:492] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79: No bootstrap required, opened a new log
I20260709 13:08:41.412621 23757 ts_tablet_manager.cc:1403] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae: Time spent bootstrapping tablet: real 0.147s	user 0.029s	sys 0.043s
I20260709 13:08:41.413059 23759 ts_tablet_manager.cc:1403] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79: Time spent bootstrapping tablet: real 0.170s	user 0.021s	sys 0.065s
I20260709 13:08:41.415243 23681 leader_election.cc:304] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 
I20260709 13:08:41.419638 23800 raft_consensus.cc:2804] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.422900 23800 raft_consensus.cc:697] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:41.419732 23759 raft_consensus.cc:359] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.423985 23759 raft_consensus.cc:385] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.424031 23780 leader_election.cc:290] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:41.424357 23759 raft_consensus.cc:740] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.424726 23640 raft_consensus.cc:2468] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 1.
I20260709 13:08:41.425784 23759 consensus_queue.cc:260] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.426225 23757 raft_consensus.cc:359] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.424777 23800 consensus_queue.cc:237] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.427088 23757 raft_consensus.cc:385] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.427428 23757 raft_consensus.cc:740] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.428449 23759 ts_tablet_manager.cc:1434] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 13:08:41.428390 23757 consensus_queue.cc:260] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.429692 23759 tablet_bootstrap.cc:492] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79: Bootstrap starting.
I20260709 13:08:41.430533 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a0c51f78c384031b9ffa8d79085d7e4" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:41.431061 23757 ts_tablet_manager.cc:1434] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.003s
I20260709 13:08:41.431208 23717 raft_consensus.cc:3060] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.433662 23757 tablet_bootstrap.cc:492] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae: Bootstrap starting.
I20260709 13:08:41.437372 23759 tablet_bootstrap.cc:654] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.441200 23757 tablet_bootstrap.cc:654] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.461759 23759 tablet_bootstrap.cc:492] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79: No bootstrap required, opened a new log
I20260709 13:08:41.467007 23759 ts_tablet_manager.cc:1403] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.004s
I20260709 13:08:41.453876 23466 catalog_manager.cc:5697] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.470995 23567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a0c51f78c384031b9ffa8d79085d7e4" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:41.471701 23567 raft_consensus.cc:3060] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.464422 23757 tablet_bootstrap.cc:492] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae: No bootstrap required, opened a new log
I20260709 13:08:41.478790 23757 ts_tablet_manager.cc:1403] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae: Time spent bootstrapping tablet: real 0.045s	user 0.020s	sys 0.011s
I20260709 13:08:41.469903 23759 raft_consensus.cc:359] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.480019 23759 raft_consensus.cc:385] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.480399 23759 raft_consensus.cc:740] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.481237 23759 consensus_queue.cc:260] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.482537 23757 raft_consensus.cc:359] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.483402 23757 raft_consensus.cc:385] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.483680 23757 raft_consensus.cc:740] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.484603 23757 consensus_queue.cc:260] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.487133 23757 ts_tablet_manager.cc:1434] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:08:41.490434 23567 raft_consensus.cc:2468] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 1.
I20260709 13:08:41.491138 23717 raft_consensus.cc:2468] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 1.
I20260709 13:08:41.492074 23607 leader_election.cc:304] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:41.497028 23780 raft_consensus.cc:2804] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.492468 23759 ts_tablet_manager.cc:1434] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:08:41.497611 23780 raft_consensus.cc:697] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 LEADER]: Becoming Leader. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:41.498636 23759 tablet_bootstrap.cc:492] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79: Bootstrap starting.
I20260709 13:08:41.498589 23780 consensus_queue.cc:237] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.499426 23800 raft_consensus.cc:493] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.500218 23800 raft_consensus.cc:515] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:41.503290 23800 leader_election.cc:290] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483), 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637)
I20260709 13:08:41.503592 23640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:41.504405 23640 raft_consensus.cc:2468] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 0.
I20260709 13:08:41.505739 23682 leader_election.cc:304] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:41.506762 23800 raft_consensus.cc:2804] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.506711 23761 raft_consensus.cc:493] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.506986 23759 tablet_bootstrap.cc:654] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.507611 23800 raft_consensus.cc:493] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.507736 23761 raft_consensus.cc:515] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.508253 23800 raft_consensus.cc:3060] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.512482 23640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2797488d20e4732a4e65c083418840b" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:41.513286 23640 raft_consensus.cc:2468] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in term 0.
I20260709 13:08:41.514756 23533 leader_election.cc:304] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:41.515758 23805 raft_consensus.cc:2804] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.516150 23805 raft_consensus.cc:493] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.516512 23805 raft_consensus.cc:3060] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.520730 23568 raft_consensus.cc:3060] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.507236 23762 raft_consensus.cc:493] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.522547 23762 raft_consensus.cc:515] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:41.524932 23762 leader_election.cc:290] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.526966 23716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
I20260709 13:08:41.525490 23464 catalog_manager.cc:5697] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.527812 23641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:41.528616 23641 raft_consensus.cc:2468] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in term 0.
I20260709 13:08:41.530172 23533 leader_election.cc:304] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:41.531049 23761 leader_election.cc:290] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:41.531733 23761 raft_consensus.cc:2804] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.532222 23761 raft_consensus.cc:493] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.532366 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2797488d20e4732a4e65c083418840b" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
I20260709 13:08:41.532720 23761 raft_consensus.cc:3060] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.533105 23717 raft_consensus.cc:2468] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in term 0.
I20260709 13:08:41.537699 23759 tablet_bootstrap.cc:492] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79: No bootstrap required, opened a new log
I20260709 13:08:41.538338 23759 ts_tablet_manager.cc:1403] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79: Time spent bootstrapping tablet: real 0.040s	user 0.026s	sys 0.013s
I20260709 13:08:41.531694 23800 raft_consensus.cc:515] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:41.541146 23716 raft_consensus.cc:2393] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in current term 1: Already voted for candidate 1f00e9da593145e68587c3779ae8acae in this term.
I20260709 13:08:41.541545 23759 raft_consensus.cc:359] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.542187 23800 leader_election.cc:290] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Requested vote from peers bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483), 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637)
I20260709 13:08:41.542387 23759 raft_consensus.cc:385] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.542888 23759 raft_consensus.cc:740] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.543632 23759 consensus_queue.cc:260] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.544116 23780 consensus_queue.cc:1048] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:41.545603 23759 ts_tablet_manager.cc:1434] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:08:41.546733 23759 tablet_bootstrap.cc:492] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79: Bootstrap starting.
I20260709 13:08:41.548983 23761 raft_consensus.cc:515] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:41.549693 23640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:41.550386 23640 raft_consensus.cc:3060] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.552114 23716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:41.552977 23716 raft_consensus.cc:2393] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in current term 1: Already voted for candidate 1f00e9da593145e68587c3779ae8acae in this term.
I20260709 13:08:41.553303 23759 tablet_bootstrap.cc:654] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.555888 23761 leader_election.cc:290] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 election: Requested vote from peers 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.557986 23641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:41.559581 23533 leader_election.cc:304] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:41.560290 23761 raft_consensus.cc:2749] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.565387 23568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:41.566745 23568 raft_consensus.cc:2393] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f00e9da593145e68587c3779ae8acae in current term 1: Already voted for candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in this term.
I20260709 13:08:41.571105 23805 raft_consensus.cc:515] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.575037 23641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2797488d20e4732a4e65c083418840b" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:41.576071 23641 raft_consensus.cc:3060] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.578517 23567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" is_pre_election: true
I20260709 13:08:41.579635 23567 raft_consensus.cc:2393] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f00e9da593145e68587c3779ae8acae in current term 1: Already voted for candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in this term.
I20260709 13:08:41.578601 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2797488d20e4732a4e65c083418840b" candidate_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:41.580307 23717 raft_consensus.cc:3060] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.581705 23466 catalog_manager.cc:5697] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 1 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:41.584131 23466 catalog_manager.cc:5697] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 1 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:41.586575 23466 catalog_manager.cc:5697] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 1 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:41.592808 23640 raft_consensus.cc:2468] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 1.
I20260709 13:08:41.594391 23682 leader_election.cc:304] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 9bc10389fc774e4eb3ef7127b3fe39e8
I20260709 13:08:41.595417 23800 raft_consensus.cc:2804] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.596181 23800 raft_consensus.cc:697] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:41.597429 23800 consensus_queue.cc:237] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:41.596168 23805 leader_election.cc:290] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 election: Requested vote from peers bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:41.600819 23780 consensus_queue.cc:1048] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:41.608780 23759 tablet_bootstrap.cc:492] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79: No bootstrap required, opened a new log
I20260709 13:08:41.609428 23759 ts_tablet_manager.cc:1403] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79: Time spent bootstrapping tablet: real 0.063s	user 0.024s	sys 0.031s
I20260709 13:08:41.612416 23759 raft_consensus.cc:359] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.613121 23759 raft_consensus.cc:385] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.613422 23759 raft_consensus.cc:740] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.614449 23759 consensus_queue.cc:260] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.616750 23759 ts_tablet_manager.cc:1434] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.006s
I20260709 13:08:41.619336 23641 raft_consensus.cc:2468] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in term 1.
I20260709 13:08:41.622251 23533 leader_election.cc:304] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:41.623030 23805 raft_consensus.cc:2804] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.623500 23805 raft_consensus.cc:697] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 LEADER]: Becoming Leader. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Running, Role: LEADER
I20260709 13:08:41.624377 23805 consensus_queue.cc:237] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.635975 23465 catalog_manager.cc:5697] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.633813 23717 raft_consensus.cc:2468] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in term 1.
I20260709 13:08:41.645630 23466 catalog_manager.cc:5697] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 1 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.653908 23768 raft_consensus.cc:493] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.654619 23768 raft_consensus.cc:515] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.656646 23768 leader_election.cc:290] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:41.657447 23567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" is_pre_election: true
I20260709 13:08:41.657831 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
I20260709 13:08:41.658398 23567 raft_consensus.cc:2393] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bcb961e4f2944eb585f6bad2a758fc79 in current term 1: Already voted for candidate 1f00e9da593145e68587c3779ae8acae in this term.
I20260709 13:08:41.659783 23607 leader_election.cc:304] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcb961e4f2944eb585f6bad2a758fc79; no voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8
I20260709 13:08:41.660508 23768 raft_consensus.cc:3060] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.667045 23768 raft_consensus.cc:2749] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.677932 23800 consensus_queue.cc:1048] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:41.688501 23768 raft_consensus.cc:493] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.689044 23768 raft_consensus.cc:515] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.690490 23769 consensus_queue.cc:1048] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:41.693351 23567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" is_pre_election: true
I20260709 13:08:41.694290 23567 raft_consensus.cc:2393] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bcb961e4f2944eb585f6bad2a758fc79 in current term 1: Already voted for candidate 1f00e9da593145e68587c3779ae8acae in this term.
I20260709 13:08:41.694397 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
I20260709 13:08:41.693202 23768 leader_election.cc:290] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:41.696311 23607 leader_election.cc:304] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcb961e4f2944eb585f6bad2a758fc79; no voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8
I20260709 13:08:41.697855 23768 raft_consensus.cc:3060] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.705247 23768 raft_consensus.cc:2749] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.712355 23769 raft_consensus.cc:493] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.712937 23769 raft_consensus.cc:515] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.714944 23769 leader_election.cc:290] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.715656 23566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" is_pre_election: true
I20260709 13:08:41.715965 23641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:41.716807 23641 raft_consensus.cc:2393] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f00e9da593145e68587c3779ae8acae in current term 1: Already voted for candidate 9bc10389fc774e4eb3ef7127b3fe39e8 in this term.
I20260709 13:08:41.717707 23682 leader_election.cc:304] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae; no voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:41.718477 23769 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.724359 23769 raft_consensus.cc:2749] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.724737 23796 raft_consensus.cc:493] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.725309 23796 raft_consensus.cc:515] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.727303 23796 leader_election.cc:290] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.728212 23566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07a9927140e49fc8273c94c155ad1cb" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" is_pre_election: true
I20260709 13:08:41.728374 23641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07a9927140e49fc8273c94c155ad1cb" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:41.729226 23566 raft_consensus.cc:2468] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 0.
I20260709 13:08:41.729226 23641 raft_consensus.cc:2468] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 0.
I20260709 13:08:41.730618 23682 leader_election.cc:304] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:41.731393 23796 raft_consensus.cc:2804] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.731741 23796 raft_consensus.cc:493] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.732028 23796 raft_consensus.cc:3060] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.738358 23796 raft_consensus.cc:515] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.740151 23796 leader_election.cc:290] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:41.741199 23566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07a9927140e49fc8273c94c155ad1cb" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:41.741318 23641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07a9927140e49fc8273c94c155ad1cb" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:41.741966 23566 raft_consensus.cc:3060] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.742034 23641 raft_consensus.cc:3060] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.748165 23641 raft_consensus.cc:2468] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 1.
I20260709 13:08:41.748303 23566 raft_consensus.cc:2468] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 1.
I20260709 13:08:41.749513 23682 leader_election.cc:304] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:41.750401 23796 raft_consensus.cc:2804] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.750830 23796 raft_consensus.cc:697] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:41.751593 23796 consensus_queue.cc:237] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:41.760416 23466 catalog_manager.cc:5697] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 1 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:41.809303 23796 consensus_queue.cc:1048] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:41.810144 23769 consensus_queue.cc:1048] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:41.817617 23769 consensus_queue.cc:1048] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:41.835163 23805 consensus_queue.cc:1048] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:41.837642 23800 consensus_queue.cc:1048] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:41.850845 23801 consensus_queue.cc:1048] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:41.854737 23805 consensus_queue.cc:1048] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:41.870936 23801 consensus_queue.cc:1048] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:41.901880 23768 consensus_queue.cc:1048] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:41.914237 23768 consensus_queue.cc:1048] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:41.966955 12725 ts_manager.cc:295] Set tserver state for 9bc10389fc774e4eb3ef7127b3fe39e8 to MAINTENANCE_MODE
I20260709 13:08:41.967847 12725 tablet_server.cc:179] TabletServer@127.12.109.65:0 shutting down...
I20260709 13:08:41.994305 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:41.994979 12725 tablet_replica.cc:333] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: stopping tablet replica
I20260709 13:08:41.995821 12725 raft_consensus.cc:2243] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:41.996384 12725 raft_consensus.cc:2272] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:41.999900 12725 tablet_replica.cc:333] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: stopping tablet replica
W20260709 13:08:42.000411 23681 consensus_peers.cc:597] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:42.000530 12725 raft_consensus.cc:2243] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.001683 12725 raft_consensus.cc:2272] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.004053 12725 tablet_replica.cc:333] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: stopping tablet replica
I20260709 13:08:42.004670 12725 raft_consensus.cc:2243] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.005309 12725 pending_rounds.cc:70] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Trying to abort 1 pending ops.
I20260709 13:08:42.005517 12725 pending_rounds.cc:77] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635928582758400 op_type: NO_OP noop_request { }
I20260709 13:08:42.005851 12725 raft_consensus.cc:2889] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:42.006168 12725 raft_consensus.cc:2272] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.008392 12725 tablet_replica.cc:333] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8: stopping tablet replica
I20260709 13:08:42.009012 12725 raft_consensus.cc:2243] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.009444 12725 raft_consensus.cc:2272] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.011847 12725 tablet_replica.cc:333] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8: stopping tablet replica
I20260709 13:08:42.012494 12725 raft_consensus.cc:2243] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.012898 12725 raft_consensus.cc:2272] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.014866 12725 tablet_replica.cc:333] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8: stopping tablet replica
I20260709 13:08:42.015416 12725 raft_consensus.cc:2243] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.015936 12725 raft_consensus.cc:2272] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.018703 12725 tablet_replica.cc:333] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8: stopping tablet replica
I20260709 13:08:42.019289 12725 raft_consensus.cc:2243] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.019728 12725 raft_consensus.cc:2272] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.022128 12725 tablet_replica.cc:333] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8: stopping tablet replica
I20260709 13:08:42.022836 12725 raft_consensus.cc:2243] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.023386 12725 raft_consensus.cc:2272] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.025833 12725 tablet_replica.cc:333] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8: stopping tablet replica
I20260709 13:08:42.026407 12725 raft_consensus.cc:2243] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.026834 12725 raft_consensus.cc:2272] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.029135 12725 tablet_replica.cc:333] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8: stopping tablet replica
I20260709 13:08:42.029646 12725 raft_consensus.cc:2243] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.030125 12725 raft_consensus.cc:2272] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.061349 12725 tablet_server.cc:196] TabletServer@127.12.109.65:0 shutdown complete.
W20260709 13:08:42.075402 23681 consensus_peers.cc:597] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:42.145797 23607 consensus_peers.cc:597] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:42.146552 23801 consensus_queue.cc:1048] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:42.184736 23801 consensus_queue.cc:1048] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:08:42.201906 23681 consensus_peers.cc:597] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:42.283371 23681 consensus_peers.cc:597] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:42.342940 23681 consensus_peers.cc:597] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:42.400115 23607 consensus_peers.cc:597] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:42.411185 23681 consensus_peers.cc:597] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:42.900286 23514 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:08:43.027637 23681 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111) [suppressed 128 similar messages]
I20260709 13:08:43.297655 23769 raft_consensus.cc:493] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:43.298475 23769 raft_consensus.cc:515] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:43.300786 23769 leader_election.cc:290] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:43.302524 23640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2797488d20e4732a4e65c083418840b" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:43.304088 23640 raft_consensus.cc:2468] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 1.
I20260709 13:08:43.305862 23682 leader_election.cc:304] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
W20260709 13:08:43.306497 23681 leader_election.cc:336] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111)
I20260709 13:08:43.306850 23796 raft_consensus.cc:2804] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:43.307322 23796 raft_consensus.cc:493] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:43.307641 23796 raft_consensus.cc:3060] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:43.316931 23796 raft_consensus.cc:515] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:43.319096 23796 leader_election.cc:290] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:43.320317 23640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2797488d20e4732a4e65c083418840b" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:43.321031 23640 raft_consensus.cc:3060] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:43.323441 23681 leader_election.cc:336] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111)
I20260709 13:08:43.328694 23640 raft_consensus.cc:2468] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 2.
I20260709 13:08:43.329936 23682 leader_election.cc:304] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 9bc10389fc774e4eb3ef7127b3fe39e8
I20260709 13:08:43.330858 23796 raft_consensus.cc:2804] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:43.331316 23796 raft_consensus.cc:697] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 2 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:43.332437 23796 consensus_queue.cc:237] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:43.341995 23464 catalog_manager.cc:5697] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 1 to 2, leader changed from 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65) to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 2 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:43.349900 23768 raft_consensus.cc:493] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9bc10389fc774e4eb3ef7127b3fe39e8)
I20260709 13:08:43.351025 23768 raft_consensus.cc:515] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:43.353654 23768 leader_election.cc:290] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:43.356110 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f00e9da593145e68587c3779ae8acae" is_pre_election: true
W20260709 13:08:43.358393 23607 leader_election.cc:336] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111)
I20260709 13:08:43.359606 23607 leader_election.cc:304] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcb961e4f2944eb585f6bad2a758fc79; no voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8
I20260709 13:08:43.360466 23796 raft_consensus.cc:493] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9bc10389fc774e4eb3ef7127b3fe39e8)
I20260709 13:08:43.360607 23768 raft_consensus.cc:2749] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:43.361333 23796 raft_consensus.cc:515] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:43.363644 23796 leader_election.cc:290] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:43.364887 23640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79" is_pre_election: true
I20260709 13:08:43.365645 23640 raft_consensus.cc:2468] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 1.
I20260709 13:08:43.366823 23682 leader_election.cc:304] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
W20260709 13:08:43.367321 23681 leader_election.cc:336] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111)
I20260709 13:08:43.367514 23796 raft_consensus.cc:2804] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:43.367995 23796 raft_consensus.cc:493] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9bc10389fc774e4eb3ef7127b3fe39e8)
I20260709 13:08:43.368356 23796 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:43.375947 23796 raft_consensus.cc:515] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:43.377959 23796 leader_election.cc:290] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:43.379246 23640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:43.380017 23640 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:43.382441 23681 leader_election.cc:336] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111)
I20260709 13:08:43.386973 23640 raft_consensus.cc:2468] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 2.
I20260709 13:08:43.388167 23682 leader_election.cc:304] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 9bc10389fc774e4eb3ef7127b3fe39e8
I20260709 13:08:43.388901 23796 raft_consensus.cc:2804] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:43.389354 23796 raft_consensus.cc:697] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 2 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:43.390386 23796 consensus_queue.cc:237] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:43.398592 23464 catalog_manager.cc:5697] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 1 to 2, leader changed from 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65) to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 2 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:43.738037 23640 raft_consensus.cc:1275] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:43.740139 23769 consensus_queue.cc:1048] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:08:43.758558 23681 consensus_peers.cc:597] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:43.867086 23640 raft_consensus.cc:1275] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:43.868755 23854 consensus_queue.cc:1048] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:08:43.885010 23681 consensus_peers.cc:597] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:43.901618 23514 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:08:44.509473 23607 consensus_peers.cc:597] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.550686 23681 consensus_peers.cc:597] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.641626 23681 consensus_peers.cc:597] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.659783 23681 consensus_peers.cc:597] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.708115 23681 consensus_peers.cc:597] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.903004 23514 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:08:44.993803 23681 consensus_peers.cc:597] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:45.037353 23681 consensus_peers.cc:597] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:45.065544 23607 consensus_peers.cc:597] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:45.904050 23514 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:08:46.073936 23681 consensus_peers.cc:597] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:46.320765 23681 consensus_peers.cc:597] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:46.807284 23681 consensus_peers.cc:597] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:46.905093 23514 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:08:46.908474 23607 consensus_peers.cc:597] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:46.934320 23681 consensus_peers.cc:597] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:47.142488 23681 consensus_peers.cc:597] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:47.152758 23681 consensus_peers.cc:597] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:47.161015 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:47.169659 23862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:47.169691 23863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:47.171094 23865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:47.172892 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:47.175390 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:47.175657 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:47.175875 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602527175836 us; error 0 us; skew 500 ppm
I20260709 13:08:47.176496 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:47.180045 12725 webserver.cc:533] Webserver started at http://127.12.109.65:41111/ using document root <none> and password file <none>
I20260709 13:08:47.180667 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:47.180867 12725 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:08:47.185590 12725 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:08:47.190456 23870 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:47.191346 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:08:47.191830 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-0-root
uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
format_stamp: "Formatted at 2026-07-09 13:08:39 on dist-test-slave-zhdc"
I20260709 13:08:47.192291 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602188851028-12725-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:47.241696 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:47.243368 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:47.245286 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:47.249776 23877 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:08:47.312219 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:08:47.312546 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.064s	user 0.003s	sys 0.000s
I20260709 13:08:47.312834 12725 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:08:47.320036 23877 tablet_bootstrap.cc:492] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:47.339659 23877 tablet_bootstrap.cc:492] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:47.340843 23877 tablet_bootstrap.cc:492] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap complete.
I20260709 13:08:47.341585 23877 ts_tablet_manager.cc:1403] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 13:08:47.345010 23877 raft_consensus.cc:359] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.345818 23877 raft_consensus.cc:740] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.346627 23877 consensus_queue.cc:260] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.349870 23877 ts_tablet_manager.cc:1434] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 13:08:47.350824 23877 tablet_bootstrap.cc:492] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:47.369668 23877 tablet_bootstrap.cc:492] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:47.370882 23877 tablet_bootstrap.cc:492] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap complete.
I20260709 13:08:47.371656 23877 ts_tablet_manager.cc:1403] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.021s	user 0.020s	sys 0.000s
I20260709 13:08:47.374971 23877 raft_consensus.cc:359] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.375735 23877 raft_consensus.cc:740] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.376557 23877 consensus_queue.cc:260] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.378669 23877 ts_tablet_manager.cc:1434] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:08:47.379531 23877 tablet_bootstrap.cc:492] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:47.384981 12725 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:08:47.385458 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.073s	user 0.059s	sys 0.011s
I20260709 13:08:47.398093 23877 tablet_bootstrap.cc:492] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:47.399294 23877 tablet_bootstrap.cc:492] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap complete.
W20260709 13:08:47.399925 23681 consensus_peers.cc:597] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:47.400391 23877 ts_tablet_manager.cc:1403] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 13:08:47.403537 23877 raft_consensus.cc:359] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:47.404201 23877 raft_consensus.cc:740] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.405128 23877 consensus_queue.cc:260] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:47.407214 23877 ts_tablet_manager.cc:1434] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:08:47.408023 23877 tablet_bootstrap.cc:492] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:47.422546 23877 tablet_bootstrap.cc:492] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:47.423439 23877 tablet_bootstrap.cc:492] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap complete.
I20260709 13:08:47.424084 23877 ts_tablet_manager.cc:1403] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.007s
I20260709 13:08:47.426330 23877 raft_consensus.cc:359] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.426893 23877 raft_consensus.cc:740] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.427564 23877 consensus_queue.cc:260] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.429351 23877 ts_tablet_manager.cc:1434] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:08:47.430119 23877 tablet_bootstrap.cc:492] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:47.447459 23877 tablet_bootstrap.cc:492] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:47.448403 23877 tablet_bootstrap.cc:492] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap complete.
I20260709 13:08:47.449062 23877 ts_tablet_manager.cc:1403] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.000s
I20260709 13:08:47.451365 23877 raft_consensus.cc:359] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.451956 23877 raft_consensus.cc:740] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.452728 23877 consensus_queue.cc:260] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.454468 23877 ts_tablet_manager.cc:1434] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:08:47.455227 23877 tablet_bootstrap.cc:492] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:47.465077 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.65:45637
I20260709 13:08:47.465209 23940 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.65:45637 every 8 connection(s)
I20260709 13:08:47.471485 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:47.472299 23877 tablet_bootstrap.cc:492] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:47.473567 23877 tablet_bootstrap.cc:492] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap complete.
I20260709 13:08:47.474653 23877 ts_tablet_manager.cc:1403] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.003s
I20260709 13:08:47.478856 23877 raft_consensus.cc:359] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.483029 23877 raft_consensus.cc:740] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.485018 23877 consensus_queue.cc:260] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.487040 23877 ts_tablet_manager.cc:1434] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:08:47.488150 23877 tablet_bootstrap.cc:492] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:47.507558 23877 tablet_bootstrap.cc:492] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:47.508813 23877 tablet_bootstrap.cc:492] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap complete.
I20260709 13:08:47.509235 23717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
mode: GRACEFUL
new_leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:40434
I20260709 13:08:47.509654 23877 ts_tablet_manager.cc:1403] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.022s	user 0.020s	sys 0.000s
I20260709 13:08:47.509963 23942 heartbeater.cc:344] Connected to a master server at 127.12.109.126:40143
I20260709 13:08:47.510175 23717 raft_consensus.cc:606] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Received request to transfer leadership to TS bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:47.510540 23942 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:47.511672 23942 heartbeater.cc:507] Master 127.12.109.126:40143 requested a full tablet report, sending...
I20260709 13:08:47.512954 23877 raft_consensus.cc:359] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.513727 23877 raft_consensus.cc:740] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.514252 23717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
mode: GRACEFUL
new_leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:40434
I20260709 13:08:47.514963 23717 raft_consensus.cc:606] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 2 LEADER]: Received request to transfer leadership to TS bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:47.514649 23877 consensus_queue.cc:260] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.517385 23877 ts_tablet_manager.cc:1434] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:08:47.517808 23717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
mode: GRACEFUL
new_leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:40434
I20260709 13:08:47.518411 23877 tablet_bootstrap.cc:492] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:47.518424 23717 raft_consensus.cc:606] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Received request to transfer leadership to TS bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:47.521239 23717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d07a9927140e49fc8273c94c155ad1cb"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
mode: GRACEFUL
new_leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:40434
I20260709 13:08:47.521836 23717 raft_consensus.cc:606] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Received request to transfer leadership to TS bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:47.523339 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:08:47.523455 23466 ts_manager.cc:194] Re-registered known tserver with Master: 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637)
I20260709 13:08:47.524232 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:47.525102 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:47.530712 23860 raft_consensus.cc:993] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae: : Instructing follower bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:47.531490 23878 raft_consensus.cc:1081] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Signalling peer bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:47.533573 23640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:08:47.534351 23640 raft_consensus.cc:493] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:47.534736 23640 raft_consensus.cc:3060] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:47.535574 23466 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50206
I20260709 13:08:47.540520 23942 heartbeater.cc:499] Master 127.12.109.126:40143 was elected leader, sending a full tablet report...
I20260709 13:08:47.544092 23877 tablet_bootstrap.cc:492] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:47.545217 23877 tablet_bootstrap.cc:492] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap complete.
I20260709 13:08:47.545974 23877 ts_tablet_manager.cc:1403] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.028s	user 0.021s	sys 0.004s
I20260709 13:08:47.547262 23640 raft_consensus.cc:515] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.548172 23877 raft_consensus.cc:359] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.548884 23877 raft_consensus.cc:740] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.549732 23877 consensus_queue.cc:260] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.549810 23640 leader_election.cc:290] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:47.551429 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:47.552263 23717 raft_consensus.cc:3055] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:47.552677 23717 raft_consensus.cc:740] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:47.553879 23717 consensus_queue.cc:260] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.556057 23717 raft_consensus.cc:3060] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:47.558612 23877 ts_tablet_manager.cc:1434] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:08:47.559829 23877 tablet_bootstrap.cc:492] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:47.567471 23717 raft_consensus.cc:2468] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 2.
I20260709 13:08:47.570796 23607 leader_election.cc:304] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:47.571887 23856 raft_consensus.cc:2804] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:47.573655 23856 raft_consensus.cc:697] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Becoming Leader. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:47.575043 23856 consensus_queue.cc:237] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.580988 23877 tablet_bootstrap.cc:492] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:47.582332 23877 tablet_bootstrap.cc:492] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap complete.
I20260709 13:08:47.583173 23877 ts_tablet_manager.cc:1403] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.007s
I20260709 13:08:47.585265 23915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:47.586102 23915 raft_consensus.cc:3060] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:47.586436 23877 raft_consensus.cc:359] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.587244 23877 raft_consensus.cc:740] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.588228 23877 consensus_queue.cc:260] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.602662 23877 ts_tablet_manager.cc:1434] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.009s
I20260709 13:08:47.603796 23877 tablet_bootstrap.cc:492] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap starting.
I20260709 13:08:47.605613 23466 catalog_manager.cc:5697] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 reported cstate change: term changed from 1 to 2, leader changed from 1f00e9da593145e68587c3779ae8acae (127.12.109.67) to bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:47.608495 23915 raft_consensus.cc:2468] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 2.
I20260709 13:08:47.622502 23877 tablet_bootstrap.cc:492] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:08:47.623762 23877 tablet_bootstrap.cc:492] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Bootstrap complete.
I20260709 13:08:47.624625 23877 ts_tablet_manager.cc:1403] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 13:08:47.627715 23877 raft_consensus.cc:359] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.628875 23877 raft_consensus.cc:740] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc10389fc774e4eb3ef7127b3fe39e8, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.629631 23877 consensus_queue.cc:260] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.631691 23877 ts_tablet_manager.cc:1434] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:08:47.726385 23878 raft_consensus.cc:993] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae: : Instructing follower bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:47.726934 23860 raft_consensus.cc:1081] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Signalling peer bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:47.728645 23640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d07a9927140e49fc8273c94c155ad1cb"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:08:47.729147 23640 raft_consensus.cc:493] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:47.729447 23640 raft_consensus.cc:3060] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:47.735872 23640 raft_consensus.cc:515] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.738222 23640 leader_election.cc:290] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:47.739187 23915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07a9927140e49fc8273c94c155ad1cb" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:47.739910 23915 raft_consensus.cc:3060] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:47.739737 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07a9927140e49fc8273c94c155ad1cb" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:47.740572 23717 raft_consensus.cc:3055] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:47.740978 23717 raft_consensus.cc:740] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:47.742045 23717 consensus_queue.cc:260] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.743542 23717 raft_consensus.cc:3060] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:47.748601 23915 raft_consensus.cc:2468] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 2.
I20260709 13:08:47.750168 23607 leader_election.cc:304] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:47.750977 23856 raft_consensus.cc:2804] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:47.751490 23856 raft_consensus.cc:697] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Becoming Leader. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:47.752167 23717 raft_consensus.cc:2468] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 2.
I20260709 13:08:47.752517 23856 consensus_queue.cc:237] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.760808 23466 catalog_manager.cc:5697] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 reported cstate change: term changed from 1 to 2, leader changed from 1f00e9da593145e68587c3779ae8acae (127.12.109.67) to bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:47.828379 23878 raft_consensus.cc:993] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae: : Instructing follower bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:47.828945 23860 raft_consensus.cc:1081] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Signalling peer bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:47.830611 23640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:08:47.831158 23640 raft_consensus.cc:493] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:47.831498 23640 raft_consensus.cc:3060] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:47.837816 23640 raft_consensus.cc:515] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.839866 23640 leader_election.cc:290] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:47.841092 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:47.841710 23717 raft_consensus.cc:3055] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:47.840868 23915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:47.841794 23913 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:47.842078 23717 raft_consensus.cc:740] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:47.842324 23915 raft_consensus.cc:3060] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:47.843185 23717 consensus_queue.cc:260] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.844643 23717 raft_consensus.cc:3060] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:47.850950 23915 raft_consensus.cc:2468] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 2.
I20260709 13:08:47.852854 23717 raft_consensus.cc:2468] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 2.
I20260709 13:08:47.853574 23607 leader_election.cc:304] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:47.856052 23952 raft_consensus.cc:2804] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:47.858383 23952 raft_consensus.cc:697] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Becoming Leader. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:47.859534 23952 consensus_queue.cc:237] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.868964 23466 catalog_manager.cc:5697] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 reported cstate change: term changed from 1 to 2, leader changed from 1f00e9da593145e68587c3779ae8acae (127.12.109.67) to bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:47.906388 23514 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:47.945549 23913 raft_consensus.cc:3060] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:47.952195 23913 pending_rounds.cc:85] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Aborting all ops after (but not including) 0
I20260709 13:08:47.952466 23913 pending_rounds.cc:107] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:08:47.952782 23913 raft_consensus.cc:2889] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:47.986982 23860 raft_consensus.cc:993] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae: : Instructing follower bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:47.987571 23878 raft_consensus.cc:1081] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 2 LEADER]: Signalling peer bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:47.989322 23640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:08:47.989912 23640 raft_consensus.cc:493] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:47.990280 23640 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:47.996312 23640 raft_consensus.cc:515] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:47.998149 23640 leader_election.cc:290] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 3 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:47.999511 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:47.999531 23913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:48.000229 23717 raft_consensus.cc:3055] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:48.000355 23913 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:48.000530 23717 raft_consensus.cc:740] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:48.001371 23717 consensus_queue.cc:260] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:48.002535 23717 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:48.006976 23913 raft_consensus.cc:2468] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 3.
I20260709 13:08:48.008463 23607 leader_election.cc:304] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:48.010016 23952 raft_consensus.cc:2804] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:48.010862 23952 raft_consensus.cc:697] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 LEADER]: Becoming Leader. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:48.011677 23717 raft_consensus.cc:2468] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 3.
I20260709 13:08:48.012754 23952 consensus_queue.cc:237] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:48.021160 23466 catalog_manager.cc:5697] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 reported cstate change: term changed from 2 to 3, leader changed from 1f00e9da593145e68587c3779ae8acae (127.12.109.67) to bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66). New cstate: current_term: 3 leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:48.108079 23717 raft_consensus.cc:1275] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:48.109668 23952 consensus_queue.cc:1048] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:48.120784 23913 raft_consensus.cc:1275] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:48.122740 23967 consensus_queue.cc:1048] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:48.243460 23913 raft_consensus.cc:1275] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:48.244891 23967 consensus_queue.cc:1048] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:48.255832 23717 raft_consensus.cc:1275] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:48.257719 23967 consensus_queue.cc:1048] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:48.305876 23717 raft_consensus.cc:1275] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:48.307273 23967 consensus_queue.cc:1048] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:48.317569 23913 raft_consensus.cc:1275] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:48.320449 23967 consensus_queue.cc:1048] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:48.419143 23717 raft_consensus.cc:1275] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:48.420713 23967 consensus_queue.cc:1048] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:48.432493 23913 raft_consensus.cc:1275] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:48.434130 23952 consensus_queue.cc:1048] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:48.538249 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:48.557036 23641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
mode: GRACEFUL
new_leader_uuid: "1f00e9da593145e68587c3779ae8acae"
 from {username='slave'} at 127.0.0.1:52826
I20260709 13:08:48.557690 23641 raft_consensus.cc:606] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Received request to transfer leadership to TS 1f00e9da593145e68587c3779ae8acae
I20260709 13:08:48.560400 23641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
mode: GRACEFUL
new_leader_uuid: "1f00e9da593145e68587c3779ae8acae"
 from {username='slave'} at 127.0.0.1:52826
I20260709 13:08:48.560938 23641 raft_consensus.cc:606] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Received request to transfer leadership to TS 1f00e9da593145e68587c3779ae8acae
I20260709 13:08:48.561977 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:08:48.562636 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:48.563308 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:48.628633 23856 raft_consensus.cc:993] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79: : Instructing follower 1f00e9da593145e68587c3779ae8acae to start an election
I20260709 13:08:48.629312 23952 raft_consensus.cc:1081] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Signalling peer 1f00e9da593145e68587c3779ae8acae to start an election
I20260709 13:08:48.631067 23717 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
 from {username='slave'} at 127.0.0.1:40424
I20260709 13:08:48.631639 23717 raft_consensus.cc:493] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:48.631976 23717 raft_consensus.cc:3060] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:48.638309 23717 raft_consensus.cc:515] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:48.640057 23717 leader_election.cc:290] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 3 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:48.641183 23913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:48.641193 23641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a1a8c29abb4f4089bbd7881a879ad6" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:48.641987 23913 raft_consensus.cc:3060] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:48.642132 23641 raft_consensus.cc:3055] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:48.642556 23641 raft_consensus.cc:740] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:48.643277 23641 consensus_queue.cc:260] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:48.644356 23641 raft_consensus.cc:3060] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:48.648145 23913 raft_consensus.cc:2468] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 3.
I20260709 13:08:48.649371 23681 leader_election.cc:304] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 
I20260709 13:08:48.650144 23979 raft_consensus.cc:2804] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:48.650657 23979 raft_consensus.cc:697] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 3 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:48.651746 23641 raft_consensus.cc:2468] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 3.
I20260709 13:08:48.651571 23979 consensus_queue.cc:237] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:48.658444 23466 catalog_manager.cc:5697] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 2 to 3, leader changed from bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66) to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 3 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:48.840953 23952 raft_consensus.cc:993] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79: : Instructing follower 1f00e9da593145e68587c3779ae8acae to start an election
I20260709 13:08:48.841640 23856 raft_consensus.cc:1081] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Signalling peer 1f00e9da593145e68587c3779ae8acae to start an election
I20260709 13:08:48.843428 23717 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
 from {username='slave'} at 127.0.0.1:40424
I20260709 13:08:48.843969 23717 raft_consensus.cc:493] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:48.844285 23717 raft_consensus.cc:3060] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:48.850189 23717 raft_consensus.cc:515] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:48.851769 23717 leader_election.cc:290] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 3 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:48.852849 23641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:48.852881 23913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:48.853542 23641 raft_consensus.cc:3055] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:48.853708 23913 raft_consensus.cc:3060] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:48.853842 23641 raft_consensus.cc:740] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:48.854894 23641 consensus_queue.cc:260] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:48.856034 23641 raft_consensus.cc:3060] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:48.860450 23913 raft_consensus.cc:2468] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 3.
I20260709 13:08:48.861795 23681 leader_election.cc:304] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 
I20260709 13:08:48.862433 23641 raft_consensus.cc:2468] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 3.
I20260709 13:08:48.862762 23979 raft_consensus.cc:2804] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:48.863343 23979 raft_consensus.cc:697] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 3 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:48.864135 23979 consensus_queue.cc:237] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:48.870870 23466 catalog_manager.cc:5697] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 2 to 3, leader changed from bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66) to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 3 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:48.907423 23514 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:49.051888 23641 raft_consensus.cc:1275] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:49.053365 23878 consensus_queue.cc:1048] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:49.063522 23913 raft_consensus.cc:1275] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:49.065778 23979 consensus_queue.cc:1048] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:49.381999 23641 raft_consensus.cc:1275] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:49.383435 23878 consensus_queue.cc:1048] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:49.395594 23913 raft_consensus.cc:1275] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:49.396978 23979 consensus_queue.cc:1048] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:49.570590 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:49.580449 23641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
mode: GRACEFUL
new_leader_uuid: "1f00e9da593145e68587c3779ae8acae"
 from {username='slave'} at 127.0.0.1:52826
I20260709 13:08:49.581204 23641 raft_consensus.cc:606] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 LEADER]: Received request to transfer leadership to TS 1f00e9da593145e68587c3779ae8acae
I20260709 13:08:49.584157 23717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2797488d20e4732a4e65c083418840b"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
mode: GRACEFUL
new_leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:40434
I20260709 13:08:49.584700 23717 raft_consensus.cc:606] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 2 LEADER]: Received request to transfer leadership to TS bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:49.587066 23717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
mode: GRACEFUL
new_leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:40434
I20260709 13:08:49.587613 23717 raft_consensus.cc:606] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 3 LEADER]: Received request to transfer leadership to TS bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:49.588581 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:08:49.589089 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:49.589722 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:49.605440 23856 raft_consensus.cc:993] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79: : Instructing follower 1f00e9da593145e68587c3779ae8acae to start an election
I20260709 13:08:49.606001 23994 raft_consensus.cc:1081] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 LEADER]: Signalling peer 1f00e9da593145e68587c3779ae8acae to start an election
I20260709 13:08:49.607573 23717 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
 from {username='slave'} at 127.0.0.1:40424
I20260709 13:08:49.608101 23717 raft_consensus.cc:493] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:49.608429 23717 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.613909 23717 raft_consensus.cc:515] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:49.615669 23717 leader_election.cc:290] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 4 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:49.616611 23913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:49.617069 23641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:49.617291 23913 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.617714 23641 raft_consensus.cc:3055] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:49.618000 23641 raft_consensus.cc:740] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:49.618763 23641 consensus_queue.cc:260] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:49.619859 23641 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.623332 23913 raft_consensus.cc:2468] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 4.
I20260709 13:08:49.624670 23681 leader_election.cc:304] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 
I20260709 13:08:49.625491 23979 raft_consensus.cc:2804] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:49.625792 23641 raft_consensus.cc:2468] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 4.
I20260709 13:08:49.625937 23979 raft_consensus.cc:697] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 4 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:49.626989 23979 consensus_queue.cc:237] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:49.634792 23466 catalog_manager.cc:5697] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 3 to 4, leader changed from bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66) to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 4 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:49.908625 23514 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:50.009862 23878 raft_consensus.cc:993] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae: : Instructing follower bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:50.010492 23979 raft_consensus.cc:1081] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 2 LEADER]: Signalling peer bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:50.012022 23641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2797488d20e4732a4e65c083418840b"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:08:50.012542 23641 raft_consensus.cc:493] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:50.012828 23641 raft_consensus.cc:3060] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:50.016876 23878 raft_consensus.cc:993] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae: : Instructing follower bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:50.017506 23878 raft_consensus.cc:1081] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 3 LEADER]: Signalling peer bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:50.018931 23640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:08:50.019376 23640 raft_consensus.cc:493] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:50.019680 23640 raft_consensus.cc:3060] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:50.020779 23641 raft_consensus.cc:515] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:50.022774 23641 leader_election.cc:290] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 3 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:50.023910 23914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2797488d20e4732a4e65c083418840b" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:50.024269 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2797488d20e4732a4e65c083418840b" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:50.024627 23914 raft_consensus.cc:3060] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:50.024925 23717 raft_consensus.cc:3055] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:50.025308 23717 raft_consensus.cc:740] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:50.025993 23717 consensus_queue.cc:260] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:50.026978 23717 raft_consensus.cc:3060] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:50.027390 23640 raft_consensus.cc:515] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:50.029675 23640 leader_election.cc:290] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 4 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:50.030851 23913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:50.032379 23913 raft_consensus.cc:3060] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:50.033911 23717 raft_consensus.cc:2468] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 3.
I20260709 13:08:50.035794 23607 leader_election.cc:304] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:50.036305 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31d9cae460a432bb196bf822834bb01" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:50.037623 23994 raft_consensus.cc:2804] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:50.037703 23914 raft_consensus.cc:2468] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 3.
I20260709 13:08:50.037971 23717 raft_consensus.cc:3055] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:50.038755 23717 raft_consensus.cc:740] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:50.038259 23994 raft_consensus.cc:697] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 3 LEADER]: Becoming Leader. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:50.040139 23717 consensus_queue.cc:260] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:50.040314 23994 consensus_queue.cc:237] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:50.042177 23717 raft_consensus.cc:3060] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:50.044577 23913 raft_consensus.cc:2468] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 4.
I20260709 13:08:50.046437 23607 leader_election.cc:304] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:50.047819 23856 raft_consensus.cc:2804] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:50.048785 23640 raft_consensus.cc:1275] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 4 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:50.050557 23878 consensus_queue.cc:1048] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:50.050618 23856 raft_consensus.cc:697] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 4 LEADER]: Becoming Leader. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:50.051978 23856 consensus_queue.cc:237] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:50.055603 23717 raft_consensus.cc:2468] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 4.
I20260709 13:08:50.062978 23466 catalog_manager.cc:5697] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 reported cstate change: term changed from 2 to 3, leader changed from 1f00e9da593145e68587c3779ae8acae (127.12.109.67) to bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66). New cstate: current_term: 3 leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:50.070303 23913 raft_consensus.cc:1275] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 4 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:50.072558 23878 consensus_queue.cc:1048] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:50.093782 23464 catalog_manager.cc:5697] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 reported cstate change: term changed from 3 to 4, leader changed from 1f00e9da593145e68587c3779ae8acae (127.12.109.67) to bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66). New cstate: current_term: 4 leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:50.448930 23717 raft_consensus.cc:1275] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 4 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:50.450399 23856 consensus_queue.cc:1048] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:50.459493 23913 raft_consensus.cc:1275] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 4 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:50.461926 23994 consensus_queue.cc:1048] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:50.525832 23717 raft_consensus.cc:1275] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:50.527226 23994 consensus_queue.cc:1048] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:50.531611 23913 raft_consensus.cc:1275] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:50.534536 23994 consensus_queue.cc:1048] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:50.597805 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:50.606640 23640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a0c51f78c384031b9ffa8d79085d7e4"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
mode: GRACEFUL
new_leader_uuid: "1f00e9da593145e68587c3779ae8acae"
 from {username='slave'} at 127.0.0.1:52826
I20260709 13:08:50.607277 23640 raft_consensus.cc:606] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 LEADER]: Received request to transfer leadership to TS 1f00e9da593145e68587c3779ae8acae
I20260709 13:08:50.609887 23717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
mode: GRACEFUL
new_leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:40434
I20260709 13:08:50.610400 23717 raft_consensus.cc:606] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Received request to transfer leadership to TS bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:50.612937 23640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2797488d20e4732a4e65c083418840b"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
mode: GRACEFUL
new_leader_uuid: "1f00e9da593145e68587c3779ae8acae"
 from {username='slave'} at 127.0.0.1:52826
I20260709 13:08:50.613433 23640 raft_consensus.cc:606] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 3 LEADER]: Received request to transfer leadership to TS 1f00e9da593145e68587c3779ae8acae
I20260709 13:08:50.614807 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:08:50.615522 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:50.616357 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:50.762022 24011 raft_consensus.cc:993] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae: : Instructing follower bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:50.762815 23878 raft_consensus.cc:1081] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Signalling peer bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:50.764380 23641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:08:50.764901 23641 raft_consensus.cc:493] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:50.765216 23641 raft_consensus.cc:3060] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:50.770944 23641 raft_consensus.cc:515] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:50.772408 23641 leader_election.cc:290] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:50.773370 23913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:50.773597 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:50.774169 23913 raft_consensus.cc:3060] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:50.774232 23717 raft_consensus.cc:3055] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:50.774628 23717 raft_consensus.cc:740] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:50.775362 23717 consensus_queue.cc:260] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:50.776309 23717 raft_consensus.cc:3060] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:50.779781 23913 raft_consensus.cc:2468] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 2.
I20260709 13:08:50.780995 23607 leader_election.cc:304] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:50.781405 23717 raft_consensus.cc:2468] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 2.
I20260709 13:08:50.781697 24000 raft_consensus.cc:2804] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:50.782261 24000 raft_consensus.cc:697] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Becoming Leader. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:50.783253 24000 consensus_queue.cc:237] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:50.791222 23466 catalog_manager.cc:5697] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 reported cstate change: term changed from 1 to 2, leader changed from 1f00e9da593145e68587c3779ae8acae (127.12.109.67) to bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:50.827090 23856 raft_consensus.cc:993] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79: : Instructing follower 1f00e9da593145e68587c3779ae8acae to start an election
I20260709 13:08:50.827617 24000 raft_consensus.cc:1081] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 LEADER]: Signalling peer 1f00e9da593145e68587c3779ae8acae to start an election
I20260709 13:08:50.829180 23717 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a0c51f78c384031b9ffa8d79085d7e4"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
 from {username='slave'} at 127.0.0.1:40424
I20260709 13:08:50.829777 23717 raft_consensus.cc:493] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:50.830163 23717 raft_consensus.cc:3060] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:50.835502 23717 raft_consensus.cc:515] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:50.837260 23717 leader_election.cc:290] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:50.838344 23641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a0c51f78c384031b9ffa8d79085d7e4" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:50.838203 23913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a0c51f78c384031b9ffa8d79085d7e4" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:50.839104 23641 raft_consensus.cc:3055] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:50.839192 23913 raft_consensus.cc:3060] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:50.839421 23641 raft_consensus.cc:740] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:50.840292 23641 consensus_queue.cc:260] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:50.841336 23641 raft_consensus.cc:3060] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:50.845325 23913 raft_consensus.cc:2468] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 2.
I20260709 13:08:50.846489 23681 leader_election.cc:304] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 
I20260709 13:08:50.847311 23878 raft_consensus.cc:2804] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:50.847764 23878 raft_consensus.cc:697] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 2 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:50.847836 23641 raft_consensus.cc:2468] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 2.
I20260709 13:08:50.848753 23878 consensus_queue.cc:237] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:50.856920 23466 catalog_manager.cc:5697] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 1 to 2, leader changed from bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66) to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 2 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:50.909665 23514 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:51.060909 23856 raft_consensus.cc:993] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79: : Instructing follower 1f00e9da593145e68587c3779ae8acae to start an election
I20260709 13:08:51.061547 24000 raft_consensus.cc:1081] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 3 LEADER]: Signalling peer 1f00e9da593145e68587c3779ae8acae to start an election
I20260709 13:08:51.063082 23717 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2797488d20e4732a4e65c083418840b"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
 from {username='slave'} at 127.0.0.1:40424
I20260709 13:08:51.063580 23717 raft_consensus.cc:493] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:51.063832 23717 raft_consensus.cc:3060] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:51.070150 23717 raft_consensus.cc:515] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:51.071806 23717 leader_election.cc:290] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 4 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:51.072781 23913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2797488d20e4732a4e65c083418840b" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:51.072947 23640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2797488d20e4732a4e65c083418840b" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:51.073589 23913 raft_consensus.cc:3060] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:51.073614 23640 raft_consensus.cc:3055] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:51.074090 23640 raft_consensus.cc:740] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:51.074801 23640 consensus_queue.cc:260] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:51.075765 23640 raft_consensus.cc:3060] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:51.078987 23913 raft_consensus.cc:2468] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 4.
I20260709 13:08:51.080150 23681 leader_election.cc:304] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 
I20260709 13:08:51.080946 23878 raft_consensus.cc:2804] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:51.081030 23640 raft_consensus.cc:2468] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 4.
I20260709 13:08:51.081468 23878 raft_consensus.cc:697] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 4 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:51.082365 23878 consensus_queue.cc:237] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:51.091724 23466 catalog_manager.cc:5697] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 3 to 4, leader changed from bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66) to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 4 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:51.183161 23913 raft_consensus.cc:1275] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:51.184571 24000 consensus_queue.cc:1048] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:51.196461 23717 raft_consensus.cc:1275] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:51.198714 23856 consensus_queue.cc:1048] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:51.325917 23640 raft_consensus.cc:1275] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:51.327387 24011 consensus_queue.cc:1048] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:51.340189 23913 raft_consensus.cc:1275] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:51.342020 24024 consensus_queue.cc:1048] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:51.624708 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:51.632912 23717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
mode: GRACEFUL
new_leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:40434
I20260709 13:08:51.633502 23717 raft_consensus.cc:606] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Received request to transfer leadership to TS bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:51.636581 23640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
mode: GRACEFUL
new_leader_uuid: "1f00e9da593145e68587c3779ae8acae"
 from {username='slave'} at 127.0.0.1:52826
I20260709 13:08:51.637252 23640 raft_consensus.cc:606] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Received request to transfer leadership to TS 1f00e9da593145e68587c3779ae8acae
I20260709 13:08:51.639969 23717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
mode: GRACEFUL
new_leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:40434
I20260709 13:08:51.640504 23717 raft_consensus.cc:606] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 4 LEADER]: Received request to transfer leadership to TS bcb961e4f2944eb585f6bad2a758fc79
I20260709 13:08:51.641574 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:08:51.642163 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:51.642805 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:51.656366 23640 raft_consensus.cc:1275] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 4 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:51.657886 23878 consensus_queue.cc:1048] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:51.666905 23913 raft_consensus.cc:1275] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 4 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:51.668658 24027 consensus_queue.cc:1048] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:51.712548 24029 raft_consensus.cc:993] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79: : Instructing follower 1f00e9da593145e68587c3779ae8acae to start an election
I20260709 13:08:51.713208 24030 raft_consensus.cc:1081] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Signalling peer 1f00e9da593145e68587c3779ae8acae to start an election
I20260709 13:08:51.714854 23717 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8"
dest_uuid: "1f00e9da593145e68587c3779ae8acae"
 from {username='slave'} at 127.0.0.1:40424
I20260709 13:08:51.715354 23717 raft_consensus.cc:493] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:51.715649 23717 raft_consensus.cc:3060] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:51.721675 23717 raft_consensus.cc:515] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:51.723469 23717 leader_election.cc:290] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 3 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483)
I20260709 13:08:51.724617 23640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
I20260709 13:08:51.724495 23913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eb01d6595f3431faf29c07dcc7c17d8" candidate_uuid: "1f00e9da593145e68587c3779ae8acae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:51.725361 23640 raft_consensus.cc:3055] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:51.725463 23913 raft_consensus.cc:3060] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:51.725657 23640 raft_consensus.cc:740] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:51.726502 23640 consensus_queue.cc:260] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:51.727483 23640 raft_consensus.cc:3060] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:51.731160 23913 raft_consensus.cc:2468] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 3.
I20260709 13:08:51.732482 23681 leader_election.cc:304] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f00e9da593145e68587c3779ae8acae, 9bc10389fc774e4eb3ef7127b3fe39e8; no voters: 
I20260709 13:08:51.732647 23640 raft_consensus.cc:2468] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f00e9da593145e68587c3779ae8acae in term 3.
I20260709 13:08:51.733263 24027 raft_consensus.cc:2804] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:51.733837 24027 raft_consensus.cc:697] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 3 LEADER]: Becoming Leader. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:51.734624 24027 consensus_queue.cc:237] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:51.741384 23464 catalog_manager.cc:5697] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae reported cstate change: term changed from 2 to 3, leader changed from bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66) to 1f00e9da593145e68587c3779ae8acae (127.12.109.67). New cstate: current_term: 3 leader_uuid: "1f00e9da593145e68587c3779ae8acae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:51.910722 23514 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:52.044901 24024 raft_consensus.cc:993] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae: : Instructing follower bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:52.045532 24027 raft_consensus.cc:1081] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 4 LEADER]: Signalling peer bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:52.047215 23640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:08:52.047716 23640 raft_consensus.cc:493] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:52.048007 23640 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:52.053464 23640 raft_consensus.cc:515] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:52.055197 23640 leader_election.cc:290] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 5 election: Requested vote from peers 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637), 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983)
I20260709 13:08:52.056214 23913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:52.056360 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eda7f9aaa60469d9c3deed0977252b6" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:52.057087 23913 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:52.057189 23717 raft_consensus.cc:3055] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:52.057653 23717 raft_consensus.cc:740] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:52.058406 23717 consensus_queue.cc:260] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:52.059360 23717 raft_consensus.cc:3060] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:52.062798 23913 raft_consensus.cc:2468] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 5.
I20260709 13:08:52.064082 23607 leader_election.cc:304] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:52.064657 23717 raft_consensus.cc:2468] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 5.
I20260709 13:08:52.064996 24030 raft_consensus.cc:2804] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:52.065660 24030 raft_consensus.cc:697] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 5 LEADER]: Becoming Leader. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:52.066560 24030 consensus_queue.cc:237] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } }
I20260709 13:08:52.074012 23464 catalog_manager.cc:5697] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 reported cstate change: term changed from 4 to 5, leader changed from 1f00e9da593145e68587c3779ae8acae (127.12.109.67) to bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66). New cstate: current_term: 5 leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:52.144701 24024 raft_consensus.cc:993] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae: : Instructing follower bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:52.145305 24027 raft_consensus.cc:1081] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Signalling peer bcb961e4f2944eb585f6bad2a758fc79 to start an election
I20260709 13:08:52.179708 23640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce"
dest_uuid: "bcb961e4f2944eb585f6bad2a758fc79"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:08:52.181675 23640 raft_consensus.cc:493] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:52.182133 23914 raft_consensus.cc:1275] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:52.182574 23640 raft_consensus.cc:3060] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:52.184320 24024 consensus_queue.cc:1048] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:52.195261 23641 raft_consensus.cc:1275] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Refusing update from remote peer 1f00e9da593145e68587c3779ae8acae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:52.198683 24024 consensus_queue.cc:1048] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:52.201696 23640 raft_consensus.cc:515] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:52.204571 23640 leader_election.cc:290] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 election: Requested vote from peers 1f00e9da593145e68587c3779ae8acae (127.12.109.67:41983), 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637)
I20260709 13:08:52.228158 23913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8"
I20260709 13:08:52.229197 23913 raft_consensus.cc:3060] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:52.242731 23717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc469ae092d40f5aca5bef046fa8cce" candidate_uuid: "bcb961e4f2944eb585f6bad2a758fc79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f00e9da593145e68587c3779ae8acae"
I20260709 13:08:52.244876 23913 raft_consensus.cc:2468] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 2.
I20260709 13:08:52.246793 23717 raft_consensus.cc:3055] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:52.247265 23717 raft_consensus.cc:740] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f00e9da593145e68587c3779ae8acae, State: Running, Role: LEADER
I20260709 13:08:52.249186 23717 consensus_queue.cc:260] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:52.251986 23717 raft_consensus.cc:3060] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:52.253294 23607 leader_election.cc:304] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc10389fc774e4eb3ef7127b3fe39e8, bcb961e4f2944eb585f6bad2a758fc79; no voters: 
I20260709 13:08:52.254791 24030 raft_consensus.cc:2804] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:52.255379 24030 raft_consensus.cc:697] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Becoming Leader. State: Replica: bcb961e4f2944eb585f6bad2a758fc79, State: Running, Role: LEADER
I20260709 13:08:52.256784 24030 consensus_queue.cc:237] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } }
I20260709 13:08:52.267195 23717 raft_consensus.cc:2468] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb961e4f2944eb585f6bad2a758fc79 in term 2.
I20260709 13:08:52.276629 23466 catalog_manager.cc:5697] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 reported cstate change: term changed from 1 to 2, leader changed from 1f00e9da593145e68587c3779ae8acae (127.12.109.67) to bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "bcb961e4f2944eb585f6bad2a758fc79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb961e4f2944eb585f6bad2a758fc79" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:52.554325 23914 raft_consensus.cc:1275] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 5 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:08:52.555366 23717 raft_consensus.cc:1275] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 5 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:08:52.555871 24030 consensus_queue.cc:1048] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc10389fc774e4eb3ef7127b3fe39e8" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:52.557073 24029 consensus_queue.cc:1048] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:52.649910 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:52.682637 23717 raft_consensus.cc:1275] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Refusing update from remote peer bcb961e4f2944eb585f6bad2a758fc79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:52.684087 24030 consensus_queue.cc:1048] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f00e9da593145e68587c3779ae8acae" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:52.689970 12725 tablet_server.cc:179] TabletServer@127.12.109.65:45637 shutting down...
W20260709 13:08:52.696470 23607 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 93 similar messages]
W20260709 13:08:52.701424 23607 consensus_peers.cc:597] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:52.725065 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:52.726169 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:52.727671 12725 raft_consensus.cc:2243] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.728685 12725 raft_consensus.cc:2272] T c2797488d20e4732a4e65c083418840b P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.732256 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:52.733008 12725 raft_consensus.cc:2243] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.733528 12725 raft_consensus.cc:2272] T 7a0c51f78c384031b9ffa8d79085d7e4 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.736397 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:52.737139 12725 raft_consensus.cc:2243] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.737716 12725 raft_consensus.cc:2272] T d07a9927140e49fc8273c94c155ad1cb P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.740499 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:52.741142 12725 raft_consensus.cc:2243] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.741684 12725 raft_consensus.cc:2272] T 0fc469ae092d40f5aca5bef046fa8cce P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.744801 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:52.745425 12725 raft_consensus.cc:2243] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.745962 12725 raft_consensus.cc:2272] T c31d9cae460a432bb196bf822834bb01 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.748834 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:52.749516 12725 raft_consensus.cc:2243] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.749992 12725 raft_consensus.cc:2272] T 9eda7f9aaa60469d9c3deed0977252b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.753391 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:52.754033 12725 raft_consensus.cc:2243] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.754565 12725 raft_consensus.cc:2272] T 540c29b91f164ba881d3bd49a33653b6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.757078 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:52.757635 12725 raft_consensus.cc:2243] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.758147 12725 raft_consensus.cc:2272] T 9eb01d6595f3431faf29c07dcc7c17d8 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.761492 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:52.762246 12725 raft_consensus.cc:2243] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.763298 12725 raft_consensus.cc:2272] T 38a1a8c29abb4f4089bbd7881a879ad6 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.765920 12725 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:52.766624 12725 raft_consensus.cc:2243] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.767169 12725 raft_consensus.cc:2272] T b96b8ed446fa4b66930a590bda784ff1 P 9bc10389fc774e4eb3ef7127b3fe39e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:52.777294 23681 consensus_peers.cc:597] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:52.806691 12725 tablet_server.cc:196] TabletServer@127.12.109.65:45637 shutdown complete.
I20260709 13:08:52.840047 12725 tablet_server.cc:179] TabletServer@127.12.109.66:0 shutting down...
I20260709 13:08:52.875484 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:52.876423 12725 tablet_replica.cc:333] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79: stopping tablet replica
W20260709 13:08:52.876807 23681 consensus_peers.cc:597] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:52.877693 12725 raft_consensus.cc:2243] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.878412 12725 raft_consensus.cc:2272] T 9eb01d6595f3431faf29c07dcc7c17d8 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.881946 12725 tablet_replica.cc:333] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79: stopping tablet replica
I20260709 13:08:52.882894 12725 raft_consensus.cc:2243] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.883625 12725 raft_consensus.cc:2272] T 38a1a8c29abb4f4089bbd7881a879ad6 P bcb961e4f2944eb585f6bad2a758fc79 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.886482 12725 tablet_replica.cc:333] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79: stopping tablet replica
I20260709 13:08:52.887104 12725 raft_consensus.cc:2243] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.887704 12725 raft_consensus.cc:2272] T c2797488d20e4732a4e65c083418840b P bcb961e4f2944eb585f6bad2a758fc79 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.890687 12725 tablet_replica.cc:333] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79: stopping tablet replica
I20260709 13:08:52.891312 12725 raft_consensus.cc:2243] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:52.892441 12725 raft_consensus.cc:2272] T d07a9927140e49fc8273c94c155ad1cb P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.895336 12725 tablet_replica.cc:333] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79: stopping tablet replica
I20260709 13:08:52.896026 12725 raft_consensus.cc:2243] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.896613 12725 raft_consensus.cc:2272] T 7a0c51f78c384031b9ffa8d79085d7e4 P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.899374 12725 tablet_replica.cc:333] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79: stopping tablet replica
I20260709 13:08:52.900024 12725 raft_consensus.cc:2243] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:52.900532 12725 raft_consensus.cc:2272] T b96b8ed446fa4b66930a590bda784ff1 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.903338 12725 tablet_replica.cc:333] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79: stopping tablet replica
I20260709 13:08:52.903950 12725 raft_consensus.cc:2243] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:52.904850 12725 raft_consensus.cc:2272] T 0fc469ae092d40f5aca5bef046fa8cce P bcb961e4f2944eb585f6bad2a758fc79 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.907797 12725 tablet_replica.cc:333] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79: stopping tablet replica
I20260709 13:08:52.908504 12725 raft_consensus.cc:2243] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:52.909775 12725 raft_consensus.cc:2272] T c31d9cae460a432bb196bf822834bb01 P bcb961e4f2944eb585f6bad2a758fc79 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:52.912098 23514 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:52.912668 12725 tablet_replica.cc:333] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79: stopping tablet replica
I20260709 13:08:52.913491 12725 raft_consensus.cc:2243] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:08:52.914495 12725 raft_consensus.cc:2272] T 9eda7f9aaa60469d9c3deed0977252b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:52.918370 23682 consensus_peers.cc:597] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae -> Peer bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483): Couldn't send request to peer bcb961e4f2944eb585f6bad2a758fc79. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:52.918633 12725 tablet_replica.cc:333] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79: stopping tablet replica
I20260709 13:08:52.919312 12725 raft_consensus.cc:2243] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:52.920222 12725 raft_consensus.cc:2272] T 540c29b91f164ba881d3bd49a33653b6 P bcb961e4f2944eb585f6bad2a758fc79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:52.957867 12725 tablet_server.cc:196] TabletServer@127.12.109.66:0 shutdown complete.
W20260709 13:08:52.957834 23682 consensus_peers.cc:597] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae -> Peer bcb961e4f2944eb585f6bad2a758fc79 (127.12.109.66:41483): Couldn't send request to peer bcb961e4f2944eb585f6bad2a758fc79. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:52.990087 23681 consensus_peers.cc:597] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae -> Peer 9bc10389fc774e4eb3ef7127b3fe39e8 (127.12.109.65:45637): Couldn't send request to peer 9bc10389fc774e4eb3ef7127b3fe39e8. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:45637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:52.991241 12725 tablet_server.cc:179] TabletServer@127.12.109.67:0 shutting down...
I20260709 13:08:53.024989 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:53.025993 12725 tablet_replica.cc:333] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae: stopping tablet replica
I20260709 13:08:53.026919 12725 raft_consensus.cc:2243] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:53.027936 12725 raft_consensus.cc:2272] T c2797488d20e4732a4e65c083418840b P 1f00e9da593145e68587c3779ae8acae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:53.030612 12725 tablet_replica.cc:333] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae: stopping tablet replica
I20260709 13:08:53.031387 12725 raft_consensus.cc:2243] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:53.032192 12725 raft_consensus.cc:2272] T d07a9927140e49fc8273c94c155ad1cb P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:53.034878 12725 tablet_replica.cc:333] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae: stopping tablet replica
I20260709 13:08:53.035512 12725 raft_consensus.cc:2243] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:53.036477 12725 raft_consensus.cc:2272] T 7a0c51f78c384031b9ffa8d79085d7e4 P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:53.038949 12725 tablet_replica.cc:333] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae: stopping tablet replica
I20260709 13:08:53.039549 12725 raft_consensus.cc:2243] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:53.040083 12725 raft_consensus.cc:2272] T c31d9cae460a432bb196bf822834bb01 P 1f00e9da593145e68587c3779ae8acae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:53.042747 12725 tablet_replica.cc:333] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae: stopping tablet replica
I20260709 13:08:53.043345 12725 raft_consensus.cc:2243] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:53.043790 12725 raft_consensus.cc:2272] T 0fc469ae092d40f5aca5bef046fa8cce P 1f00e9da593145e68587c3779ae8acae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:53.047230 12725 tablet_replica.cc:333] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae: stopping tablet replica
I20260709 13:08:53.047865 12725 raft_consensus.cc:2243] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:53.048365 12725 raft_consensus.cc:2272] T 9eda7f9aaa60469d9c3deed0977252b6 P 1f00e9da593145e68587c3779ae8acae [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:53.052171 12725 tablet_replica.cc:333] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae: stopping tablet replica
I20260709 13:08:53.052732 12725 raft_consensus.cc:2243] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:53.053197 12725 raft_consensus.cc:2272] T 540c29b91f164ba881d3bd49a33653b6 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:53.055526 12725 tablet_replica.cc:333] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae: stopping tablet replica
I20260709 13:08:53.055991 12725 raft_consensus.cc:2243] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:53.057024 12725 raft_consensus.cc:2272] T 9eb01d6595f3431faf29c07dcc7c17d8 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:53.059973 12725 tablet_replica.cc:333] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae: stopping tablet replica
I20260709 13:08:53.060810 12725 raft_consensus.cc:2243] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:53.061811 12725 raft_consensus.cc:2272] T 38a1a8c29abb4f4089bbd7881a879ad6 P 1f00e9da593145e68587c3779ae8acae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:53.064121 12725 tablet_replica.cc:333] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae: stopping tablet replica
I20260709 13:08:53.064594 12725 raft_consensus.cc:2243] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:53.065446 12725 raft_consensus.cc:2272] T b96b8ed446fa4b66930a590bda784ff1 P 1f00e9da593145e68587c3779ae8acae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:53.102257 12725 tablet_server.cc:196] TabletServer@127.12.109.67:0 shutdown complete.
I20260709 13:08:53.135138 12725 master.cc:562] Master@127.12.109.126:40143 shutting down...
I20260709 13:08:53.162232 12725 raft_consensus.cc:2243] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:53.162894 12725 raft_consensus.cc:2272] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:53.163287 12725 tablet_replica.cc:333] T 00000000000000000000000000000000 P acbfdf33d795451fbe6e1122a6868f8d: stopping tablet replica
I20260709 13:08:53.186815 12725 master.cc:584] Master@127.12.109.126:40143 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13500 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:08:53.237181 12725 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.109.126:41471
I20260709 13:08:53.238596 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:53.245117 24052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:53.245204 24051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:53.246116 24054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:53.247648 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:53.248665 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:53.248891 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:53.249053 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602533249032 us; error 0 us; skew 500 ppm
I20260709 13:08:53.249631 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:53.252229 12725 webserver.cc:533] Webserver started at http://127.12.109.126:36967/ using document root <none> and password file <none>
I20260709 13:08:53.252791 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:53.252990 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:53.253276 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:53.254628 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/master-0-root/instance:
uuid: "a6694b64064941c0a7cf6b7b0126b003"
format_stamp: "Formatted at 2026-07-09 13:08:53 on dist-test-slave-zhdc"
I20260709 13:08:53.259325 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.005s
I20260709 13:08:53.262951 24059 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:53.263716 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:08:53.264036 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/master-0-root
uuid: "a6694b64064941c0a7cf6b7b0126b003"
format_stamp: "Formatted at 2026-07-09 13:08:53 on dist-test-slave-zhdc"
I20260709 13:08:53.264333 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:53.284413 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:53.285852 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:53.331784 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.126:41471
I20260709 13:08:53.331914 24110 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.126:41471 every 8 connection(s)
I20260709 13:08:53.336585 24111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.350888 24111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003: Bootstrap starting.
I20260709 13:08:53.356669 24111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.361793 24111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003: No bootstrap required, opened a new log
I20260709 13:08:53.364385 24111 raft_consensus.cc:359] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6694b64064941c0a7cf6b7b0126b003" member_type: VOTER }
I20260709 13:08:53.364821 24111 raft_consensus.cc:385] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.365012 24111 raft_consensus.cc:740] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6694b64064941c0a7cf6b7b0126b003, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.365598 24111 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6694b64064941c0a7cf6b7b0126b003" member_type: VOTER }
I20260709 13:08:53.366153 24111 raft_consensus.cc:399] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:08:53.366379 24111 raft_consensus.cc:493] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:08:53.366683 24111 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:53.372467 24111 raft_consensus.cc:515] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6694b64064941c0a7cf6b7b0126b003" member_type: VOTER }
I20260709 13:08:53.373039 24111 leader_election.cc:304] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6694b64064941c0a7cf6b7b0126b003; no voters: 
I20260709 13:08:53.374446 24111 leader_election.cc:290] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:08:53.374833 24114 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:53.376448 24114 raft_consensus.cc:697] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [term 1 LEADER]: Becoming Leader. State: Replica: a6694b64064941c0a7cf6b7b0126b003, State: Running, Role: LEADER
I20260709 13:08:53.377229 24114 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6694b64064941c0a7cf6b7b0126b003" member_type: VOTER }
I20260709 13:08:53.377997 24111 sys_catalog.cc:565] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:08:53.383093 24116 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6694b64064941c0a7cf6b7b0126b003. Latest consensus state: current_term: 1 leader_uuid: "a6694b64064941c0a7cf6b7b0126b003" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6694b64064941c0a7cf6b7b0126b003" member_type: VOTER } }
I20260709 13:08:53.383229 24115 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6694b64064941c0a7cf6b7b0126b003" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6694b64064941c0a7cf6b7b0126b003" member_type: VOTER } }
I20260709 13:08:53.383841 24116 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:53.385923 24115 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:53.390949 24121 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:08:53.399823 24121 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:08:53.400996 12725 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:08:53.410408 24121 catalog_manager.cc:1383] Generated new cluster ID: 253bb81c796f4330b93b28d1d4fbc822
I20260709 13:08:53.410710 24121 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:08:53.445065 24121 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:08:53.446682 24121 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:08:53.459998 24121 catalog_manager.cc:6070] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003: Generated new TSK 0
I20260709 13:08:53.460749 24121 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:08:53.468016 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:53.474706 24132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:53.476051 24133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:53.477691 24135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:53.478359 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:53.479550 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:53.479738 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:53.479960 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602533479942 us; error 0 us; skew 500 ppm
I20260709 13:08:53.480479 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:53.483057 12725 webserver.cc:533] Webserver started at http://127.12.109.65:33519/ using document root <none> and password file <none>
I20260709 13:08:53.483523 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:53.483681 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:53.483917 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:53.485222 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-0-root/instance:
uuid: "276e7a6d32de44f8bb951b3b723f7cfa"
format_stamp: "Formatted at 2026-07-09 13:08:53 on dist-test-slave-zhdc"
I20260709 13:08:53.490378 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:08:53.494237 24140 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:53.495122 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:08:53.495572 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-0-root
uuid: "276e7a6d32de44f8bb951b3b723f7cfa"
format_stamp: "Formatted at 2026-07-09 13:08:53 on dist-test-slave-zhdc"
I20260709 13:08:53.495890 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:53.526324 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:53.527834 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:53.529472 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:53.532018 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:53.532256 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:53.532495 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:53.532686 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:53.579782 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.65:44673
I20260709 13:08:53.579926 24202 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.65:44673 every 8 connection(s)
I20260709 13:08:53.585803 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:53.594766 24207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:53.596879 24208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:53.600247 24210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:53.602231 24203 heartbeater.cc:344] Connected to a master server at 127.12.109.126:41471
I20260709 13:08:53.602771 24203 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:53.602855 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:53.603876 24203 heartbeater.cc:507] Master 127.12.109.126:41471 requested a full tablet report, sending...
I20260709 13:08:53.604216 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:53.604501 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:53.604679 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602533604660 us; error 0 us; skew 500 ppm
I20260709 13:08:53.605623 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:53.606837 24076 ts_manager.cc:194] Registered new tserver with Master: 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65:44673)
I20260709 13:08:53.609427 12725 webserver.cc:533] Webserver started at http://127.12.109.66:37339/ using document root <none> and password file <none>
I20260709 13:08:53.610029 24076 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35404
I20260709 13:08:53.610307 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:53.610692 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:53.611085 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:53.613075 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-1-root/instance:
uuid: "8a4876fe3cda499ea7a3e94ffb07764f"
format_stamp: "Formatted at 2026-07-09 13:08:53 on dist-test-slave-zhdc"
I20260709 13:08:53.619366 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:08:53.623127 24215 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:53.623976 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:08:53.624255 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-1-root
uuid: "8a4876fe3cda499ea7a3e94ffb07764f"
format_stamp: "Formatted at 2026-07-09 13:08:53 on dist-test-slave-zhdc"
I20260709 13:08:53.624514 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:53.640864 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:53.642326 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:53.643977 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:53.646577 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:53.646795 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:53.647043 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:53.647212 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:53.695391 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.66:37213
I20260709 13:08:53.695513 24277 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.66:37213 every 8 connection(s)
I20260709 13:08:53.700932 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:53.709898 24281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:53.711457 24282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:53.714824 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:53.715440 24278 heartbeater.cc:344] Connected to a master server at 127.12.109.126:41471
W20260709 13:08:53.715565 24284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:53.716297 24278 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:53.717137 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:53.717499 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:53.717414 24278 heartbeater.cc:507] Master 127.12.109.126:41471 requested a full tablet report, sending...
I20260709 13:08:53.717864 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602533717844 us; error 0 us; skew 500 ppm
I20260709 13:08:53.718767 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:53.720361 24076 ts_manager.cc:194] Registered new tserver with Master: 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66:37213)
I20260709 13:08:53.721601 12725 webserver.cc:533] Webserver started at http://127.12.109.67:32811/ using document root <none> and password file <none>
I20260709 13:08:53.722255 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:53.722460 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:53.722414 24076 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35412
I20260709 13:08:53.722977 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:53.724483 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-2-root/instance:
uuid: "31a85d150bca40e59ef64ff9f8972a1a"
format_stamp: "Formatted at 2026-07-09 13:08:53 on dist-test-slave-zhdc"
I20260709 13:08:53.729296 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:08:53.733021 24289 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:53.734025 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:08:53.734391 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-2-root
uuid: "31a85d150bca40e59ef64ff9f8972a1a"
format_stamp: "Formatted at 2026-07-09 13:08:53 on dist-test-slave-zhdc"
I20260709 13:08:53.734683 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602188851028-12725-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:53.752002 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:53.753404 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:53.755052 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:53.757556 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:53.757752 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:53.758045 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:53.758275 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.001s
I20260709 13:08:53.806310 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.67:43857
I20260709 13:08:53.806504 24351 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.67:43857 every 8 connection(s)
I20260709 13:08:53.821574 24352 heartbeater.cc:344] Connected to a master server at 127.12.109.126:41471
I20260709 13:08:53.822144 24352 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:53.822983 24352 heartbeater.cc:507] Master 127.12.109.126:41471 requested a full tablet report, sending...
I20260709 13:08:53.825160 24076 ts_manager.cc:194] Registered new tserver with Master: 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:53.825759 12725 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015883861s
I20260709 13:08:53.827071 24076 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35422
I20260709 13:08:53.855510 24076 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35430:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:08:53.857925 24076 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:08:53.913081 24243 tablet_service.cc:1511] Processing CreateTablet for tablet f8e4b5290bc944f6bff92a9413f185d6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=81afa170397f44b684cb940564d5d168]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:08:53.914975 24243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8e4b5290bc944f6bff92a9413f185d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.914718 24242 tablet_service.cc:1511] Processing CreateTablet for tablet fbb52c3e2309408ab7e0b36e793379b4 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=81afa170397f44b684cb940564d5d168]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:08:53.915457 24168 tablet_service.cc:1511] Processing CreateTablet for tablet f8e4b5290bc944f6bff92a9413f185d6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=81afa170397f44b684cb940564d5d168]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:08:53.916388 24242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb52c3e2309408ab7e0b36e793379b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.916903 24168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8e4b5290bc944f6bff92a9413f185d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.917512 24317 tablet_service.cc:1511] Processing CreateTablet for tablet f8e4b5290bc944f6bff92a9413f185d6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=81afa170397f44b684cb940564d5d168]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:08:53.918133 24167 tablet_service.cc:1511] Processing CreateTablet for tablet fbb52c3e2309408ab7e0b36e793379b4 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=81afa170397f44b684cb940564d5d168]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:08:53.919009 24317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8e4b5290bc944f6bff92a9413f185d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.919505 24167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb52c3e2309408ab7e0b36e793379b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.924886 24316 tablet_service.cc:1511] Processing CreateTablet for tablet fbb52c3e2309408ab7e0b36e793379b4 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=81afa170397f44b684cb940564d5d168]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:08:53.926472 24316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb52c3e2309408ab7e0b36e793379b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.954262 24367 tablet_bootstrap.cc:492] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f: Bootstrap starting.
I20260709 13:08:53.959757 24368 tablet_bootstrap.cc:492] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa: Bootstrap starting.
I20260709 13:08:53.965409 24369 tablet_bootstrap.cc:492] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a: Bootstrap starting.
I20260709 13:08:53.966346 24367 tablet_bootstrap.cc:654] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.969012 24368 tablet_bootstrap.cc:654] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.974105 24367 tablet_bootstrap.cc:492] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f: No bootstrap required, opened a new log
I20260709 13:08:53.974344 24369 tablet_bootstrap.cc:654] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.974749 24367 ts_tablet_manager.cc:1403] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.011s
I20260709 13:08:53.975181 24368 tablet_bootstrap.cc:492] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa: No bootstrap required, opened a new log
I20260709 13:08:53.975663 24368 ts_tablet_manager.cc:1403] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.005s
I20260709 13:08:53.978716 24367 raft_consensus.cc:359] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:53.978796 24368 raft_consensus.cc:359] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:53.979489 24367 raft_consensus.cc:385] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.979701 24368 raft_consensus.cc:385] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.979938 24367 raft_consensus.cc:740] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a4876fe3cda499ea7a3e94ffb07764f, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.980082 24368 raft_consensus.cc:740] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 276e7a6d32de44f8bb951b3b723f7cfa, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.980926 24368 consensus_queue.cc:260] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:53.981170 24369 tablet_bootstrap.cc:492] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a: No bootstrap required, opened a new log
I20260709 13:08:53.981055 24367 consensus_queue.cc:260] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:53.981909 24369 ts_tablet_manager.cc:1403] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a: Time spent bootstrapping tablet: real 0.017s	user 0.004s	sys 0.011s
I20260709 13:08:53.986606 24369 raft_consensus.cc:359] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:53.987628 24369 raft_consensus.cc:385] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.987952 24369 raft_consensus.cc:740] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a85d150bca40e59ef64ff9f8972a1a, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.988866 24203 heartbeater.cc:499] Master 127.12.109.126:41471 was elected leader, sending a full tablet report...
I20260709 13:08:53.989094 24368 ts_tablet_manager.cc:1434] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.010s
I20260709 13:08:53.988963 24369 consensus_queue.cc:260] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:53.990483 24368 tablet_bootstrap.cc:492] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa: Bootstrap starting.
I20260709 13:08:53.990851 24367 ts_tablet_manager.cc:1434] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.005s
I20260709 13:08:53.992293 24278 heartbeater.cc:499] Master 127.12.109.126:41471 was elected leader, sending a full tablet report...
I20260709 13:08:53.999547 24369 ts_tablet_manager.cc:1434] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.006s
I20260709 13:08:54.000173 24368 tablet_bootstrap.cc:654] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.000506 24352 heartbeater.cc:499] Master 127.12.109.126:41471 was elected leader, sending a full tablet report...
I20260709 13:08:54.006381 24367 tablet_bootstrap.cc:492] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f: Bootstrap starting.
I20260709 13:08:54.010603 24369 tablet_bootstrap.cc:492] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a: Bootstrap starting.
I20260709 13:08:54.015081 24367 tablet_bootstrap.cc:654] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.020514 24369 tablet_bootstrap.cc:654] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.022761 24368 tablet_bootstrap.cc:492] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa: No bootstrap required, opened a new log
I20260709 13:08:54.023459 24368 ts_tablet_manager.cc:1403] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.016s
I20260709 13:08:54.025079 24367 tablet_bootstrap.cc:492] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f: No bootstrap required, opened a new log
I20260709 13:08:54.025748 24367 ts_tablet_manager.cc:1403] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.003s
I20260709 13:08:54.026945 24369 tablet_bootstrap.cc:492] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a: No bootstrap required, opened a new log
I20260709 13:08:54.026978 24368 raft_consensus.cc:359] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.027683 24369 ts_tablet_manager.cc:1403] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:08:54.028018 24368 raft_consensus.cc:385] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.028388 24368 raft_consensus.cc:740] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 276e7a6d32de44f8bb951b3b723f7cfa, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.028970 24367 raft_consensus.cc:359] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.029628 24367 raft_consensus.cc:385] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.029336 24368 consensus_queue.cc:260] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.029912 24367 raft_consensus.cc:740] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a4876fe3cda499ea7a3e94ffb07764f, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.030858 24367 consensus_queue.cc:260] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.031044 24369 raft_consensus.cc:359] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.031884 24369 raft_consensus.cc:385] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.032223 24369 raft_consensus.cc:740] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a85d150bca40e59ef64ff9f8972a1a, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.032042 24368 ts_tablet_manager.cc:1434] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.003s
I20260709 13:08:54.033211 24369 consensus_queue.cc:260] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.034977 24367 ts_tablet_manager.cc:1434] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:08:54.037146 24369 ts_tablet_manager.cc:1434] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:08:54.038146 24374 raft_consensus.cc:493] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.038717 24374 raft_consensus.cc:515] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.041703 24373 raft_consensus.cc:493] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.041738 24374 leader_election.cc:290] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65:44673), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:54.042387 24373 raft_consensus.cc:515] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.050293 24373 leader_election.cc:290] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66:37213), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:54.073390 24178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e4b5290bc944f6bff92a9413f185d6" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" is_pre_election: true
I20260709 13:08:54.074430 24178 raft_consensus.cc:2468] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 0.
I20260709 13:08:54.077080 24219 leader_election.cc:304] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 276e7a6d32de44f8bb951b3b723f7cfa, 8a4876fe3cda499ea7a3e94ffb07764f; no voters: 
I20260709 13:08:54.078495 24374 raft_consensus.cc:2804] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:54.078902 24374 raft_consensus.cc:493] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.079236 24374 raft_consensus.cc:3060] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.079319 24327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e4b5290bc944f6bff92a9413f185d6" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a" is_pre_election: true
I20260709 13:08:54.080217 24327 raft_consensus.cc:2468] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 0.
I20260709 13:08:54.081568 24253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e4b5290bc944f6bff92a9413f185d6" candidate_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" is_pre_election: true
I20260709 13:08:54.081985 24327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e4b5290bc944f6bff92a9413f185d6" candidate_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a" is_pre_election: true
I20260709 13:08:54.082732 24327 raft_consensus.cc:2468] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 276e7a6d32de44f8bb951b3b723f7cfa in term 0.
I20260709 13:08:54.083963 24144 leader_election.cc:304] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 276e7a6d32de44f8bb951b3b723f7cfa, 31a85d150bca40e59ef64ff9f8972a1a; no voters: 
I20260709 13:08:54.084997 24373 raft_consensus.cc:2804] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:54.085378 24373 raft_consensus.cc:493] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.085712 24373 raft_consensus.cc:3060] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.088984 24382 raft_consensus.cc:493] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.088717 24374 raft_consensus.cc:515] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.089509 24382 raft_consensus.cc:515] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.090224 24253 raft_consensus.cc:2393] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 276e7a6d32de44f8bb951b3b723f7cfa in current term 1: Already voted for candidate 8a4876fe3cda499ea7a3e94ffb07764f in this term.
I20260709 13:08:54.092041 24382 leader_election.cc:290] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65:44673), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:54.093020 24178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e4b5290bc944f6bff92a9413f185d6" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "276e7a6d32de44f8bb951b3b723f7cfa"
I20260709 13:08:54.093456 24177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb52c3e2309408ab7e0b36e793379b4" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" is_pre_election: true
I20260709 13:08:54.094273 24177 raft_consensus.cc:2468] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 0.
I20260709 13:08:54.094835 24327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb52c3e2309408ab7e0b36e793379b4" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a" is_pre_election: true
I20260709 13:08:54.095083 24374 leader_election.cc:290] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 1 election: Requested vote from peers 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65:44673), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:54.095541 24327 raft_consensus.cc:2468] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 0.
I20260709 13:08:54.095683 24326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e4b5290bc944f6bff92a9413f185d6" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a"
I20260709 13:08:54.096342 24326 raft_consensus.cc:3060] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.097177 24373 raft_consensus.cc:515] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.098627 24178 raft_consensus.cc:2393] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a4876fe3cda499ea7a3e94ffb07764f in current term 1: Already voted for candidate 276e7a6d32de44f8bb951b3b723f7cfa in this term.
I20260709 13:08:54.098417 24219 leader_election.cc:304] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a85d150bca40e59ef64ff9f8972a1a, 8a4876fe3cda499ea7a3e94ffb07764f; no voters: 
I20260709 13:08:54.099575 24373 leader_election.cc:290] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [CANDIDATE]: Term 1 election: Requested vote from peers 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66:37213), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:54.100543 24327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e4b5290bc944f6bff92a9413f185d6" candidate_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a"
I20260709 13:08:54.101639 24374 raft_consensus.cc:2804] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:54.102015 24374 raft_consensus.cc:493] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.102396 24374 raft_consensus.cc:3060] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.102690 24253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e4b5290bc944f6bff92a9413f185d6" candidate_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a4876fe3cda499ea7a3e94ffb07764f"
I20260709 13:08:54.103513 24253 raft_consensus.cc:2393] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 276e7a6d32de44f8bb951b3b723f7cfa in current term 1: Already voted for candidate 8a4876fe3cda499ea7a3e94ffb07764f in this term.
I20260709 13:08:54.104696 24142 leader_election.cc:304] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 276e7a6d32de44f8bb951b3b723f7cfa; no voters: 31a85d150bca40e59ef64ff9f8972a1a, 8a4876fe3cda499ea7a3e94ffb07764f
I20260709 13:08:54.105413 24326 raft_consensus.cc:2468] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 1.
I20260709 13:08:54.106029 24373 raft_consensus.cc:2749] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:54.106600 24219 leader_election.cc:304] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31a85d150bca40e59ef64ff9f8972a1a, 8a4876fe3cda499ea7a3e94ffb07764f; no voters: 276e7a6d32de44f8bb951b3b723f7cfa
I20260709 13:08:54.107399 24382 raft_consensus.cc:2804] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:54.108784 24382 raft_consensus.cc:697] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Becoming Leader. State: Replica: 8a4876fe3cda499ea7a3e94ffb07764f, State: Running, Role: LEADER
I20260709 13:08:54.109745 24382 consensus_queue.cc:237] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.109898 24374 raft_consensus.cc:515] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.112814 24178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb52c3e2309408ab7e0b36e793379b4" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "276e7a6d32de44f8bb951b3b723f7cfa"
I20260709 13:08:54.113134 24326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb52c3e2309408ab7e0b36e793379b4" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a"
I20260709 13:08:54.113612 24178 raft_consensus.cc:3060] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.112128 24374 leader_election.cc:290] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 1 election: Requested vote from peers 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65:44673), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:54.113744 24326 raft_consensus.cc:3060] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.122401 24178 raft_consensus.cc:2468] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 1.
I20260709 13:08:54.123843 24326 raft_consensus.cc:2468] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 1.
I20260709 13:08:54.124001 24219 leader_election.cc:304] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 276e7a6d32de44f8bb951b3b723f7cfa, 8a4876fe3cda499ea7a3e94ffb07764f; no voters: 
I20260709 13:08:54.125613 24382 raft_consensus.cc:2804] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:54.125245 24075 catalog_manager.cc:5697] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66). New cstate: current_term: 1 leader_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.126252 24382 raft_consensus.cc:697] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Becoming Leader. State: Replica: 8a4876fe3cda499ea7a3e94ffb07764f, State: Running, Role: LEADER
I20260709 13:08:54.127231 24382 consensus_queue.cc:237] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.139448 24075 catalog_manager.cc:5697] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66). New cstate: current_term: 1 leader_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.201947 24075 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35436:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:08:54.204309 24075 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:08:54.230412 24168 tablet_service.cc:1511] Processing CreateTablet for tablet 000afa448059464b95bce1ced63f9ccd (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5d9c850793cb4badaa8a8853610ee19b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:08:54.231773 24243 tablet_service.cc:1511] Processing CreateTablet for tablet 000afa448059464b95bce1ced63f9ccd (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5d9c850793cb4badaa8a8853610ee19b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:08:54.231886 24168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 000afa448059464b95bce1ced63f9ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.233337 24243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 000afa448059464b95bce1ced63f9ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.233210 24316 tablet_service.cc:1511] Processing CreateTablet for tablet 000afa448059464b95bce1ced63f9ccd (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5d9c850793cb4badaa8a8853610ee19b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:08:54.234647 24316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 000afa448059464b95bce1ced63f9ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.241529 24167 tablet_service.cc:1511] Processing CreateTablet for tablet 35968142104d4d3fa16bc4c030ee2b31 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5d9c850793cb4badaa8a8853610ee19b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:08:54.243063 24167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35968142104d4d3fa16bc4c030ee2b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.246183 24242 tablet_service.cc:1511] Processing CreateTablet for tablet 35968142104d4d3fa16bc4c030ee2b31 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5d9c850793cb4badaa8a8853610ee19b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:08:54.247641 24242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35968142104d4d3fa16bc4c030ee2b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.249001 24317 tablet_service.cc:1511] Processing CreateTablet for tablet 35968142104d4d3fa16bc4c030ee2b31 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5d9c850793cb4badaa8a8853610ee19b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:08:54.250396 24317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35968142104d4d3fa16bc4c030ee2b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.251892 24368 tablet_bootstrap.cc:492] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa: Bootstrap starting.
I20260709 13:08:54.259629 24368 tablet_bootstrap.cc:654] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.261976 24367 tablet_bootstrap.cc:492] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f: Bootstrap starting.
I20260709 13:08:54.268923 24369 tablet_bootstrap.cc:492] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a: Bootstrap starting.
I20260709 13:08:54.271728 24368 tablet_bootstrap.cc:492] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa: No bootstrap required, opened a new log
I20260709 13:08:54.272315 24368 ts_tablet_manager.cc:1403] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.004s
I20260709 13:08:54.275839 24368 raft_consensus.cc:359] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.278595 24368 raft_consensus.cc:385] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.278910 24368 raft_consensus.cc:740] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 276e7a6d32de44f8bb951b3b723f7cfa, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.279713 24368 consensus_queue.cc:260] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.280375 24367 tablet_bootstrap.cc:654] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.280972 24369 tablet_bootstrap.cc:654] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.282570 24368 ts_tablet_manager.cc:1434] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:08:54.283797 24368 tablet_bootstrap.cc:492] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa: Bootstrap starting.
I20260709 13:08:54.293470 24368 tablet_bootstrap.cc:654] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.307533 24368 tablet_bootstrap.cc:492] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa: No bootstrap required, opened a new log
I20260709 13:08:54.308147 24368 ts_tablet_manager.cc:1403] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa: Time spent bootstrapping tablet: real 0.025s	user 0.022s	sys 0.000s
I20260709 13:08:54.308583 24369 tablet_bootstrap.cc:492] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a: No bootstrap required, opened a new log
I20260709 13:08:54.309170 24367 tablet_bootstrap.cc:492] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f: No bootstrap required, opened a new log
I20260709 13:08:54.309242 24369 ts_tablet_manager.cc:1403] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.017s
I20260709 13:08:54.309825 24367 ts_tablet_manager.cc:1403] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f: Time spent bootstrapping tablet: real 0.048s	user 0.018s	sys 0.013s
I20260709 13:08:54.312367 24367 raft_consensus.cc:359] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.311650 24368 raft_consensus.cc:359] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.312927 24369 raft_consensus.cc:359] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.313400 24367 raft_consensus.cc:385] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.313491 24368 raft_consensus.cc:385] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.313657 24369 raft_consensus.cc:385] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.313807 24367 raft_consensus.cc:740] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a4876fe3cda499ea7a3e94ffb07764f, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.313985 24368 raft_consensus.cc:740] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 276e7a6d32de44f8bb951b3b723f7cfa, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.314181 24369 raft_consensus.cc:740] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a85d150bca40e59ef64ff9f8972a1a, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.315088 24369 consensus_queue.cc:260] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.314970 24367 consensus_queue.cc:260] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.314937 24368 consensus_queue.cc:260] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.320169 24368 ts_tablet_manager.cc:1434] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:08:54.320309 24369 ts_tablet_manager.cc:1434] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:08:54.321141 24367 ts_tablet_manager.cc:1434] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:08:54.321563 24369 tablet_bootstrap.cc:492] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a: Bootstrap starting.
I20260709 13:08:54.322013 24367 tablet_bootstrap.cc:492] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f: Bootstrap starting.
I20260709 13:08:54.330021 24369 tablet_bootstrap.cc:654] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.331506 24367 tablet_bootstrap.cc:654] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.343921 24369 tablet_bootstrap.cc:492] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a: No bootstrap required, opened a new log
I20260709 13:08:54.344540 24369 ts_tablet_manager.cc:1403] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.000s
I20260709 13:08:54.345834 24367 tablet_bootstrap.cc:492] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f: No bootstrap required, opened a new log
I20260709 13:08:54.346426 24367 ts_tablet_manager.cc:1403] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.007s
I20260709 13:08:54.347611 24369 raft_consensus.cc:359] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.348281 24369 raft_consensus.cc:385] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.348512 24369 raft_consensus.cc:740] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a85d150bca40e59ef64ff9f8972a1a, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.348960 24367 raft_consensus.cc:359] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.349206 24369 consensus_queue.cc:260] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.349648 24367 raft_consensus.cc:385] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.349967 24367 raft_consensus.cc:740] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a4876fe3cda499ea7a3e94ffb07764f, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.350762 24367 consensus_queue.cc:260] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.351593 24369 ts_tablet_manager.cc:1434] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:08:54.353009 24367 ts_tablet_manager.cc:1434] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:08:54.363066 24382 raft_consensus.cc:493] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.363600 24382 raft_consensus.cc:515] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.365590 24382 leader_election.cc:290] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65:44673), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:54.366329 24178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35968142104d4d3fa16bc4c030ee2b31" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" is_pre_election: true
I20260709 13:08:54.366791 24326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35968142104d4d3fa16bc4c030ee2b31" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a" is_pre_election: true
I20260709 13:08:54.367120 24178 raft_consensus.cc:2468] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 0.
I20260709 13:08:54.367547 24326 raft_consensus.cc:2468] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 0.
I20260709 13:08:54.368265 24219 leader_election.cc:304] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 276e7a6d32de44f8bb951b3b723f7cfa, 8a4876fe3cda499ea7a3e94ffb07764f; no voters: 
I20260709 13:08:54.369071 24382 raft_consensus.cc:2804] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:54.369385 24382 raft_consensus.cc:493] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.369675 24382 raft_consensus.cc:3060] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.371105 24373 raft_consensus.cc:493] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.371665 24373 raft_consensus.cc:515] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.373996 24373 leader_election.cc:290] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66:37213), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:54.374670 24253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000afa448059464b95bce1ced63f9ccd" candidate_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" is_pre_election: true
I20260709 13:08:54.375104 24326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000afa448059464b95bce1ced63f9ccd" candidate_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a" is_pre_election: true
I20260709 13:08:54.375487 24253 raft_consensus.cc:2468] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 276e7a6d32de44f8bb951b3b723f7cfa in term 0.
I20260709 13:08:54.375828 24326 raft_consensus.cc:2468] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 276e7a6d32de44f8bb951b3b723f7cfa in term 0.
I20260709 13:08:54.376725 24142 leader_election.cc:304] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 276e7a6d32de44f8bb951b3b723f7cfa, 8a4876fe3cda499ea7a3e94ffb07764f; no voters: 
I20260709 13:08:54.377822 24373 raft_consensus.cc:2804] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:54.378239 24373 raft_consensus.cc:493] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.378518 24373 raft_consensus.cc:3060] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.379279 24382 raft_consensus.cc:515] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.381065 24382 leader_election.cc:290] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 1 election: Requested vote from peers 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65:44673), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:54.381965 24178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35968142104d4d3fa16bc4c030ee2b31" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "276e7a6d32de44f8bb951b3b723f7cfa"
I20260709 13:08:54.382278 24326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35968142104d4d3fa16bc4c030ee2b31" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a"
I20260709 13:08:54.382707 24178 raft_consensus.cc:3060] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.382864 24326 raft_consensus.cc:3060] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.386194 24373 raft_consensus.cc:515] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.388564 24373 leader_election.cc:290] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [CANDIDATE]: Term 1 election: Requested vote from peers 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66:37213), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:54.389276 24326 raft_consensus.cc:2468] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 1.
I20260709 13:08:54.389437 24253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000afa448059464b95bce1ced63f9ccd" candidate_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a4876fe3cda499ea7a3e94ffb07764f"
I20260709 13:08:54.389801 24327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000afa448059464b95bce1ced63f9ccd" candidate_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a"
I20260709 13:08:54.390172 24253 raft_consensus.cc:3060] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.390502 24178 raft_consensus.cc:2468] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 1.
W20260709 13:08:54.392289 24124 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:54.392354 24219 leader_election.cc:304] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a85d150bca40e59ef64ff9f8972a1a, 8a4876fe3cda499ea7a3e94ffb07764f; no voters: 
I20260709 13:08:54.393818 24382 raft_consensus.cc:2804] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:54.390532 24327 raft_consensus.cc:3060] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.394337 24382 raft_consensus.cc:697] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Becoming Leader. State: Replica: 8a4876fe3cda499ea7a3e94ffb07764f, State: Running, Role: LEADER
I20260709 13:08:54.395417 24382 consensus_queue.cc:237] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.399045 24253 raft_consensus.cc:2468] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 276e7a6d32de44f8bb951b3b723f7cfa in term 1.
I20260709 13:08:54.402068 24327 raft_consensus.cc:2468] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 276e7a6d32de44f8bb951b3b723f7cfa in term 1.
I20260709 13:08:54.402791 24142 leader_election.cc:304] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 276e7a6d32de44f8bb951b3b723f7cfa, 8a4876fe3cda499ea7a3e94ffb07764f; no voters: 
I20260709 13:08:54.403960 24373 raft_consensus.cc:2804] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:54.408390 24074 catalog_manager.cc:5697] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66). New cstate: current_term: 1 leader_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.409559 24373 raft_consensus.cc:697] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 LEADER]: Becoming Leader. State: Replica: 276e7a6d32de44f8bb951b3b723f7cfa, State: Running, Role: LEADER
I20260709 13:08:54.414196 24373 consensus_queue.cc:237] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.423629 24074 catalog_manager.cc:5697] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa reported cstate change: term changed from 0 to 1, leader changed from <none> to 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65). New cstate: current_term: 1 leader_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.479867 24253 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f8e4b5290bc944f6bff92a9413f185d6"
dest_uuid: "8a4876fe3cda499ea7a3e94ffb07764f"
mode: GRACEFUL
new_leader_uuid: "276e7a6d32de44f8bb951b3b723f7cfa"
 from {username='slave'} at 127.0.0.1:58990
I20260709 13:08:54.480561 24253 raft_consensus.cc:606] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Received request to transfer leadership to TS 276e7a6d32de44f8bb951b3b723f7cfa
I20260709 13:08:54.486215 24253 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbb52c3e2309408ab7e0b36e793379b4"
dest_uuid: "8a4876fe3cda499ea7a3e94ffb07764f"
mode: GRACEFUL
new_leader_uuid: "31a85d150bca40e59ef64ff9f8972a1a"
 from {username='slave'} at 127.0.0.1:58990
I20260709 13:08:54.486740 24253 raft_consensus.cc:606] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Received request to transfer leadership to TS 31a85d150bca40e59ef64ff9f8972a1a
I20260709 13:08:54.496085 24253 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35968142104d4d3fa16bc4c030ee2b31"
dest_uuid: "8a4876fe3cda499ea7a3e94ffb07764f"
mode: GRACEFUL
new_leader_uuid: "276e7a6d32de44f8bb951b3b723f7cfa"
 from {username='slave'} at 127.0.0.1:58990
I20260709 13:08:54.496649 24253 raft_consensus.cc:606] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Received request to transfer leadership to TS 276e7a6d32de44f8bb951b3b723f7cfa
I20260709 13:08:54.554250 24382 consensus_queue.cc:1048] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [LEADER]: Connected to new peer: Peer: permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:54.564004 24382 raft_consensus.cc:993] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f: : Instructing follower 276e7a6d32de44f8bb951b3b723f7cfa to start an election
I20260709 13:08:54.564494 24382 raft_consensus.cc:1081] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Signalling peer 276e7a6d32de44f8bb951b3b723f7cfa to start an election
I20260709 13:08:54.567610 24177 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f8e4b5290bc944f6bff92a9413f185d6"
dest_uuid: "276e7a6d32de44f8bb951b3b723f7cfa"
 from {username='slave'} at 127.0.0.1:60622
I20260709 13:08:54.568322 24177 raft_consensus.cc:493] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:54.568802 24177 raft_consensus.cc:3060] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.572721 24383 consensus_queue.cc:1048] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [LEADER]: Connected to new peer: Peer: permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:54.581360 24177 raft_consensus.cc:515] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.582610 24178 raft_consensus.cc:1240] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Rejecting Update request from peer 8a4876fe3cda499ea7a3e94ffb07764f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:54.584420 24177 leader_election.cc:290] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [CANDIDATE]: Term 2 election: Requested vote from peers 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66:37213), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:54.584671 24253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e4b5290bc944f6bff92a9413f185d6" candidate_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a4876fe3cda499ea7a3e94ffb07764f"
I20260709 13:08:54.585376 24253 raft_consensus.cc:3055] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:54.585557 24327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e4b5290bc944f6bff92a9413f185d6" candidate_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a"
I20260709 13:08:54.585753 24253 raft_consensus.cc:740] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a4876fe3cda499ea7a3e94ffb07764f, State: Running, Role: LEADER
I20260709 13:08:54.587246 24327 raft_consensus.cc:3060] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.586803 24253 consensus_queue.cc:260] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.588166 24253 raft_consensus.cc:3060] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.597476 24327 raft_consensus.cc:2468] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 276e7a6d32de44f8bb951b3b723f7cfa in term 2.
I20260709 13:08:54.598831 24144 leader_election.cc:304] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 276e7a6d32de44f8bb951b3b723f7cfa, 31a85d150bca40e59ef64ff9f8972a1a; no voters: 
I20260709 13:08:54.598829 24253 raft_consensus.cc:2468] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 276e7a6d32de44f8bb951b3b723f7cfa in term 2.
I20260709 13:08:54.599891 24373 raft_consensus.cc:2804] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:54.600507 24373 raft_consensus.cc:697] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 LEADER]: Becoming Leader. State: Replica: 276e7a6d32de44f8bb951b3b723f7cfa, State: Running, Role: LEADER
I20260709 13:08:54.601426 24373 consensus_queue.cc:237] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.608589 24075 catalog_manager.cc:5697] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa reported cstate change: term changed from 1 to 2, leader changed from 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66) to 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65). New cstate: current_term: 2 leader_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.648362 24383 consensus_queue.cc:1048] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [LEADER]: Connected to new peer: Peer: permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:54.660111 24383 consensus_queue.cc:1048] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [LEADER]: Connected to new peer: Peer: permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:54.670258 24382 raft_consensus.cc:993] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f: : Instructing follower 31a85d150bca40e59ef64ff9f8972a1a to start an election
I20260709 13:08:54.670811 24383 raft_consensus.cc:1081] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Signalling peer 31a85d150bca40e59ef64ff9f8972a1a to start an election
I20260709 13:08:54.672557 24327 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fbb52c3e2309408ab7e0b36e793379b4"
dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a"
 from {username='slave'} at 127.0.0.1:44196
I20260709 13:08:54.673211 24327 raft_consensus.cc:493] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:54.673625 24327 raft_consensus.cc:3060] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.680716 24327 raft_consensus.cc:515] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.685571 24327 leader_election.cc:290] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [CANDIDATE]: Term 2 election: Requested vote from peers 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65:44673), 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66:37213)
I20260709 13:08:54.699726 24178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb52c3e2309408ab7e0b36e793379b4" candidate_uuid: "31a85d150bca40e59ef64ff9f8972a1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "276e7a6d32de44f8bb951b3b723f7cfa"
I20260709 13:08:54.700349 24178 raft_consensus.cc:3060] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.703850 24253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb52c3e2309408ab7e0b36e793379b4" candidate_uuid: "31a85d150bca40e59ef64ff9f8972a1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a4876fe3cda499ea7a3e94ffb07764f"
I20260709 13:08:54.704514 24253 raft_consensus.cc:3055] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:54.704816 24253 raft_consensus.cc:740] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a4876fe3cda499ea7a3e94ffb07764f, State: Running, Role: LEADER
I20260709 13:08:54.705523 24253 consensus_queue.cc:260] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.706605 24253 raft_consensus.cc:3060] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.707351 24178 raft_consensus.cc:2468] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a85d150bca40e59ef64ff9f8972a1a in term 2.
I20260709 13:08:54.708572 24293 leader_election.cc:304] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 276e7a6d32de44f8bb951b3b723f7cfa, 31a85d150bca40e59ef64ff9f8972a1a; no voters: 
I20260709 13:08:54.709504 24375 raft_consensus.cc:2804] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:54.710675 24375 raft_consensus.cc:697] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 LEADER]: Becoming Leader. State: Replica: 31a85d150bca40e59ef64ff9f8972a1a, State: Running, Role: LEADER
I20260709 13:08:54.711601 24375 consensus_queue.cc:237] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.713964 24253 raft_consensus.cc:2468] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a85d150bca40e59ef64ff9f8972a1a in term 2.
I20260709 13:08:54.719122 24074 catalog_manager.cc:5697] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a reported cstate change: term changed from 1 to 2, leader changed from 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66) to 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67). New cstate: current_term: 2 leader_uuid: "31a85d150bca40e59ef64ff9f8972a1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:54.781670 24383 consensus_queue.cc:1048] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [LEADER]: Connected to new peer: Peer: permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:54.789219 24382 raft_consensus.cc:993] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f: : Instructing follower 276e7a6d32de44f8bb951b3b723f7cfa to start an election
I20260709 13:08:54.789779 24374 raft_consensus.cc:1081] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Signalling peer 276e7a6d32de44f8bb951b3b723f7cfa to start an election
I20260709 13:08:54.791777 24177 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35968142104d4d3fa16bc4c030ee2b31"
dest_uuid: "276e7a6d32de44f8bb951b3b723f7cfa"
 from {username='slave'} at 127.0.0.1:60622
I20260709 13:08:54.796144 24177 raft_consensus.cc:493] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:54.796649 24177 raft_consensus.cc:3060] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.796387 24382 consensus_queue.cc:1048] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [LEADER]: Connected to new peer: Peer: permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:54.803956 24177 raft_consensus.cc:515] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.808985 24177 leader_election.cc:290] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [CANDIDATE]: Term 2 election: Requested vote from peers 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66:37213), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:54.810156 24326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35968142104d4d3fa16bc4c030ee2b31" candidate_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a"
I20260709 13:08:54.810890 24326 raft_consensus.cc:3060] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.811441 24373 consensus_queue.cc:1048] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:54.814005 24253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35968142104d4d3fa16bc4c030ee2b31" candidate_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a4876fe3cda499ea7a3e94ffb07764f"
I20260709 13:08:54.814679 24253 raft_consensus.cc:3055] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:54.815050 24253 raft_consensus.cc:740] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a4876fe3cda499ea7a3e94ffb07764f, State: Running, Role: LEADER
I20260709 13:08:54.815970 24253 consensus_queue.cc:260] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.817415 24253 raft_consensus.cc:3060] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.820770 24326 raft_consensus.cc:2468] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 276e7a6d32de44f8bb951b3b723f7cfa in term 2.
I20260709 13:08:54.824491 24144 leader_election.cc:304] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 276e7a6d32de44f8bb951b3b723f7cfa, 31a85d150bca40e59ef64ff9f8972a1a; no voters: 
I20260709 13:08:54.825758 24398 raft_consensus.cc:2804] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:54.833707 24398 raft_consensus.cc:697] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 LEADER]: Becoming Leader. State: Replica: 276e7a6d32de44f8bb951b3b723f7cfa, State: Running, Role: LEADER
I20260709 13:08:54.825665 24253 raft_consensus.cc:2468] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 276e7a6d32de44f8bb951b3b723f7cfa in term 2.
I20260709 13:08:54.834230 24416 consensus_queue.cc:1048] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:54.834898 24398 consensus_queue.cc:237] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:54.848542 24076 catalog_manager.cc:5697] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa reported cstate change: term changed from 1 to 2, leader changed from 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66) to 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65). New cstate: current_term: 2 leader_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:55.001314 12725 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:08:55.003137 12725 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:08:55.003607 12725 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:08:55.016049 24177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "000afa448059464b95bce1ced63f9ccd"
dest_uuid: "276e7a6d32de44f8bb951b3b723f7cfa"
mode: GRACEFUL
new_leader_uuid: "8a4876fe3cda499ea7a3e94ffb07764f"
 from {username='slave'} at 127.0.0.1:60640
I20260709 13:08:55.016662 24177 raft_consensus.cc:606] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 LEADER]: Received request to transfer leadership to TS 8a4876fe3cda499ea7a3e94ffb07764f
I20260709 13:08:55.017783 12725 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:08:55.134683 24326 raft_consensus.cc:1275] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Refusing update from remote peer 276e7a6d32de44f8bb951b3b723f7cfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:55.136153 24398 consensus_queue.cc:1048] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:55.147832 24252 raft_consensus.cc:1275] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Refusing update from remote peer 276e7a6d32de44f8bb951b3b723f7cfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:55.149710 24415 consensus_queue.cc:1048] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:55.206799 24398 raft_consensus.cc:993] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa: : Instructing follower 8a4876fe3cda499ea7a3e94ffb07764f to start an election
I20260709 13:08:55.207360 24415 raft_consensus.cc:1081] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 LEADER]: Signalling peer 8a4876fe3cda499ea7a3e94ffb07764f to start an election
I20260709 13:08:55.209244 24252 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "000afa448059464b95bce1ced63f9ccd"
dest_uuid: "8a4876fe3cda499ea7a3e94ffb07764f"
 from {username='slave'} at 127.0.0.1:58982
I20260709 13:08:55.209810 24252 raft_consensus.cc:493] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:55.210184 24252 raft_consensus.cc:3060] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:55.217253 24252 raft_consensus.cc:515] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:55.219262 24252 leader_election.cc:290] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 2 election: Requested vote from peers 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65:44673), 31a85d150bca40e59ef64ff9f8972a1a (127.12.109.67:43857)
I20260709 13:08:55.220269 24177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000afa448059464b95bce1ced63f9ccd" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "276e7a6d32de44f8bb951b3b723f7cfa"
I20260709 13:08:55.220516 24326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000afa448059464b95bce1ced63f9ccd" candidate_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31a85d150bca40e59ef64ff9f8972a1a"
I20260709 13:08:55.221064 24177 raft_consensus.cc:3055] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:55.221064 24326 raft_consensus.cc:3060] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:55.221580 24177 raft_consensus.cc:740] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 276e7a6d32de44f8bb951b3b723f7cfa, State: Running, Role: LEADER
I20260709 13:08:55.222513 24177 consensus_queue.cc:260] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:55.223127 24252 raft_consensus.cc:1275] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Refusing update from remote peer 31a85d150bca40e59ef64ff9f8972a1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:55.223913 24177 raft_consensus.cc:3060] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:55.225071 24375 consensus_queue.cc:1048] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:55.229938 24326 raft_consensus.cc:2468] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 2.
I20260709 13:08:55.232204 24177 raft_consensus.cc:2468] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a4876fe3cda499ea7a3e94ffb07764f in term 2.
I20260709 13:08:55.232455 24219 leader_election.cc:304] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a85d150bca40e59ef64ff9f8972a1a, 8a4876fe3cda499ea7a3e94ffb07764f; no voters: 
I20260709 13:08:55.233527 24382 raft_consensus.cc:2804] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:55.234134 24382 raft_consensus.cc:697] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 LEADER]: Becoming Leader. State: Replica: 8a4876fe3cda499ea7a3e94ffb07764f, State: Running, Role: LEADER
I20260709 13:08:55.235144 24382 consensus_queue.cc:237] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } }
I20260709 13:08:55.243392 24177 raft_consensus.cc:1275] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Refusing update from remote peer 31a85d150bca40e59ef64ff9f8972a1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:55.246381 24375 consensus_queue.cc:1048] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:55.249773 24075 catalog_manager.cc:5697] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f reported cstate change: term changed from 1 to 2, leader changed from 276e7a6d32de44f8bb951b3b723f7cfa (127.12.109.65) to 8a4876fe3cda499ea7a3e94ffb07764f (127.12.109.66). New cstate: current_term: 2 leader_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276e7a6d32de44f8bb951b3b723f7cfa" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 44673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:55.257349 24252 raft_consensus.cc:1275] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Refusing update from remote peer 276e7a6d32de44f8bb951b3b723f7cfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:55.260334 24415 consensus_queue.cc:1048] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a4876fe3cda499ea7a3e94ffb07764f" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:55.269618 24326 raft_consensus.cc:1275] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Refusing update from remote peer 276e7a6d32de44f8bb951b3b723f7cfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:55.271574 24416 consensus_queue.cc:1048] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "31a85d150bca40e59ef64ff9f8972a1a" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 43857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:55.389635 12725 tablet_server.cc:179] TabletServer@127.12.109.65:0 shutting down...
I20260709 13:08:55.419759 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:55.420588 12725 tablet_replica.cc:333] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa: stopping tablet replica
I20260709 13:08:55.421262 12725 raft_consensus.cc:2243] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:55.422366 12725 raft_consensus.cc:2272] T 35968142104d4d3fa16bc4c030ee2b31 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.425150 12725 tablet_replica.cc:333] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa: stopping tablet replica
I20260709 13:08:55.425791 12725 raft_consensus.cc:2243] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:55.426450 12725 raft_consensus.cc:2272] T 000afa448059464b95bce1ced63f9ccd P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.429057 12725 tablet_replica.cc:333] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa: stopping tablet replica
I20260709 13:08:55.429744 12725 raft_consensus.cc:2243] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:55.430277 12725 raft_consensus.cc:2272] T fbb52c3e2309408ab7e0b36e793379b4 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.432765 12725 tablet_replica.cc:333] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa: stopping tablet replica
I20260709 13:08:55.433272 12725 raft_consensus.cc:2243] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:55.434194 12725 raft_consensus.cc:2272] T f8e4b5290bc944f6bff92a9413f185d6 P 276e7a6d32de44f8bb951b3b723f7cfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.459093 12725 tablet_server.cc:196] TabletServer@127.12.109.65:0 shutdown complete.
I20260709 13:08:55.474025 12725 tablet_server.cc:179] TabletServer@127.12.109.66:0 shutting down...
I20260709 13:08:55.495231 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:55.496101 12725 tablet_replica.cc:333] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f: stopping tablet replica
I20260709 13:08:55.496778 12725 raft_consensus.cc:2243] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:55.497257 12725 raft_consensus.cc:2272] T 35968142104d4d3fa16bc4c030ee2b31 P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.499735 12725 tablet_replica.cc:333] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f: stopping tablet replica
I20260709 13:08:55.500254 12725 raft_consensus.cc:2243] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:55.500882 12725 pending_rounds.cc:70] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f: Trying to abort 1 pending ops.
I20260709 13:08:55.501032 12725 pending_rounds.cc:77] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305635984335495168 op_type: NO_OP noop_request { }
I20260709 13:08:55.501338 12725 raft_consensus.cc:2889] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:55.501565 12725 raft_consensus.cc:2272] T 000afa448059464b95bce1ced63f9ccd P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.503979 12725 tablet_replica.cc:333] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f: stopping tablet replica
I20260709 13:08:55.504484 12725 raft_consensus.cc:2243] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:55.504889 12725 raft_consensus.cc:2272] T f8e4b5290bc944f6bff92a9413f185d6 P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.507296 12725 tablet_replica.cc:333] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f: stopping tablet replica
I20260709 13:08:55.507751 12725 raft_consensus.cc:2243] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:55.508136 12725 raft_consensus.cc:2272] T fbb52c3e2309408ab7e0b36e793379b4 P 8a4876fe3cda499ea7a3e94ffb07764f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.522537 12725 tablet_server.cc:196] TabletServer@127.12.109.66:0 shutdown complete.
I20260709 13:08:55.536523 12725 tablet_server.cc:179] TabletServer@127.12.109.67:0 shutting down...
I20260709 13:08:55.558694 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:55.559425 12725 tablet_replica.cc:333] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a: stopping tablet replica
I20260709 13:08:55.560032 12725 raft_consensus.cc:2243] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:55.560550 12725 raft_consensus.cc:2272] T 000afa448059464b95bce1ced63f9ccd P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.563087 12725 tablet_replica.cc:333] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a: stopping tablet replica
I20260709 13:08:55.563660 12725 raft_consensus.cc:2243] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:55.564139 12725 raft_consensus.cc:2272] T 35968142104d4d3fa16bc4c030ee2b31 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.567057 12725 tablet_replica.cc:333] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a: stopping tablet replica
I20260709 13:08:55.567595 12725 raft_consensus.cc:2243] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:55.568048 12725 raft_consensus.cc:2272] T f8e4b5290bc944f6bff92a9413f185d6 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.570478 12725 tablet_replica.cc:333] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a: stopping tablet replica
I20260709 13:08:55.570994 12725 raft_consensus.cc:2243] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:55.571913 12725 raft_consensus.cc:2272] T fbb52c3e2309408ab7e0b36e793379b4 P 31a85d150bca40e59ef64ff9f8972a1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.596712 12725 tablet_server.cc:196] TabletServer@127.12.109.67:0 shutdown complete.
I20260709 13:08:55.611371 12725 master.cc:562] Master@127.12.109.126:41471 shutting down...
I20260709 13:08:55.628122 12725 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:55.628810 12725 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:55.629172 12725 tablet_replica.cc:333] T 00000000000000000000000000000000 P a6694b64064941c0a7cf6b7b0126b003: stopping tablet replica
I20260709 13:08:55.643034 12725 master.cc:584] Master@127.12.109.126:41471 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2438 ms)
[----------] 6 tests from LeaderRebalancerTest (346658 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:08:55.676227 12725 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.109.126:38639
I20260709 13:08:55.677532 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:55.683159 24421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:55.683156 24420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:55.686214 24423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:55.686367 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:55.687450 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:55.687664 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:55.687840 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602535687822 us; error 0 us; skew 500 ppm
I20260709 13:08:55.688431 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:55.691036 12725 webserver.cc:533] Webserver started at http://127.12.109.126:41139/ using document root <none> and password file <none>
I20260709 13:08:55.691572 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:55.691784 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:55.692060 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:55.693244 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/master-0-root/instance:
uuid: "d8bfccc8a5774a3e96866061556447b2"
format_stamp: "Formatted at 2026-07-09 13:08:55 on dist-test-slave-zhdc"
I20260709 13:08:55.698913 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 13:08:55.702692 24428 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:55.703473 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:08:55.703804 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/master-0-root
uuid: "d8bfccc8a5774a3e96866061556447b2"
format_stamp: "Formatted at 2026-07-09 13:08:55 on dist-test-slave-zhdc"
I20260709 13:08:55.704093 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:55.715860 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:55.717065 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:55.757325 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.126:38639
I20260709 13:08:55.757481 24479 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.126:38639 every 8 connection(s)
I20260709 13:08:55.761626 24480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:55.774175 24480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2: Bootstrap starting.
I20260709 13:08:55.779382 24480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:55.783854 24480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2: No bootstrap required, opened a new log
I20260709 13:08:55.786108 24480 raft_consensus.cc:359] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bfccc8a5774a3e96866061556447b2" member_type: VOTER }
I20260709 13:08:55.786528 24480 raft_consensus.cc:385] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:55.786700 24480 raft_consensus.cc:740] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8bfccc8a5774a3e96866061556447b2, State: Initialized, Role: FOLLOWER
I20260709 13:08:55.787230 24480 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bfccc8a5774a3e96866061556447b2" member_type: VOTER }
I20260709 13:08:55.787623 24480 raft_consensus.cc:399] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:08:55.787815 24480 raft_consensus.cc:493] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:08:55.788023 24480 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.793391 24480 raft_consensus.cc:515] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bfccc8a5774a3e96866061556447b2" member_type: VOTER }
I20260709 13:08:55.793911 24480 leader_election.cc:304] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8bfccc8a5774a3e96866061556447b2; no voters: 
I20260709 13:08:55.795166 24480 leader_election.cc:290] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:08:55.795549 24483 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:55.797050 24483 raft_consensus.cc:697] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [term 1 LEADER]: Becoming Leader. State: Replica: d8bfccc8a5774a3e96866061556447b2, State: Running, Role: LEADER
I20260709 13:08:55.797749 24483 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bfccc8a5774a3e96866061556447b2" member_type: VOTER }
I20260709 13:08:55.798442 24480 sys_catalog.cc:565] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:08:55.801075 24485 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8bfccc8a5774a3e96866061556447b2. Latest consensus state: current_term: 1 leader_uuid: "d8bfccc8a5774a3e96866061556447b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bfccc8a5774a3e96866061556447b2" member_type: VOTER } }
I20260709 13:08:55.801951 24485 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:55.801075 24484 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8bfccc8a5774a3e96866061556447b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bfccc8a5774a3e96866061556447b2" member_type: VOTER } }
I20260709 13:08:55.803083 24484 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:55.805645 24489 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:08:55.812009 24489 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:08:55.814337 12725 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:08:55.821961 24489 catalog_manager.cc:1383] Generated new cluster ID: 9963a40cd1df4b289ed1f6686498086c
I20260709 13:08:55.822265 24489 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:08:55.838104 24489 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:08:55.839594 24489 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:08:55.856063 24489 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2: Generated new TSK 0
I20260709 13:08:55.856808 24489 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:08:55.881349 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:55.888033 24501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:55.889274 24502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:55.892232 24504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:55.892268 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:55.893445 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:55.893668 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:55.893846 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602535893818 us; error 0 us; skew 500 ppm
I20260709 13:08:55.894459 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:55.896858 12725 webserver.cc:533] Webserver started at http://127.12.109.65:44867/ using document root <none> and password file <none>
I20260709 13:08:55.897384 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:55.897591 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:55.897879 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:55.899102 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-0-root/instance:
uuid: "3fc09a7ccf854d5da51529f58bd82889"
format_stamp: "Formatted at 2026-07-09 13:08:55 on dist-test-slave-zhdc"
I20260709 13:08:55.904089 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:08:55.907721 24509 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:55.908578 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:08:55.908905 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-0-root
uuid: "3fc09a7ccf854d5da51529f58bd82889"
format_stamp: "Formatted at 2026-07-09 13:08:55 on dist-test-slave-zhdc"
I20260709 13:08:55.909227 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:55.928588 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:55.929912 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:55.931615 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:55.934165 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:55.934389 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:55.934634 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:55.934806 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:55.980736 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.65:39367
I20260709 13:08:55.980952 24571 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.65:39367 every 8 connection(s)
I20260709 13:08:55.986486 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:55.994974 24576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:55.996191 24577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:55.997777 24579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:55.998019 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:56.000442 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:56.000802 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:56.001029 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602536001008 us; error 0 us; skew 500 ppm
I20260709 13:08:56.001796 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:56.002151 24572 heartbeater.cc:344] Connected to a master server at 127.12.109.126:38639
I20260709 13:08:56.002655 24572 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:56.003768 24572 heartbeater.cc:507] Master 127.12.109.126:38639 requested a full tablet report, sending...
I20260709 13:08:56.004985 12725 webserver.cc:533] Webserver started at http://127.12.109.66:44699/ using document root <none> and password file <none>
I20260709 13:08:56.005631 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:56.005982 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:56.006345 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:56.006487 24445 ts_manager.cc:194] Registered new tserver with Master: 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367)
I20260709 13:08:56.007972 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-1-root/instance:
uuid: "76f7ff986dda425baf2cae8ca9b96487"
format_stamp: "Formatted at 2026-07-09 13:08:56 on dist-test-slave-zhdc"
I20260709 13:08:56.009214 24445 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34120
I20260709 13:08:56.014340 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:08:56.018041 24584 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:56.018899 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:08:56.019207 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-1-root
uuid: "76f7ff986dda425baf2cae8ca9b96487"
format_stamp: "Formatted at 2026-07-09 13:08:56 on dist-test-slave-zhdc"
I20260709 13:08:56.019516 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:56.034891 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:56.036182 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:56.037772 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:56.040356 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:56.040596 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:56.040856 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:56.041033 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:56.087810 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.66:38999
I20260709 13:08:56.087950 24646 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.66:38999 every 8 connection(s)
I20260709 13:08:56.093734 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:56.101423 24650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:56.103570 24651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:56.108347 24647 heartbeater.cc:344] Connected to a master server at 127.12.109.126:38639
I20260709 13:08:56.109112 24647 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:56.110572 24647 heartbeater.cc:507] Master 127.12.109.126:38639 requested a full tablet report, sending...
W20260709 13:08:56.111788 24653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:56.111886 12725 server_base.cc:1061] running on GCE node
I20260709 13:08:56.113384 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:56.113646 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:56.113535 24445 ts_manager.cc:194] Registered new tserver with Master: 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:08:56.114116 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602536114087 us; error 0 us; skew 500 ppm
I20260709 13:08:56.114856 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:56.115691 24445 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34132
I20260709 13:08:56.117938 12725 webserver.cc:533] Webserver started at http://127.12.109.67:35033/ using document root <none> and password file <none>
I20260709 13:08:56.118637 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:56.118883 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:56.119180 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:56.120483 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-2-root/instance:
uuid: "bd09d620c95a44bcb4ac800897adc6f8"
format_stamp: "Formatted at 2026-07-09 13:08:56 on dist-test-slave-zhdc"
I20260709 13:08:56.125805 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:08:56.129679 24658 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:56.130632 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:08:56.131012 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-2-root
uuid: "bd09d620c95a44bcb4ac800897adc6f8"
format_stamp: "Formatted at 2026-07-09 13:08:56 on dist-test-slave-zhdc"
I20260709 13:08:56.131383 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602188851028-12725-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:56.168543 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:56.169929 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:56.171670 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:56.174419 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:56.174682 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:56.174968 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:56.175170 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:56.224195 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.67:46355
I20260709 13:08:56.224313 24720 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.67:46355 every 8 connection(s)
I20260709 13:08:56.240031 24721 heartbeater.cc:344] Connected to a master server at 127.12.109.126:38639
I20260709 13:08:56.240556 24721 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:56.241303 24721 heartbeater.cc:507] Master 127.12.109.126:38639 requested a full tablet report, sending...
I20260709 13:08:56.243484 24445 ts_manager.cc:194] Registered new tserver with Master: bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:56.243696 12725 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015932923s
I20260709 13:08:56.245713 24445 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34140
I20260709 13:08:56.274137 24445 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34154:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:08:56.277146 24445 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:08:56.352044 24612 tablet_service.cc:1511] Processing CreateTablet for tablet efc2d1bcdba24abca06cff7b82e9e10c (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:08:56.353682 24612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efc2d1bcdba24abca06cff7b82e9e10c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.355506 24610 tablet_service.cc:1511] Processing CreateTablet for tablet 81d8224ca1a24fafab3cb86dead9af56 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:08:56.357611 24610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d8224ca1a24fafab3cb86dead9af56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.353255 24611 tablet_service.cc:1511] Processing CreateTablet for tablet 5a99475fd525406090ecf4d7481216f6 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:08:56.367751 24611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a99475fd525406090ecf4d7481216f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.375758 24609 tablet_service.cc:1511] Processing CreateTablet for tablet f032c4d4202141c6b52d130b37bafbe8 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:08:56.377490 24609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f032c4d4202141c6b52d130b37bafbe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.378741 24537 tablet_service.cc:1511] Processing CreateTablet for tablet efc2d1bcdba24abca06cff7b82e9e10c (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:08:56.380383 24537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efc2d1bcdba24abca06cff7b82e9e10c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.382407 24683 tablet_service.cc:1511] Processing CreateTablet for tablet f032c4d4202141c6b52d130b37bafbe8 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:08:56.382820 24536 tablet_service.cc:1511] Processing CreateTablet for tablet 5a99475fd525406090ecf4d7481216f6 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:08:56.384073 24683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f032c4d4202141c6b52d130b37bafbe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.384186 24536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a99475fd525406090ecf4d7481216f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.387042 24535 tablet_service.cc:1511] Processing CreateTablet for tablet 81d8224ca1a24fafab3cb86dead9af56 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:08:56.388625 24535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d8224ca1a24fafab3cb86dead9af56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.391238 24534 tablet_service.cc:1511] Processing CreateTablet for tablet f032c4d4202141c6b52d130b37bafbe8 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:08:56.393652 24686 tablet_service.cc:1511] Processing CreateTablet for tablet efc2d1bcdba24abca06cff7b82e9e10c (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:08:56.395303 24686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efc2d1bcdba24abca06cff7b82e9e10c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.395044 24681 tablet_service.cc:1511] Processing CreateTablet for tablet 281735360a8f4634b84dc1e9262ea6b2 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:08:56.396529 24681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 281735360a8f4634b84dc1e9262ea6b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.397471 24685 tablet_service.cc:1511] Processing CreateTablet for tablet 5a99475fd525406090ecf4d7481216f6 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:08:56.398995 24685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a99475fd525406090ecf4d7481216f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.401273 24684 tablet_service.cc:1511] Processing CreateTablet for tablet 81d8224ca1a24fafab3cb86dead9af56 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:08:56.388644 24682 tablet_service.cc:1511] Processing CreateTablet for tablet f1d3461c689940d7a393a912f7774f94 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:08:56.401273 24680 tablet_service.cc:1511] Processing CreateTablet for tablet 62426280cfa2407189d6a8ad14efea38 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:08:56.410763 24534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f032c4d4202141c6b52d130b37bafbe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.413203 24530 tablet_service.cc:1511] Processing CreateTablet for tablet 6a990da3933643cca63dbfbd7b18205c (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:08:56.414371 24682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d3461c689940d7a393a912f7774f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.415050 24530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a990da3933643cca63dbfbd7b18205c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.415374 24533 tablet_service.cc:1511] Processing CreateTablet for tablet f1d3461c689940d7a393a912f7774f94 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:08:56.416915 24533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d3461c689940d7a393a912f7774f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.421104 24608 tablet_service.cc:1511] Processing CreateTablet for tablet f1d3461c689940d7a393a912f7774f94 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:08:56.424408 24532 tablet_service.cc:1511] Processing CreateTablet for tablet 281735360a8f4634b84dc1e9262ea6b2 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:08:56.424947 24608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d3461c689940d7a393a912f7774f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.425844 24532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 281735360a8f4634b84dc1e9262ea6b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.424610 24531 tablet_service.cc:1511] Processing CreateTablet for tablet 62426280cfa2407189d6a8ad14efea38 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:08:56.426254 24679 tablet_service.cc:1511] Processing CreateTablet for tablet 6a990da3933643cca63dbfbd7b18205c (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:08:56.427011 24531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62426280cfa2407189d6a8ad14efea38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.428836 24736 tablet_bootstrap.cc:492] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:08:56.430380 24680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62426280cfa2407189d6a8ad14efea38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.433471 24679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a990da3933643cca63dbfbd7b18205c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.437372 24607 tablet_service.cc:1511] Processing CreateTablet for tablet 281735360a8f4634b84dc1e9262ea6b2 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:08:56.438935 24607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 281735360a8f4634b84dc1e9262ea6b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.439141 24529 tablet_service.cc:1511] Processing CreateTablet for tablet 3c896072f0f34ec2b5d9a2c0ee528cde (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:08:56.439875 24678 tablet_service.cc:1511] Processing CreateTablet for tablet 3c896072f0f34ec2b5d9a2c0ee528cde (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:08:56.440563 24529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c896072f0f34ec2b5d9a2c0ee528cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.443637 24606 tablet_service.cc:1511] Processing CreateTablet for tablet 62426280cfa2407189d6a8ad14efea38 (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:08:56.445034 24606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62426280cfa2407189d6a8ad14efea38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.447722 24605 tablet_service.cc:1511] Processing CreateTablet for tablet 6a990da3933643cca63dbfbd7b18205c (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:08:56.449481 24605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a990da3933643cca63dbfbd7b18205c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.455478 24604 tablet_service.cc:1511] Processing CreateTablet for tablet 3c896072f0f34ec2b5d9a2c0ee528cde (DEFAULT_TABLE table=test-workload [id=5d83d304047c4c9fbcd66d28adc57965]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:08:56.456990 24604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c896072f0f34ec2b5d9a2c0ee528cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.437445 24684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d8224ca1a24fafab3cb86dead9af56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.496729 24678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c896072f0f34ec2b5d9a2c0ee528cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:56.531049 24736 tablet_bootstrap.cc:654] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.601871 24739 tablet_bootstrap.cc:492] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:08:56.602833 24737 tablet_bootstrap.cc:492] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:08:56.607834 24736 tablet_bootstrap.cc:492] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:08:56.608479 24736 ts_tablet_manager.cc:1403] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.180s	user 0.015s	sys 0.000s
I20260709 13:08:56.609354 24739 tablet_bootstrap.cc:654] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.614106 24736 raft_consensus.cc:359] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.614807 24739 tablet_bootstrap.cc:492] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:08:56.615005 24736 raft_consensus.cc:385] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.615653 24737 tablet_bootstrap.cc:654] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.615604 24739 ts_tablet_manager.cc:1403] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 13:08:56.615617 24736 raft_consensus.cc:740] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.616875 24736 consensus_queue.cc:260] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.619645 24739 raft_consensus.cc:359] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.620312 24736 ts_tablet_manager.cc:1434] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:08:56.620463 24739 raft_consensus.cc:385] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.620682 24737 tablet_bootstrap.cc:492] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:08:56.620959 24739 raft_consensus.cc:740] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.621263 24737 ts_tablet_manager.cc:1403] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.004s
I20260709 13:08:56.621253 24647 heartbeater.cc:499] Master 127.12.109.126:38639 was elected leader, sending a full tablet report...
I20260709 13:08:56.621552 24736 tablet_bootstrap.cc:492] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:08:56.622117 24739 consensus_queue.cc:260] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.624580 24737 raft_consensus.cc:359] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.625386 24737 raft_consensus.cc:385] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.625670 24737 raft_consensus.cc:740] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.626621 24737 consensus_queue.cc:260] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.630131 24736 tablet_bootstrap.cc:654] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.630983 24721 heartbeater.cc:499] Master 127.12.109.126:38639 was elected leader, sending a full tablet report...
I20260709 13:08:56.632112 24739 ts_tablet_manager.cc:1434] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.004s
I20260709 13:08:56.633669 24739 tablet_bootstrap.cc:492] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:08:56.641398 24572 heartbeater.cc:499] Master 127.12.109.126:38639 was elected leader, sending a full tablet report...
I20260709 13:08:56.642531 24739 tablet_bootstrap.cc:654] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.647449 24737 ts_tablet_manager.cc:1434] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:08:56.648841 24737 tablet_bootstrap.cc:492] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:08:56.649590 24739 tablet_bootstrap.cc:492] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:08:56.650779 24739 ts_tablet_manager.cc:1403] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:08:56.652522 24736 tablet_bootstrap.cc:492] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:08:56.653162 24736 ts_tablet_manager.cc:1403] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.008s
I20260709 13:08:56.654155 24739 raft_consensus.cc:359] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.654892 24739 raft_consensus.cc:385] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.655144 24739 raft_consensus.cc:740] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.656183 24737 tablet_bootstrap.cc:654] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.656021 24739 consensus_queue.cc:260] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.656183 24736 raft_consensus.cc:359] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.657315 24736 raft_consensus.cc:385] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.657665 24736 raft_consensus.cc:740] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.658825 24739 ts_tablet_manager.cc:1434] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:08:56.658699 24736 consensus_queue.cc:260] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.660617 24739 tablet_bootstrap.cc:492] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:08:56.663240 24736 ts_tablet_manager.cc:1434] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 13:08:56.664407 24736 tablet_bootstrap.cc:492] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:08:56.667799 24739 tablet_bootstrap.cc:654] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.669313 24737 tablet_bootstrap.cc:492] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:08:56.669955 24737 ts_tablet_manager.cc:1403] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 13:08:56.671119 24736 tablet_bootstrap.cc:654] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.673492 24737 raft_consensus.cc:359] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.674538 24737 raft_consensus.cc:385] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.674885 24737 raft_consensus.cc:740] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.675900 24737 consensus_queue.cc:260] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.678519 24737 ts_tablet_manager.cc:1434] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:08:56.679587 24737 tablet_bootstrap.cc:492] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:08:56.686275 24737 tablet_bootstrap.cc:654] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.698133 24737 tablet_bootstrap.cc:492] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:08:56.698822 24737 ts_tablet_manager.cc:1403] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.019s	user 0.004s	sys 0.014s
I20260709 13:08:56.701977 24737 raft_consensus.cc:359] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.702968 24737 raft_consensus.cc:385] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.703342 24737 raft_consensus.cc:740] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.704324 24737 consensus_queue.cc:260] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.709316 24739 tablet_bootstrap.cc:492] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:08:56.709939 24739 ts_tablet_manager.cc:1403] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.050s	user 0.027s	sys 0.019s
I20260709 13:08:56.710857 24736 tablet_bootstrap.cc:492] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:08:56.711472 24736 ts_tablet_manager.cc:1403] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.047s	user 0.017s	sys 0.029s
I20260709 13:08:56.713171 24739 raft_consensus.cc:359] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.713949 24737 ts_tablet_manager.cc:1434] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:08:56.714167 24739 raft_consensus.cc:385] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.714684 24739 raft_consensus.cc:740] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.714633 24736 raft_consensus.cc:359] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.715288 24737 tablet_bootstrap.cc:492] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:08:56.715612 24736 raft_consensus.cc:385] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.715982 24736 raft_consensus.cc:740] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.716053 24739 consensus_queue.cc:260] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.716866 24736 consensus_queue.cc:260] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.719336 24736 ts_tablet_manager.cc:1434] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:08:56.720464 24736 tablet_bootstrap.cc:492] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:08:56.724756 24737 tablet_bootstrap.cc:654] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.726552 24739 ts_tablet_manager.cc:1434] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:08:56.730024 24739 tablet_bootstrap.cc:492] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:08:56.730315 24736 tablet_bootstrap.cc:654] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.739368 24739 tablet_bootstrap.cc:654] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.739518 24737 tablet_bootstrap.cc:492] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:08:56.740401 24737 ts_tablet_manager.cc:1403] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.005s
I20260709 13:08:56.745373 24737 raft_consensus.cc:359] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.747161 24737 raft_consensus.cc:385] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.747584 24737 raft_consensus.cc:740] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.748462 24737 consensus_queue.cc:260] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.749375 24739 tablet_bootstrap.cc:492] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:08:56.750133 24739 ts_tablet_manager.cc:1403] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.010s
I20260709 13:08:56.749377 24736 tablet_bootstrap.cc:492] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:08:56.750801 24737 ts_tablet_manager.cc:1434] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:08:56.750873 24736 ts_tablet_manager.cc:1403] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.012s
I20260709 13:08:56.752228 24737 tablet_bootstrap.cc:492] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:08:56.753757 24739 raft_consensus.cc:359] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.754719 24739 raft_consensus.cc:385] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.754508 24736 raft_consensus.cc:359] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.755182 24739 raft_consensus.cc:740] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.755519 24736 raft_consensus.cc:385] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.756032 24736 raft_consensus.cc:740] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.756316 24739 consensus_queue.cc:260] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.757094 24736 consensus_queue.cc:260] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.759934 24737 tablet_bootstrap.cc:654] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.761139 24744 raft_consensus.cc:493] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.761893 24739 ts_tablet_manager.cc:1434] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:08:56.761732 24744 raft_consensus.cc:515] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.763236 24739 tablet_bootstrap.cc:492] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:08:56.764766 24736 ts_tablet_manager.cc:1434] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:08:56.765986 24736 tablet_bootstrap.cc:492] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:08:56.766803 24744 leader_election.cc:290] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:56.772867 24743 raft_consensus.cc:493] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.773543 24743 raft_consensus.cc:515] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.781036 24736 tablet_bootstrap.cc:654] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.785383 24743 leader_election.cc:290] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367)
W20260709 13:08:56.810487 24493 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:56.813598 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c896072f0f34ec2b5d9a2c0ee528cde" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:08:56.815001 24739 tablet_bootstrap.cc:654] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:56.815852 24512 leader_election.cc:343] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.816704 24743 raft_consensus.cc:493] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.817260 24743 raft_consensus.cc:515] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.819692 24743 leader_election.cc:290] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:08:56.823628 24737 tablet_bootstrap.cc:492] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:08:56.825580 24737 ts_tablet_manager.cc:1403] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.074s	user 0.027s	sys 0.011s
I20260709 13:08:56.828917 24737 raft_consensus.cc:359] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.829676 24737 raft_consensus.cc:385] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.830044 24737 raft_consensus.cc:740] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.830451 24547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2d1bcdba24abca06cff7b82e9e10c" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:08:56.830892 24737 consensus_queue.cc:260] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.832125 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d3461c689940d7a393a912f7774f94" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:08:56.833397 24737 ts_tablet_manager.cc:1434] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260709 13:08:56.829710 24742 raft_consensus.cc:493] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:56.834120 24660 leader_election.cc:343] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.834491 24742 raft_consensus.cc:515] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
W20260709 13:08:56.835021 24660 leader_election.cc:343] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.835551 24739 tablet_bootstrap.cc:492] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:08:56.836163 24739 ts_tablet_manager.cc:1403] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.073s	user 0.012s	sys 0.006s
I20260709 13:08:56.837109 24736 tablet_bootstrap.cc:492] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:08:56.837653 24736 ts_tablet_manager.cc:1403] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.072s	user 0.011s	sys 0.016s
I20260709 13:08:56.840391 24737 tablet_bootstrap.cc:492] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:08:56.841341 24742 leader_election.cc:290] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:56.844051 24622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c896072f0f34ec2b5d9a2c0ee528cde" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:08:56.841125 24736 raft_consensus.cc:359] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.847312 24736 raft_consensus.cc:385] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.848307 24736 raft_consensus.cc:740] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.849566 24736 consensus_queue.cc:260] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.847267 24739 raft_consensus.cc:359] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.851861 24739 raft_consensus.cc:385] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.852246 24739 raft_consensus.cc:740] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.852419 24737 tablet_bootstrap.cc:654] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.853189 24739 consensus_queue.cc:260] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.855893 24739 ts_tablet_manager.cc:1434] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:08:56.855916 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2d1bcdba24abca06cff7b82e9e10c" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:08:56.856849 24621 raft_consensus.cc:2468] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 0.
I20260709 13:08:56.857164 24739 tablet_bootstrap.cc:492] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:08:56.858431 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d3461c689940d7a393a912f7774f94" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:08:56.858462 24622 raft_consensus.cc:2468] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 0.
I20260709 13:08:56.863020 24662 leader_election.cc:304] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 3fc09a7ccf854d5da51529f58bd82889
I20260709 13:08:56.865051 24743 raft_consensus.cc:2804] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.865557 24743 raft_consensus.cc:493] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.865990 24743 raft_consensus.cc:3060] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.866436 24736 ts_tablet_manager.cc:1434] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.028s	user 0.001s	sys 0.009s
I20260709 13:08:56.867062 24513 leader_election.cc:304] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:08:56.867686 24736 tablet_bootstrap.cc:492] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:08:56.868542 24744 raft_consensus.cc:493] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.869087 24744 raft_consensus.cc:515] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
W20260709 13:08:56.877216 24662 leader_election.cc:343] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.879365 24662 leader_election.cc:304] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd09d620c95a44bcb4ac800897adc6f8; no voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:08:56.880766 24744 leader_election.cc:290] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:56.886237 24744 raft_consensus.cc:2804] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.890255 24744 raft_consensus.cc:493] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.890668 24744 raft_consensus.cc:3060] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.899704 24622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f032c4d4202141c6b52d130b37bafbe8" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:08:56.901237 24622 raft_consensus.cc:2468] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 0.
I20260709 13:08:56.901063 24743 raft_consensus.cc:515] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.902365 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62426280cfa2407189d6a8ad14efea38" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:08:56.902781 24513 leader_election.cc:304] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:08:56.903055 24546 raft_consensus.cc:2468] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:08:56.903682 24737 tablet_bootstrap.cc:492] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:08:56.904284 24737 ts_tablet_manager.cc:1403] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.064s	user 0.005s	sys 0.020s
I20260709 13:08:56.905287 24586 leader_election.cc:304] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:08:56.906738 24736 tablet_bootstrap.cc:654] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.906636 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2d1bcdba24abca06cff7b82e9e10c" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:08:56.906904 24765 raft_consensus.cc:2804] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.907349 24765 raft_consensus.cc:493] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.907629 24742 raft_consensus.cc:2804] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.907686 24765 raft_consensus.cc:3060] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:56.908331 24660 leader_election.cc:343] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.908154 24744 raft_consensus.cc:515] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.908342 24742 raft_consensus.cc:493] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.909286 24742 raft_consensus.cc:3060] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.910161 24622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2d1bcdba24abca06cff7b82e9e10c" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:08:56.910486 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f032c4d4202141c6b52d130b37bafbe8" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:08:56.910859 24622 raft_consensus.cc:3060] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.911564 24696 raft_consensus.cc:2468] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 0.
I20260709 13:08:56.913053 24744 leader_election.cc:290] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 election: Requested vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:56.910698 24737 raft_consensus.cc:359] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.914028 24737 raft_consensus.cc:385] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.914383 24737 raft_consensus.cc:740] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.914894 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c896072f0f34ec2b5d9a2c0ee528cde" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:08:56.915553 24621 raft_consensus.cc:3060] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.915200 24737 consensus_queue.cc:260] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.919993 24765 raft_consensus.cc:515] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.920451 24742 raft_consensus.cc:515] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.923029 24744 raft_consensus.cc:493] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.923595 24744 raft_consensus.cc:515] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.924204 24621 raft_consensus.cc:2468] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 1.
I20260709 13:08:56.925777 24744 leader_election.cc:290] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:56.926016 24620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f032c4d4202141c6b52d130b37bafbe8" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:08:56.927023 24620 raft_consensus.cc:3060] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.930572 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c896072f0f34ec2b5d9a2c0ee528cde" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:08:56.932026 24513 leader_election.cc:304] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:08:56.933272 24765 leader_election.cc:290] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 election: Requested vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:56.933813 24739 tablet_bootstrap.cc:654] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.934312 24765 raft_consensus.cc:2804] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:56.935047 24765 raft_consensus.cc:697] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 1 LEADER]: Becoming Leader. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:08:56.932026 24766 raft_consensus.cc:2749] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:56.936828 24765 consensus_queue.cc:237] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.929600 24743 leader_election.cc:290] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Requested vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367)
W20260709 13:08:56.940133 24512 leader_election.cc:343] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.944489 24622 raft_consensus.cc:2468] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 1.
I20260709 13:08:56.955065 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "281735360a8f4634b84dc1e9262ea6b2" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:08:56.956060 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62426280cfa2407189d6a8ad14efea38" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:08:56.956210 24696 raft_consensus.cc:2468] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 0.
I20260709 13:08:56.956884 24546 raft_consensus.cc:3060] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.957262 24695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62426280cfa2407189d6a8ad14efea38" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:08:56.958011 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f032c4d4202141c6b52d130b37bafbe8" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:08:56.958633 24696 raft_consensus.cc:3060] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:56.959129 24587 leader_election.cc:343] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.961161 24512 leader_election.cc:304] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:08:56.959203 24737 ts_tablet_manager.cc:1434] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.055s	user 0.023s	sys 0.023s
I20260709 13:08:56.954993 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "281735360a8f4634b84dc1e9262ea6b2" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:08:56.962308 24744 raft_consensus.cc:2804] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.961338 24662 leader_election.cc:304] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 3fc09a7ccf854d5da51529f58bd82889
I20260709 13:08:56.962879 24744 raft_consensus.cc:493] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.964103 24744 raft_consensus.cc:3060] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:56.964154 24513 leader_election.cc:343] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.965677 24743 raft_consensus.cc:2804] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:56.966454 24743 raft_consensus.cc:697] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:08:56.967702 24743 consensus_queue.cc:237] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.969223 24737 tablet_bootstrap.cc:492] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:08:56.971272 24695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62426280cfa2407189d6a8ad14efea38" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:08:56.963378 24742 leader_election.cc:290] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:56.953071 24620 raft_consensus.cc:2468] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 1.
I20260709 13:08:56.971370 24736 tablet_bootstrap.cc:492] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
W20260709 13:08:56.975267 24587 leader_election.cc:343] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.980377 24513 leader_election.cc:304] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:08:56.981876 24767 raft_consensus.cc:2804] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:56.985056 24737 tablet_bootstrap.cc:654] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.986403 24739 tablet_bootstrap.cc:492] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:08:56.987114 24739 ts_tablet_manager.cc:1403] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.130s	user 0.019s	sys 0.011s
I20260709 13:08:56.988317 24767 raft_consensus.cc:697] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 LEADER]: Becoming Leader. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:08:56.989882 24767 consensus_queue.cc:237] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:56.978513 24736 ts_tablet_manager.cc:1403] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.111s	user 0.010s	sys 0.007s
I20260709 13:08:56.979161 24696 raft_consensus.cc:2468] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 1.
I20260709 13:08:57.003089 24546 raft_consensus.cc:2468] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 1.
I20260709 13:08:57.001807 24744 raft_consensus.cc:515] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.007064 24586 leader_election.cc:304] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:08:57.005537 24443 catalog_manager.cc:5697] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "3fc09a7ccf854d5da51529f58bd82889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:57.007505 24736 raft_consensus.cc:359] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.008575 24736 raft_consensus.cc:385] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.009167 24736 raft_consensus.cc:740] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.010354 24742 raft_consensus.cc:2804] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:57.014222 24739 raft_consensus.cc:359] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.015072 24739 raft_consensus.cc:385] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.015398 24739 raft_consensus.cc:740] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.016008 24620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "281735360a8f4634b84dc1e9262ea6b2" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:08:57.016286 24739 consensus_queue.cc:260] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.016121 24444 catalog_manager.cc:5697] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:57.017788 24766 raft_consensus.cc:493] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.018291 24744 leader_election.cc:290] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 election: Requested vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:57.018321 24766 raft_consensus.cc:515] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.019645 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "281735360a8f4634b84dc1e9262ea6b2" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:08:57.020481 24696 raft_consensus.cc:3060] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.011607 24736 consensus_queue.cc:260] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.021492 24620 raft_consensus.cc:3060] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.023002 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a99475fd525406090ecf4d7481216f6" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:08:57.024087 24766 leader_election.cc:290] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
W20260709 13:08:57.025219 24662 leader_election.cc:343] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.027371 24739 ts_tablet_manager.cc:1434] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.040s	user 0.004s	sys 0.005s
I20260709 13:08:57.027413 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a99475fd525406090ecf4d7481216f6" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:08:57.028730 24739 tablet_bootstrap.cc:492] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:08:57.028539 24546 raft_consensus.cc:2468] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 0.
I20260709 13:08:57.029837 24620 raft_consensus.cc:2468] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 1.
I20260709 13:08:57.030859 24660 leader_election.cc:304] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:08:57.032830 24766 raft_consensus.cc:2804] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.033330 24766 raft_consensus.cc:493] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.033699 24766 raft_consensus.cc:3060] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.039366 24696 raft_consensus.cc:2468] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 1.
I20260709 13:08:57.040979 24512 leader_election.cc:304] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:08:57.041882 24744 raft_consensus.cc:2804] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:57.046211 24744 raft_consensus.cc:697] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 LEADER]: Becoming Leader. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:08:57.049479 24744 consensus_queue.cc:237] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.053350 24739 tablet_bootstrap.cc:654] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.055461 24737 tablet_bootstrap.cc:492] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:08:57.056294 24737 ts_tablet_manager.cc:1403] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.087s	user 0.022s	sys 0.004s
I20260709 13:08:57.062633 24766 raft_consensus.cc:515] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.062649 24737 raft_consensus.cc:359] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.060920 24742 raft_consensus.cc:697] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:08:57.064355 24736 ts_tablet_manager.cc:1434] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.070s	user 0.011s	sys 0.014s
I20260709 13:08:57.064811 24737 raft_consensus.cc:385] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.065512 24742 consensus_queue.cc:237] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.066996 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a99475fd525406090ecf4d7481216f6" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:08:57.067281 24737 raft_consensus.cc:740] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.067726 24546 raft_consensus.cc:3060] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.068559 24766 leader_election.cc:290] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:08:57.069219 24736 tablet_bootstrap.cc:492] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:08:57.071794 24620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a99475fd525406090ecf4d7481216f6" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
W20260709 13:08:57.073355 24662 leader_election.cc:343] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.074326 24737 consensus_queue.cc:260] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.081933 24737 ts_tablet_manager.cc:1434] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.002s
I20260709 13:08:57.085017 24778 raft_consensus.cc:493] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.085827 24778 raft_consensus.cc:515] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.087112 24737 tablet_bootstrap.cc:492] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:08:57.096558 24766 raft_consensus.cc:493] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.097224 24766 raft_consensus.cc:515] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.097913 24737 tablet_bootstrap.cc:654] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.098724 24778 leader_election.cc:290] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:57.101938 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:08:57.099066 24547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
W20260709 13:08:57.104588 24587 leader_election.cc:343] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.111146 24545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:08:57.112218 24545 raft_consensus.cc:2468] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 0.
I20260709 13:08:57.110518 24547 raft_consensus.cc:2468] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:08:57.114346 24660 leader_election.cc:304] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:08:57.114748 24766 leader_election.cc:290] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:08:57.115445 24766 raft_consensus.cc:2804] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.115237 24586 leader_election.cc:304] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:08:57.115847 24766 raft_consensus.cc:493] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.116281 24766 raft_consensus.cc:3060] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.121790 24620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:08:57.122591 24736 tablet_bootstrap.cc:654] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:57.123857 24662 leader_election.cc:343] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.124771 24739 tablet_bootstrap.cc:492] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:08:57.125275 24739 ts_tablet_manager.cc:1403] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.097s	user 0.018s	sys 0.024s
I20260709 13:08:57.118500 24742 raft_consensus.cc:2804] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.126451 24742 raft_consensus.cc:493] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.126896 24742 raft_consensus.cc:3060] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.128649 24739 raft_consensus.cc:359] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.129541 24739 raft_consensus.cc:385] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.134579 24546 raft_consensus.cc:2468] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 1.
I20260709 13:08:57.134760 24444 catalog_manager.cc:5697] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:57.137195 24766 raft_consensus.cc:515] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.129876 24739 raft_consensus.cc:740] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.143988 24739 consensus_queue.cc:260] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.144452 24660 leader_election.cc:304] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:08:57.145239 24737 tablet_bootstrap.cc:492] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:08:57.145895 24737 ts_tablet_manager.cc:1403] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.059s	user 0.015s	sys 0.005s
I20260709 13:08:57.148109 24620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:08:57.149317 24739 ts_tablet_manager.cc:1434] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:08:57.149820 24766 leader_election.cc:290] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
W20260709 13:08:57.150209 24662 leader_election.cc:343] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.150024 24737 raft_consensus.cc:359] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.151528 24737 raft_consensus.cc:385] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.151881 24737 raft_consensus.cc:740] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.152844 24737 consensus_queue.cc:260] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.152945 24739 tablet_bootstrap.cc:492] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:08:57.156733 24737 ts_tablet_manager.cc:1434] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:08:57.158352 24737 tablet_bootstrap.cc:492] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:08:57.161751 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:08:57.163204 24546 raft_consensus.cc:3060] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.167455 24443 catalog_manager.cc:5697] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "3fc09a7ccf854d5da51529f58bd82889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:57.169842 24443 catalog_manager.cc:5697] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65). New cstate: current_term: 1 leader_uuid: "3fc09a7ccf854d5da51529f58bd82889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:57.173050 24736 tablet_bootstrap.cc:492] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:08:57.164716 24739 tablet_bootstrap.cc:654] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.174815 24737 tablet_bootstrap.cc:654] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.174975 24736 ts_tablet_manager.cc:1403] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.106s	user 0.020s	sys 0.017s
I20260709 13:08:57.178633 24546 raft_consensus.cc:2468] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 1.
I20260709 13:08:57.180294 24660 leader_election.cc:304] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:08:57.181578 24736 raft_consensus.cc:359] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.181659 24743 raft_consensus.cc:2804] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:57.195115 24737 tablet_bootstrap.cc:492] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:08:57.189006 24739 tablet_bootstrap.cc:492] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:08:57.198772 24737 ts_tablet_manager.cc:1403] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.009s
I20260709 13:08:57.191839 24742 raft_consensus.cc:515] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.199079 24739 ts_tablet_manager.cc:1403] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.046s	user 0.021s	sys 0.001s
I20260709 13:08:57.197790 24736 raft_consensus.cc:385] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.201431 24736 raft_consensus.cc:740] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.153716 24766 raft_consensus.cc:2804] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:57.202163 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:08:57.202996 24546 raft_consensus.cc:3060] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.203711 24742 leader_election.cc:290] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:57.203939 24739 raft_consensus.cc:359] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.204879 24739 raft_consensus.cc:385] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.205246 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:08:57.203002 24736 consensus_queue.cc:260] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
W20260709 13:08:57.207424 24587 leader_election.cc:343] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.202301 24737 raft_consensus.cc:359] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.208369 24737 raft_consensus.cc:385] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.208662 24737 raft_consensus.cc:740] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.209446 24737 consensus_queue.cc:260] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.205255 24739 raft_consensus.cc:740] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.209820 24736 ts_tablet_manager.cc:1434] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.034s	user 0.010s	sys 0.004s
I20260709 13:08:57.211438 24739 consensus_queue.cc:260] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.213128 24736 tablet_bootstrap.cc:492] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:08:57.216342 24737 ts_tablet_manager.cc:1434] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:08:57.213563 24743 raft_consensus.cc:697] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:08:57.217626 24743 consensus_queue.cc:237] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.219002 24546 raft_consensus.cc:2468] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 1.
I20260709 13:08:57.220741 24586 leader_election.cc:304] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:08:57.225216 24739 ts_tablet_manager.cc:1434] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.006s
I20260709 13:08:57.226457 24739 tablet_bootstrap.cc:492] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:08:57.231769 24736 tablet_bootstrap.cc:654] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.237306 24742 raft_consensus.cc:2804] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:57.238781 24742 raft_consensus.cc:697] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:08:57.239748 24742 consensus_queue.cc:237] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.237644 24739 tablet_bootstrap.cc:654] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.246007 24766 raft_consensus.cc:697] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:08:57.247535 24766 consensus_queue.cc:237] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.252279 24736 tablet_bootstrap.cc:492] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:08:57.252993 24736 ts_tablet_manager.cc:1403] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.041s	user 0.023s	sys 0.001s
I20260709 13:08:57.255800 24739 tablet_bootstrap.cc:492] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:08:57.256397 24739 ts_tablet_manager.cc:1403] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.006s
I20260709 13:08:57.257299 24736 raft_consensus.cc:359] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.259315 24736 raft_consensus.cc:385] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.259651 24736 raft_consensus.cc:740] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.259403 24739 raft_consensus.cc:359] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.260200 24739 raft_consensus.cc:385] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.260499 24739 raft_consensus.cc:740] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.260588 24736 consensus_queue.cc:260] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.261180 24739 consensus_queue.cc:260] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.263110 24736 ts_tablet_manager.cc:1434] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:08:57.263381 24739 ts_tablet_manager.cc:1434] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.006s
I20260709 13:08:57.262539 24443 catalog_manager.cc:5697] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:57.264226 24736 tablet_bootstrap.cc:492] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:08:57.266151 24445 catalog_manager.cc:5697] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:57.268465 24445 catalog_manager.cc:5697] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:57.272208 24736 tablet_bootstrap.cc:654] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.277819 24736 tablet_bootstrap.cc:492] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:08:57.278420 24736 ts_tablet_manager.cc:1403] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.014s	user 0.014s	sys 0.000s
I20260709 13:08:57.281721 24736 raft_consensus.cc:359] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.282538 24736 raft_consensus.cc:385] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.282823 24736 raft_consensus.cc:740] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.283630 24736 consensus_queue.cc:260] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.285681 24736 ts_tablet_manager.cc:1434] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:08:57.304962 24778 raft_consensus.cc:493] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.305531 24778 raft_consensus.cc:515] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.307547 24778 leader_election.cc:290] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:57.308737 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a99475fd525406090ecf4d7481216f6" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:08:57.308487 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a99475fd525406090ecf4d7481216f6" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:08:57.309754 24546 raft_consensus.cc:2393] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76f7ff986dda425baf2cae8ca9b96487 in current term 1: Already voted for candidate bd09d620c95a44bcb4ac800897adc6f8 in this term.
I20260709 13:08:57.310086 24744 raft_consensus.cc:493] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.310669 24744 raft_consensus.cc:515] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.311139 24586 leader_election.cc:304] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487; no voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:08:57.312033 24778 raft_consensus.cc:3060] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.313091 24744 leader_election.cc:290] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:57.313710 24620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2d1bcdba24abca06cff7b82e9e10c" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:08:57.314544 24620 raft_consensus.cc:2393] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fc09a7ccf854d5da51529f58bd82889 in current term 1: Already voted for candidate bd09d620c95a44bcb4ac800897adc6f8 in this term.
I20260709 13:08:57.314380 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2d1bcdba24abca06cff7b82e9e10c" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:08:57.315796 24512 leader_election.cc:304] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889; no voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:08:57.316280 24766 raft_consensus.cc:493] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.316605 24744 raft_consensus.cc:3060] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.316838 24766 raft_consensus.cc:515] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.319973 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c896072f0f34ec2b5d9a2c0ee528cde" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:08:57.320150 24766 leader_election.cc:290] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:08:57.321898 24620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c896072f0f34ec2b5d9a2c0ee528cde" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:08:57.322822 24620 raft_consensus.cc:2393] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd09d620c95a44bcb4ac800897adc6f8 in current term 1: Already voted for candidate 3fc09a7ccf854d5da51529f58bd82889 in this term.
I20260709 13:08:57.323843 24662 leader_election.cc:304] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd09d620c95a44bcb4ac800897adc6f8; no voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:08:57.324218 24778 raft_consensus.cc:2749] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:57.324766 24766 raft_consensus.cc:3060] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.328863 24744 raft_consensus.cc:2749] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:57.331741 24766 raft_consensus.cc:2749] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:57.340013 24744 consensus_queue.cc:1048] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.352051 24744 consensus_queue.cc:1048] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.358151 24766 raft_consensus.cc:493] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.358848 24766 raft_consensus.cc:515] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.361953 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62426280cfa2407189d6a8ad14efea38" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:08:57.362495 24766 leader_election.cc:290] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:08:57.362789 24546 raft_consensus.cc:2393] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd09d620c95a44bcb4ac800897adc6f8 in current term 1: Already voted for candidate 76f7ff986dda425baf2cae8ca9b96487 in this term.
I20260709 13:08:57.363677 24620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62426280cfa2407189d6a8ad14efea38" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:08:57.365036 24662 leader_election.cc:304] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd09d620c95a44bcb4ac800897adc6f8; no voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:08:57.365880 24766 raft_consensus.cc:3060] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.372359 24766 raft_consensus.cc:2749] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:57.423247 24778 raft_consensus.cc:493] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.423938 24778 raft_consensus.cc:515] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.426294 24778 leader_election.cc:290] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:57.426203 24766 consensus_queue.cc:1048] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.427160 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:08:57.427481 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:08:57.427917 24546 raft_consensus.cc:2393] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76f7ff986dda425baf2cae8ca9b96487 in current term 1: Already voted for candidate bd09d620c95a44bcb4ac800897adc6f8 in this term.
I20260709 13:08:57.429265 24586 leader_election.cc:304] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487; no voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:08:57.430131 24778 raft_consensus.cc:3060] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.440003 24778 raft_consensus.cc:2749] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:57.440944 24766 consensus_queue.cc:1048] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.455740 24767 consensus_queue.cc:1048] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.463069 24778 raft_consensus.cc:493] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.463644 24778 raft_consensus.cc:515] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.466198 24778 leader_election.cc:290] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:57.467427 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d3461c689940d7a393a912f7774f94" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:08:57.468238 24696 raft_consensus.cc:2468] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:08:57.469393 24587 leader_election.cc:304] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:08:57.470355 24778 raft_consensus.cc:2804] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.470754 24778 raft_consensus.cc:493] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.471130 24778 raft_consensus.cc:3060] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.470860 24771 consensus_queue.cc:1048] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.473035 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d3461c689940d7a393a912f7774f94" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:08:57.473850 24546 raft_consensus.cc:2468] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:08:57.478884 24771 consensus_queue.cc:1048] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.479728 24778 raft_consensus.cc:515] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.481909 24778 leader_election.cc:290] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:57.484396 24694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d3461c689940d7a393a912f7774f94" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:08:57.485311 24694 raft_consensus.cc:3060] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.485813 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d3461c689940d7a393a912f7774f94" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:08:57.486631 24546 raft_consensus.cc:3060] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.495195 24546 raft_consensus.cc:2468] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 1.
I20260709 13:08:57.496877 24586 leader_election.cc:304] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:08:57.497973 24778 raft_consensus.cc:2804] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:57.498502 24778 raft_consensus.cc:697] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:08:57.499492 24771 consensus_queue.cc:1048] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.499423 24778 consensus_queue.cc:237] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.500262 24694 raft_consensus.cc:2468] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 1.
I20260709 13:08:57.517992 24443 catalog_manager.cc:5697] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:57.529975 24778 consensus_queue.cc:1048] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.542104 24742 consensus_queue.cc:1048] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.543000 24766 raft_consensus.cc:493] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.543579 24766 raft_consensus.cc:515] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.547417 24547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:08:57.548475 24547 raft_consensus.cc:2393] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd09d620c95a44bcb4ac800897adc6f8 in current term 1: Already voted for candidate 76f7ff986dda425baf2cae8ca9b96487 in this term.
I20260709 13:08:57.550810 24766 leader_election.cc:290] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:08:57.552186 24620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:08:57.553881 24662 leader_election.cc:304] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd09d620c95a44bcb4ac800897adc6f8; no voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:08:57.555438 24766 raft_consensus.cc:3060] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.585413 24766 raft_consensus.cc:2749] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:57.592954 24695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efc2d1bcdba24abca06cff7b82e9e10c"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
mode: GRACEFUL
new_leader_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:45626
I20260709 13:08:57.593782 24695 raft_consensus.cc:606] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Received request to transfer leadership to TS 3fc09a7ccf854d5da51529f58bd82889
I20260709 13:08:57.600121 24695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a99475fd525406090ecf4d7481216f6"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
mode: GRACEFUL
new_leader_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:45626
I20260709 13:08:57.600857 24695 raft_consensus.cc:606] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Received request to transfer leadership to TS 3fc09a7ccf854d5da51529f58bd82889
I20260709 13:08:57.616214 24620 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56"
dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
mode: GRACEFUL
new_leader_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:38818
I20260709 13:08:57.616796 24620 raft_consensus.cc:606] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Received request to transfer leadership to TS 3fc09a7ccf854d5da51529f58bd82889
I20260709 13:08:57.635430 24766 consensus_queue.cc:1048] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.638293 24547 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "281735360a8f4634b84dc1e9262ea6b2"
dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
mode: GRACEFUL
new_leader_uuid: "76f7ff986dda425baf2cae8ca9b96487"
 from {username='slave'} at 127.0.0.1:48028
I20260709 13:08:57.638935 24547 raft_consensus.cc:606] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 LEADER]: Received request to transfer leadership to TS 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:08:57.644086 24790 raft_consensus.cc:993] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8: : Instructing follower 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:08:57.644570 24743 raft_consensus.cc:1081] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Signalling peer 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:08:57.646348 24546 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a99475fd525406090ecf4d7481216f6"
dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:48012
I20260709 13:08:57.647022 24546 raft_consensus.cc:493] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:57.647478 24546 raft_consensus.cc:3060] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.650837 24695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
mode: GRACEFUL
new_leader_uuid: "76f7ff986dda425baf2cae8ca9b96487"
 from {username='slave'} at 127.0.0.1:45626
I20260709 13:08:57.651297 24743 consensus_queue.cc:1048] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.651605 24695 raft_consensus.cc:606] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Received request to transfer leadership to TS 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:08:57.659869 24546 raft_consensus.cc:515] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.661103 24547 raft_consensus.cc:1240] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Rejecting Update request from peer bd09d620c95a44bcb4ac800897adc6f8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:57.662395 24546 leader_election.cc:290] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Requested vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:57.663000 24790 consensus_queue.cc:1059] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.663954 24790 raft_consensus.cc:3055] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:57.664292 24790 raft_consensus.cc:740] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:08:57.664441 24695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a99475fd525406090ecf4d7481216f6" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:08:57.663281 24620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a99475fd525406090ecf4d7481216f6" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:08:57.665177 24790 consensus_queue.cc:260] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.665791 24620 raft_consensus.cc:3060] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.667869 24743 consensus_queue.cc:1048] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.668462 24790 raft_consensus.cc:3060] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.670348 24545 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c896072f0f34ec2b5d9a2c0ee528cde"
dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
mode: GRACEFUL
new_leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
 from {username='slave'} at 127.0.0.1:48028
I20260709 13:08:57.670953 24545 raft_consensus.cc:606] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 1 LEADER]: Received request to transfer leadership to TS bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:08:57.676278 24620 raft_consensus.cc:2468] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:08:57.684082 24513 leader_election.cc:304] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:08:57.685102 24767 raft_consensus.cc:2804] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:57.685631 24767 raft_consensus.cc:697] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Becoming Leader. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:08:57.686614 24767 consensus_queue.cc:237] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.691045 24766 consensus_queue.cc:1048] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.692392 24695 raft_consensus.cc:2468] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:08:57.699055 24790 raft_consensus.cc:993] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8: : Instructing follower 76f7ff986dda425baf2cae8ca9b96487 to start an election
I20260709 13:08:57.699731 24788 raft_consensus.cc:1081] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Signalling peer 76f7ff986dda425baf2cae8ca9b96487 to start an election
I20260709 13:08:57.699956 24443 catalog_manager.cc:5697] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 reported cstate change: term changed from 1 to 2, leader changed from bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67) to 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "3fc09a7ccf854d5da51529f58bd82889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:57.701323 24621 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c"
dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
 from {username='slave'} at 127.0.0.1:38802
I20260709 13:08:57.701913 24621 raft_consensus.cc:493] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:57.702277 24621 raft_consensus.cc:3060] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.708848 24621 raft_consensus.cc:515] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.711082 24621 leader_election.cc:290] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 2 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:57.711982 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:08:57.712498 24695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:08:57.712733 24546 raft_consensus.cc:3060] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.713171 24695 raft_consensus.cc:3055] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:57.713513 24695 raft_consensus.cc:740] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:08:57.714543 24695 consensus_queue.cc:260] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.715968 24695 raft_consensus.cc:3060] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.720889 24546 raft_consensus.cc:2468] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 2.
I20260709 13:08:57.722263 24586 leader_election.cc:304] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:08:57.723199 24810 raft_consensus.cc:2804] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:57.723788 24810 raft_consensus.cc:697] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:08:57.724100 24695 raft_consensus.cc:2468] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 2.
I20260709 13:08:57.724786 24810 consensus_queue.cc:237] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.731644 24445 catalog_manager.cc:5697] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 1 to 2, leader changed from bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67) to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:57.741464 24810 consensus_queue.cc:1048] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.754458 24776 consensus_queue.cc:1048] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.763023 24776 raft_consensus.cc:993] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487: : Instructing follower 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:08:57.763549 24778 raft_consensus.cc:1081] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Signalling peer 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:08:57.765324 24546 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56"
dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:48014
I20260709 13:08:57.765857 24546 raft_consensus.cc:493] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:57.766279 24546 raft_consensus.cc:3060] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.772138 24546 raft_consensus.cc:515] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.773896 24546 leader_election.cc:290] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Requested vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:57.774966 24695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:08:57.775018 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:08:57.775859 24695 raft_consensus.cc:3060] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.775943 24621 raft_consensus.cc:3055] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:57.776438 24621 raft_consensus.cc:740] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:08:57.777348 24621 consensus_queue.cc:260] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.778800 24621 raft_consensus.cc:3060] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.784271 24695 raft_consensus.cc:2468] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:08:57.785678 24512 leader_election.cc:304] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:08:57.786477 24767 raft_consensus.cc:2804] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:57.786522 24621 raft_consensus.cc:2468] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:08:57.787086 24767 raft_consensus.cc:697] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Becoming Leader. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:08:57.788048 24767 consensus_queue.cc:237] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.794829 24443 catalog_manager.cc:5697] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 reported cstate change: term changed from 1 to 2, leader changed from 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66) to 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "3fc09a7ccf854d5da51529f58bd82889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:57.906894 24771 raft_consensus.cc:993] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889: : Instructing follower bd09d620c95a44bcb4ac800897adc6f8 to start an election
I20260709 13:08:57.907373 24767 raft_consensus.cc:1081] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 1 LEADER]: Signalling peer bd09d620c95a44bcb4ac800897adc6f8 to start an election
I20260709 13:08:57.909039 24695 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c896072f0f34ec2b5d9a2c0ee528cde"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
 from {username='slave'} at 127.0.0.1:45608
I20260709 13:08:57.909633 24695 raft_consensus.cc:493] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:57.909936 24695 raft_consensus.cc:3060] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.917593 24695 raft_consensus.cc:515] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.919816 24695 leader_election.cc:290] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 2 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:08:57.920452 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c896072f0f34ec2b5d9a2c0ee528cde" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:08:57.921303 24546 raft_consensus.cc:3055] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:57.921324 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c896072f0f34ec2b5d9a2c0ee528cde" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:08:57.921659 24546 raft_consensus.cc:740] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:08:57.922111 24621 raft_consensus.cc:3060] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.922586 24546 consensus_queue.cc:260] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.923749 24546 raft_consensus.cc:3060] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.928610 24621 raft_consensus.cc:2468] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 2.
I20260709 13:08:57.929454 24546 raft_consensus.cc:2468] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 2.
I20260709 13:08:57.930223 24662 leader_election.cc:304] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:08:57.931128 24788 raft_consensus.cc:2804] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:57.931505 24788 raft_consensus.cc:697] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 2 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:08:57.932542 24788 consensus_queue.cc:237] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.939642 24765 raft_consensus.cc:993] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889: : Instructing follower 76f7ff986dda425baf2cae8ca9b96487 to start an election
I20260709 13:08:57.940191 24767 raft_consensus.cc:1081] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 LEADER]: Signalling peer 76f7ff986dda425baf2cae8ca9b96487 to start an election
I20260709 13:08:57.940719 24443 catalog_manager.cc:5697] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 1 to 2, leader changed from 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65) to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:57.941813 24621 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "281735360a8f4634b84dc1e9262ea6b2"
dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
 from {username='slave'} at 127.0.0.1:38786
I20260709 13:08:57.942543 24621 raft_consensus.cc:493] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:57.942858 24621 raft_consensus.cc:3060] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.950430 24621 raft_consensus.cc:515] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.952450 24621 leader_election.cc:290] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 2 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:57.953341 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "281735360a8f4634b84dc1e9262ea6b2" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:08:57.953558 24695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "281735360a8f4634b84dc1e9262ea6b2" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:08:57.954177 24546 raft_consensus.cc:3055] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:57.954214 24695 raft_consensus.cc:3060] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.954619 24546 raft_consensus.cc:740] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:08:57.955543 24546 consensus_queue.cc:260] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.956657 24546 raft_consensus.cc:3060] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.960687 24695 raft_consensus.cc:2468] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 2.
I20260709 13:08:57.962234 24587 leader_election.cc:304] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:08:57.962949 24546 raft_consensus.cc:2468] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 2.
I20260709 13:08:57.963078 24778 raft_consensus.cc:2804] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:57.963744 24778 raft_consensus.cc:697] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:08:57.964708 24778 consensus_queue.cc:237] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:57.971625 24443 catalog_manager.cc:5697] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 1 to 2, leader changed from 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65) to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:58.002276 24778 consensus_queue.cc:1048] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:58.019332 24778 consensus_queue.cc:1048] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:58.022073 24790 raft_consensus.cc:993] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8: : Instructing follower 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:08:58.022598 24790 raft_consensus.cc:1081] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Signalling peer 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:08:58.024789 24547 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "efc2d1bcdba24abca06cff7b82e9e10c"
dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:48012
I20260709 13:08:58.025403 24547 raft_consensus.cc:493] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:58.025825 24547 raft_consensus.cc:3060] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:58.033897 24547 raft_consensus.cc:515] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:58.036537 24547 leader_election.cc:290] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Requested vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:08:58.036831 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2d1bcdba24abca06cff7b82e9e10c" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:08:58.037196 24695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2d1bcdba24abca06cff7b82e9e10c" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:08:58.037922 24621 raft_consensus.cc:3060] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:58.038328 24695 raft_consensus.cc:3055] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:58.038671 24695 raft_consensus.cc:740] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:08:58.039559 24695 consensus_queue.cc:260] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:58.040922 24695 raft_consensus.cc:3060] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:58.046130 24621 raft_consensus.cc:2468] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:08:58.047679 24513 leader_election.cc:304] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:08:58.048462 24767 raft_consensus.cc:2804] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:58.048725 24695 raft_consensus.cc:2468] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:08:58.049006 24767 raft_consensus.cc:697] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Becoming Leader. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:08:58.049976 24767 consensus_queue.cc:237] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:08:58.056893 24445 catalog_manager.cc:5697] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 reported cstate change: term changed from 1 to 2, leader changed from bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67) to 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "3fc09a7ccf854d5da51529f58bd82889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:58.174666 24621 raft_consensus.cc:1275] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.176229 24767 consensus_queue.cc:1048] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:58.180032 24695 raft_consensus.cc:1275] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Refusing update from remote peer 76f7ff986dda425baf2cae8ca9b96487: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.181756 24778 consensus_queue.cc:1048] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:58.185881 24694 raft_consensus.cc:1275] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.187613 24765 consensus_queue.cc:1048] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:58.195533 24547 raft_consensus.cc:1275] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Refusing update from remote peer 76f7ff986dda425baf2cae8ca9b96487: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.197883 24742 consensus_queue.cc:1048] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:58.209168 24694 raft_consensus.cc:1275] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.211045 24765 consensus_queue.cc:1048] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:58.219730 24621 raft_consensus.cc:1275] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.221736 24767 consensus_queue.cc:1048] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:58.424784 24694 raft_consensus.cc:1275] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Refusing update from remote peer 76f7ff986dda425baf2cae8ca9b96487: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.426383 24742 consensus_queue.cc:1048] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:58.430378 24547 raft_consensus.cc:1275] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Refusing update from remote peer bd09d620c95a44bcb4ac800897adc6f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.431890 24788 consensus_queue.cc:1048] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:58.435582 24546 raft_consensus.cc:1275] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Refusing update from remote peer 76f7ff986dda425baf2cae8ca9b96487: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.439798 24810 consensus_queue.cc:1048] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:58.452124 24621 raft_consensus.cc:1275] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Refusing update from remote peer bd09d620c95a44bcb4ac800897adc6f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.453697 24825 consensus_queue.cc:1048] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:58.458458 24694 raft_consensus.cc:1275] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.460803 24767 consensus_queue.cc:1048] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:58.475036 24621 raft_consensus.cc:1275] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.476776 24771 consensus_queue.cc:1048] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:00.700543 24445 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38576:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:00.703128 24445 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:00.761799 24535 tablet_service.cc:1511] Processing CreateTablet for tablet 93156e55d4034667965228d19a646e50 (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:00.763530 24535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93156e55d4034667965228d19a646e50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.764256 24684 tablet_service.cc:1511] Processing CreateTablet for tablet 93156e55d4034667965228d19a646e50 (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:00.765707 24684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93156e55d4034667965228d19a646e50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.770164 24683 tablet_service.cc:1511] Processing CreateTablet for tablet ee5f0962f65146f3b7ee29957ffda06b (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:00.771251 24605 tablet_service.cc:1511] Processing CreateTablet for tablet 93156e55d4034667965228d19a646e50 (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:00.771974 24683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee5f0962f65146f3b7ee29957ffda06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.772859 24605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93156e55d4034667965228d19a646e50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.775025 24530 tablet_service.cc:1511] Processing CreateTablet for tablet 6bf66736913843acae755cc3c5a97638 (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:00.776625 24530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf66736913843acae755cc3c5a97638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.776252 24533 tablet_service.cc:1511] Processing CreateTablet for tablet 86d348aca96c4a3296d4d0b6b028c1fd (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:00.777710 24533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86d348aca96c4a3296d4d0b6b028c1fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.779065 24608 tablet_service.cc:1511] Processing CreateTablet for tablet ee5f0962f65146f3b7ee29957ffda06b (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:00.780102 24604 tablet_service.cc:1511] Processing CreateTablet for tablet d3aabffc54154f92895ab62d0e23456f (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:00.780757 24608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee5f0962f65146f3b7ee29957ffda06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.781100 24680 tablet_service.cc:1511] Processing CreateTablet for tablet 6bf66736913843acae755cc3c5a97638 (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:00.782490 24604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3aabffc54154f92895ab62d0e23456f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.782549 24680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf66736913843acae755cc3c5a97638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.783212 24679 tablet_service.cc:1511] Processing CreateTablet for tablet d3aabffc54154f92895ab62d0e23456f (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:00.784709 24679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3aabffc54154f92895ab62d0e23456f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.787253 24611 tablet_service.cc:1511] Processing CreateTablet for tablet 86d348aca96c4a3296d4d0b6b028c1fd (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:00.787940 24681 tablet_service.cc:1511] Processing CreateTablet for tablet 3aefcd1446da4fa0b24d4d5402d5c930 (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:00.788833 24611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86d348aca96c4a3296d4d0b6b028c1fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.789340 24681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aefcd1446da4fa0b24d4d5402d5c930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.791217 24678 tablet_service.cc:1511] Processing CreateTablet for tablet 86d348aca96c4a3296d4d0b6b028c1fd (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:00.793084 24536 tablet_service.cc:1511] Processing CreateTablet for tablet d3aabffc54154f92895ab62d0e23456f (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:00.794539 24536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3aabffc54154f92895ab62d0e23456f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.794553 24532 tablet_service.cc:1511] Processing CreateTablet for tablet 3aefcd1446da4fa0b24d4d5402d5c930 (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:00.796113 24532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aefcd1446da4fa0b24d4d5402d5c930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.798256 24609 tablet_service.cc:1511] Processing CreateTablet for tablet 3aefcd1446da4fa0b24d4d5402d5c930 (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:00.799782 24609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aefcd1446da4fa0b24d4d5402d5c930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.801693 24610 tablet_service.cc:1511] Processing CreateTablet for tablet ec03d81ef4cd465ea21e967d288c7faf (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:00.803231 24610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec03d81ef4cd465ea21e967d288c7faf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.805619 24529 tablet_service.cc:1511] Processing CreateTablet for tablet ec03d81ef4cd465ea21e967d288c7faf (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:00.807104 24529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec03d81ef4cd465ea21e967d288c7faf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.775086 24537 tablet_service.cc:1511] Processing CreateTablet for tablet ee5f0962f65146f3b7ee29957ffda06b (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:00.810426 24537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee5f0962f65146f3b7ee29957ffda06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.779162 24607 tablet_service.cc:1511] Processing CreateTablet for tablet 6bf66736913843acae755cc3c5a97638 (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:00.817687 24678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86d348aca96c4a3296d4d0b6b028c1fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.817977 24682 tablet_service.cc:1511] Processing CreateTablet for tablet ec03d81ef4cd465ea21e967d288c7faf (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:00.822235 24682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec03d81ef4cd465ea21e967d288c7faf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.826264 24607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf66736913843acae755cc3c5a97638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.831426 24534 tablet_service.cc:1511] Processing CreateTablet for tablet 3ffb6ee5eb0c4379ac6d560539f4ed41 (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:00.832998 24534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ffb6ee5eb0c4379ac6d560539f4ed41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.834533 24606 tablet_service.cc:1511] Processing CreateTablet for tablet 3ffb6ee5eb0c4379ac6d560539f4ed41 (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:00.836140 24606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ffb6ee5eb0c4379ac6d560539f4ed41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.839277 24686 tablet_service.cc:1511] Processing CreateTablet for tablet 3ffb6ee5eb0c4379ac6d560539f4ed41 (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:00.840893 24686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ffb6ee5eb0c4379ac6d560539f4ed41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.845644 24531 tablet_service.cc:1511] Processing CreateTablet for tablet 6e2f983a77dd426a80a8474fd87779db (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:00.847235 24531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e2f983a77dd426a80a8474fd87779db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.847223 24612 tablet_service.cc:1511] Processing CreateTablet for tablet 6e2f983a77dd426a80a8474fd87779db (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:00.848737 24612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e2f983a77dd426a80a8474fd87779db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.851087 24685 tablet_service.cc:1511] Processing CreateTablet for tablet 6e2f983a77dd426a80a8474fd87779db (DEFAULT_TABLE table=soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
W20260709 13:09:00.851562 24493 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:00.852445 24685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e2f983a77dd426a80a8474fd87779db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.857124 24843 tablet_bootstrap.cc:492] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:09:00.864506 24843 tablet_bootstrap.cc:654] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.939373 24842 tablet_bootstrap.cc:492] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:09:00.984421 24842 tablet_bootstrap.cc:654] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.991391 24843 tablet_bootstrap.cc:492] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:09:00.992046 24843 ts_tablet_manager.cc:1403] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.135s	user 0.010s	sys 0.004s
I20260709 13:09:00.992401 24844 tablet_bootstrap.cc:492] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:09:00.996630 24842 tablet_bootstrap.cc:492] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:09:00.996476 24843 raft_consensus.cc:359] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:00.997248 24842 ts_tablet_manager.cc:1403] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.058s	user 0.010s	sys 0.008s
I20260709 13:09:00.997447 24843 raft_consensus.cc:385] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.997856 24843 raft_consensus.cc:740] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.998782 24843 consensus_queue.cc:260] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.000394 24842 raft_consensus.cc:359] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.001020 24844 tablet_bootstrap.cc:654] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.001227 24842 raft_consensus.cc:385] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.001430 24843 ts_tablet_manager.cc:1434] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:09:01.001637 24842 raft_consensus.cc:740] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.002885 24843 tablet_bootstrap.cc:492] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:09:01.002560 24842 consensus_queue.cc:260] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.004985 24842 ts_tablet_manager.cc:1434] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:09:01.006513 24842 tablet_bootstrap.cc:492] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:09:01.012686 24843 tablet_bootstrap.cc:654] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.013620 24842 tablet_bootstrap.cc:654] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.014407 24844 tablet_bootstrap.cc:492] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:09:01.015075 24844 ts_tablet_manager.cc:1403] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.010s
I20260709 13:09:01.019768 24844 raft_consensus.cc:359] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.021067 24844 raft_consensus.cc:385] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.021559 24844 raft_consensus.cc:740] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.022809 24844 consensus_queue.cc:260] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.041947 24844 ts_tablet_manager.cc:1434] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:09:01.043488 24844 tablet_bootstrap.cc:492] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:09:01.047062 24843 tablet_bootstrap.cc:492] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:09:01.047821 24843 ts_tablet_manager.cc:1403] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.045s	user 0.030s	sys 0.010s
I20260709 13:09:01.048408 24842 tablet_bootstrap.cc:492] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:09:01.049104 24842 ts_tablet_manager.cc:1403] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.025s
I20260709 13:09:01.051339 24843 raft_consensus.cc:359] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.052284 24843 raft_consensus.cc:385] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.052690 24843 raft_consensus.cc:740] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.052394 24842 raft_consensus.cc:359] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.053388 24842 raft_consensus.cc:385] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.053907 24842 raft_consensus.cc:740] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.053683 24843 consensus_queue.cc:260] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.054847 24842 consensus_queue.cc:260] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.056861 24843 ts_tablet_manager.cc:1434] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:09:01.061506 24843 tablet_bootstrap.cc:492] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:09:01.062430 24842 ts_tablet_manager.cc:1434] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:09:01.063585 24842 tablet_bootstrap.cc:492] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:09:01.059280 24844 tablet_bootstrap.cc:654] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.068477 24843 tablet_bootstrap.cc:654] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.070504 24842 tablet_bootstrap.cc:654] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.085942 24842 tablet_bootstrap.cc:492] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:09:01.086597 24842 ts_tablet_manager.cc:1403] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.012s
I20260709 13:09:01.088784 24843 tablet_bootstrap.cc:492] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:09:01.089416 24843 ts_tablet_manager.cc:1403] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.011s
I20260709 13:09:01.090181 24842 raft_consensus.cc:359] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.091076 24842 raft_consensus.cc:385] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.091383 24842 raft_consensus.cc:740] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.092454 24843 raft_consensus.cc:359] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.093254 24843 raft_consensus.cc:385] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.093386 24844 tablet_bootstrap.cc:492] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:09:01.093568 24843 raft_consensus.cc:740] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.094010 24844 ts_tablet_manager.cc:1403] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.012s
I20260709 13:09:01.094501 24843 consensus_queue.cc:260] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.092229 24842 consensus_queue.cc:260] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.097174 24843 ts_tablet_manager.cc:1434] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:09:01.097481 24842 ts_tablet_manager.cc:1434] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:09:01.098279 24843 tablet_bootstrap.cc:492] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:09:01.097857 24844 raft_consensus.cc:359] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.098558 24842 tablet_bootstrap.cc:492] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:09:01.098675 24844 raft_consensus.cc:385] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.099180 24844 raft_consensus.cc:740] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.100123 24844 consensus_queue.cc:260] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.108227 24843 tablet_bootstrap.cc:654] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.108539 24844 ts_tablet_manager.cc:1434] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:09:01.109387 24842 tablet_bootstrap.cc:654] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.120972 24844 tablet_bootstrap.cc:492] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:09:01.132539 24843 tablet_bootstrap.cc:492] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:09:01.133399 24843 ts_tablet_manager.cc:1403] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.035s	user 0.027s	sys 0.008s
I20260709 13:09:01.136353 24844 tablet_bootstrap.cc:654] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.136976 24843 raft_consensus.cc:359] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.137887 24843 raft_consensus.cc:385] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.138216 24843 raft_consensus.cc:740] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.139078 24843 consensus_queue.cc:260] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.141769 24843 ts_tablet_manager.cc:1434] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:09:01.142904 24843 tablet_bootstrap.cc:492] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:09:01.145678 24842 tablet_bootstrap.cc:492] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:09:01.146355 24842 ts_tablet_manager.cc:1403] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.048s	user 0.031s	sys 0.015s
I20260709 13:09:01.149673 24842 raft_consensus.cc:359] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.150475 24842 raft_consensus.cc:385] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.150779 24842 raft_consensus.cc:740] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.151362 24843 tablet_bootstrap.cc:654] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.151541 24842 consensus_queue.cc:260] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.154265 24842 ts_tablet_manager.cc:1434] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:09:01.155395 24842 tablet_bootstrap.cc:492] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:09:01.157142 24844 tablet_bootstrap.cc:492] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:09:01.157956 24844 ts_tablet_manager.cc:1403] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.001s
I20260709 13:09:01.162367 24844 raft_consensus.cc:359] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.162971 24842 tablet_bootstrap.cc:654] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.163194 24844 raft_consensus.cc:385] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.163614 24844 raft_consensus.cc:740] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.164573 24843 tablet_bootstrap.cc:492] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:09:01.164515 24844 consensus_queue.cc:260] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.165182 24843 ts_tablet_manager.cc:1403] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.000s
I20260709 13:09:01.168498 24843 raft_consensus.cc:359] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.169204 24843 raft_consensus.cc:385] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.169481 24843 raft_consensus.cc:740] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.170358 24843 consensus_queue.cc:260] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.173043 24843 ts_tablet_manager.cc:1434] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:09:01.174185 24843 tablet_bootstrap.cc:492] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:09:01.175530 24842 tablet_bootstrap.cc:492] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:09:01.175882 24844 ts_tablet_manager.cc:1434] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 13:09:01.176136 24842 ts_tablet_manager.cc:1403] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.009s
I20260709 13:09:01.179711 24842 raft_consensus.cc:359] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.180471 24842 raft_consensus.cc:385] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.180751 24842 raft_consensus.cc:740] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.181600 24843 tablet_bootstrap.cc:654] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.181439 24842 consensus_queue.cc:260] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.184100 24842 ts_tablet_manager.cc:1434] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:01.186098 24844 tablet_bootstrap.cc:492] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:09:01.185173 24842 tablet_bootstrap.cc:492] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:09:01.195333 24843 tablet_bootstrap.cc:492] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:09:01.196117 24843 ts_tablet_manager.cc:1403] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.011s
I20260709 13:09:01.198349 24842 tablet_bootstrap.cc:654] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.199901 24843 raft_consensus.cc:359] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.200659 24843 raft_consensus.cc:385] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.200970 24843 raft_consensus.cc:740] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.201747 24843 consensus_queue.cc:260] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.204246 24843 ts_tablet_manager.cc:1434] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:09:01.204869 24844 tablet_bootstrap.cc:654] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.205281 24843 tablet_bootstrap.cc:492] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:09:01.212595 24843 tablet_bootstrap.cc:654] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.227257 24825 raft_consensus.cc:493] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.228166 24825 raft_consensus.cc:515] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.242301 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3aabffc54154f92895ab62d0e23456f" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:09:01.243649 24546 raft_consensus.cc:2468] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 0.
I20260709 13:09:01.246029 24862 raft_consensus.cc:493] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.246902 24862 raft_consensus.cc:515] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.247087 24660 leader_election.cc:304] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:01.250808 24825 leader_election.cc:290] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:01.251638 24620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec03d81ef4cd465ea21e967d288c7faf" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:09:01.253129 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3aabffc54154f92895ab62d0e23456f" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:09:01.255622 24844 tablet_bootstrap.cc:492] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:09:01.253201 24620 raft_consensus.cc:2468] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 0.
I20260709 13:09:01.256680 24844 ts_tablet_manager.cc:1403] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.071s	user 0.016s	sys 0.007s
I20260709 13:09:01.255694 24621 raft_consensus.cc:2468] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 0.
I20260709 13:09:01.254455 24825 raft_consensus.cc:2804] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.258093 24825 raft_consensus.cc:493] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.258585 24825 raft_consensus.cc:3060] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.258729 24843 tablet_bootstrap.cc:492] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:09:01.260092 24662 leader_election.cc:304] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:01.261294 24843 ts_tablet_manager.cc:1403] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.056s	user 0.032s	sys 0.019s
I20260709 13:09:01.263785 24862 leader_election.cc:290] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:01.267032 24842 tablet_bootstrap.cc:492] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:09:01.267817 24842 ts_tablet_manager.cc:1403] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.083s	user 0.015s	sys 0.050s
I20260709 13:09:01.268007 24862 raft_consensus.cc:2804] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.269320 24862 raft_consensus.cc:493] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.270224 24862 raft_consensus.cc:3060] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.273919 24843 raft_consensus.cc:359] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.274849 24843 raft_consensus.cc:385] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.274546 24842 raft_consensus.cc:359] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.275177 24843 raft_consensus.cc:740] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.275344 24842 raft_consensus.cc:385] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.276108 24843 consensus_queue.cc:260] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.275693 24842 raft_consensus.cc:740] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.278148 24844 raft_consensus.cc:359] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.278465 24842 consensus_queue.cc:260] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.279024 24844 raft_consensus.cc:385] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.279469 24844 raft_consensus.cc:740] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.280331 24843 ts_tablet_manager.cc:1434] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.002s
I20260709 13:09:01.280277 24844 consensus_queue.cc:260] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.281937 24843 tablet_bootstrap.cc:492] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:09:01.282912 24864 raft_consensus.cc:493] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.283634 24842 ts_tablet_manager.cc:1434] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:09:01.283532 24864 raft_consensus.cc:515] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.284799 24842 tablet_bootstrap.cc:492] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:09:01.290890 24843 tablet_bootstrap.cc:654] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.293542 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aefcd1446da4fa0b24d4d5402d5c930" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:09:01.295210 24621 raft_consensus.cc:2468] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 0.
I20260709 13:09:01.296882 24662 leader_election.cc:304] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:01.296885 24844 ts_tablet_manager.cc:1434] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.004s
I20260709 13:09:01.302472 24844 tablet_bootstrap.cc:492] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:09:01.302613 24842 tablet_bootstrap.cc:654] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.297868 24863 raft_consensus.cc:2804] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.303062 24863 raft_consensus.cc:493] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.303524 24863 raft_consensus.cc:3060] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.311949 24864 leader_election.cc:290] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:01.315836 24767 raft_consensus.cc:493] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.316751 24767 raft_consensus.cc:515] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.322211 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aefcd1446da4fa0b24d4d5402d5c930" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:09:01.323328 24621 raft_consensus.cc:2468] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 0.
I20260709 13:09:01.325147 24837 raft_consensus.cc:493] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.325373 24862 raft_consensus.cc:515] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.325907 24837 raft_consensus.cc:515] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.327018 24513 leader_election.cc:304] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:09:01.328135 24837 leader_election.cc:290] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.329596 24862 leader_election.cc:290] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:01.329473 24620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec03d81ef4cd465ea21e967d288c7faf" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:01.330293 24620 raft_consensus.cc:3060] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.332265 24694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2f983a77dd426a80a8474fd87779db" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:09:01.333400 24694 raft_consensus.cc:2468] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:09:01.334141 24842 tablet_bootstrap.cc:492] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:09:01.334755 24842 ts_tablet_manager.cc:1403] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.014s
I20260709 13:09:01.335754 24587 leader_election.cc:304] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:01.337882 24837 raft_consensus.cc:2804] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.338507 24837 raft_consensus.cc:493] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.338380 24842 raft_consensus.cc:359] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.339314 24842 raft_consensus.cc:385] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.339267 24837 raft_consensus.cc:3060] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.339941 24842 raft_consensus.cc:740] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.340792 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2f983a77dd426a80a8474fd87779db" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
W20260709 13:09:01.343415 24586 leader_election.cc:343] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.345440 24767 leader_election.cc:290] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
W20260709 13:09:01.293609 24660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.109.65:39367, user_credentials={real_user=slave}} blocked reactor thread for 46712.2us
I20260709 13:09:01.348716 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec03d81ef4cd465ea21e967d288c7faf" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:09:01.346146 24767 raft_consensus.cc:2804] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.351305 24767 raft_consensus.cc:493] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.351591 24694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aefcd1446da4fa0b24d4d5402d5c930" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:09:01.351692 24767 raft_consensus.cc:3060] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.355415 24863 raft_consensus.cc:515] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.358655 24842 consensus_queue.cc:260] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.356521 24825 raft_consensus.cc:515] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.370100 24825 leader_election.cc:290] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:01.360244 24620 raft_consensus.cc:2468] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 1.
I20260709 13:09:01.360951 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aefcd1446da4fa0b24d4d5402d5c930" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
W20260709 13:09:01.363037 24660 leader_election.cc:343] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.372148 24622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aefcd1446da4fa0b24d4d5402d5c930" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:01.372903 24619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3aabffc54154f92895ab62d0e23456f" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:01.372893 24622 raft_consensus.cc:3060] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.373740 24619 raft_consensus.cc:3060] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.373797 24545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aefcd1446da4fa0b24d4d5402d5c930" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:01.364238 24844 tablet_bootstrap.cc:654] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.330297 24862 raft_consensus.cc:493] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.367210 24547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec03d81ef4cd465ea21e967d288c7faf" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:01.375927 24862 raft_consensus.cc:515] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.379323 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93156e55d4034667965228d19a646e50" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:09:01.380048 24621 raft_consensus.cc:2468] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 0.
I20260709 13:09:01.380502 24547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93156e55d4034667965228d19a646e50" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:09:01.381282 24547 raft_consensus.cc:2468] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 0.
I20260709 13:09:01.367411 24843 tablet_bootstrap.cc:492] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:09:01.381623 24662 leader_election.cc:304] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:01.375170 24544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3aabffc54154f92895ab62d0e23456f" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:01.382879 24544 raft_consensus.cc:3060] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:01.382972 24660 leader_election.cc:343] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.383665 24862 leader_election.cc:290] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:01.384214 24862 raft_consensus.cc:2804] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.384560 24862 raft_consensus.cc:493] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.384904 24862 raft_consensus.cc:3060] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.382371 24843 ts_tablet_manager.cc:1403] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.101s	user 0.044s	sys 0.038s
I20260709 13:09:01.386420 24863 leader_election.cc:290] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:01.383234 24868 raft_consensus.cc:493] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.387455 24868 raft_consensus.cc:515] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.389915 24662 leader_election.cc:304] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 3fc09a7ccf854d5da51529f58bd82889
I20260709 13:09:01.391609 24547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93156e55d4034667965228d19a646e50" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:09:01.392444 24547 raft_consensus.cc:2468] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:09:01.397320 24544 raft_consensus.cc:2468] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 1.
I20260709 13:09:01.389355 24843 raft_consensus.cc:359] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.398269 24843 raft_consensus.cc:385] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.398615 24843 raft_consensus.cc:740] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.399878 24694 raft_consensus.cc:2393] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fc09a7ccf854d5da51529f58bd82889 in current term 1: Already voted for candidate bd09d620c95a44bcb4ac800897adc6f8 in this term.
I20260709 13:09:01.400445 24825 raft_consensus.cc:2804] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.401314 24825 raft_consensus.cc:697] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:01.402294 24825 consensus_queue.cc:237] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.399482 24843 consensus_queue.cc:260] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.406016 24622 raft_consensus.cc:2468] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 1.
I20260709 13:09:01.406419 24660 leader_election.cc:304] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:01.407449 24619 raft_consensus.cc:2468] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 1.
I20260709 13:09:01.408736 24837 raft_consensus.cc:515] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.407775 24843 ts_tablet_manager.cc:1434] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:09:01.407428 24662 leader_election.cc:304] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 3fc09a7ccf854d5da51529f58bd82889
I20260709 13:09:01.409191 24863 raft_consensus.cc:2804] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.410274 24864 raft_consensus.cc:2804] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.415181 24843 tablet_bootstrap.cc:492] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8: Bootstrap starting.
I20260709 13:09:01.415791 24842 ts_tablet_manager.cc:1434] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.006s
I20260709 13:09:01.416085 24844 tablet_bootstrap.cc:492] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:09:01.415872 24586 leader_election.cc:304] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:09:01.417152 24842 tablet_bootstrap.cc:492] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
I20260709 13:09:01.424072 24844 ts_tablet_manager.cc:1403] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.122s	user 0.016s	sys 0.013s
I20260709 13:09:01.425314 24837 leader_election.cc:290] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.414081 24767 raft_consensus.cc:515] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.426940 24546 raft_consensus.cc:2393] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd09d620c95a44bcb4ac800897adc6f8 in current term 1: Already voted for candidate 3fc09a7ccf854d5da51529f58bd82889 in this term.
I20260709 13:09:01.427507 24864 raft_consensus.cc:697] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:01.428285 24837 raft_consensus.cc:2804] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.428885 24544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2f983a77dd426a80a8474fd87779db" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
W20260709 13:09:01.431196 24586 leader_election.cc:343] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.431718 24619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aefcd1446da4fa0b24d4d5402d5c930" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:01.432689 24619 raft_consensus.cc:2393] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3fc09a7ccf854d5da51529f58bd82889 in current term 1: Already voted for candidate bd09d620c95a44bcb4ac800897adc6f8 in this term.
I20260709 13:09:01.432407 24864 consensus_queue.cc:237] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.428893 24837 raft_consensus.cc:493] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.436923 24695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aefcd1446da4fa0b24d4d5402d5c930" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:01.437212 24767 leader_election.cc:290] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 election: Requested vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.440304 24842 tablet_bootstrap.cc:654] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.443197 24843 tablet_bootstrap.cc:654] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.443385 24695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93156e55d4034667965228d19a646e50" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:09:01.437933 24694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2f983a77dd426a80a8474fd87779db" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:01.440750 24512 leader_election.cc:304] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889; no voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:09:01.437271 24837 raft_consensus.cc:3060] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.440613 24844 raft_consensus.cc:359] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.444530 24868 leader_election.cc:290] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.446242 24767 raft_consensus.cc:2749] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:01.446543 24844 raft_consensus.cc:385] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.449352 24844 raft_consensus.cc:740] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.423053 24862 raft_consensus.cc:515] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.444926 24694 raft_consensus.cc:3060] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.450306 24844 consensus_queue.cc:260] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.450873 24695 raft_consensus.cc:2393] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76f7ff986dda425baf2cae8ca9b96487 in current term 1: Already voted for candidate bd09d620c95a44bcb4ac800897adc6f8 in this term.
I20260709 13:09:01.452896 24844 ts_tablet_manager.cc:1434] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:09:01.455307 24844 tablet_bootstrap.cc:492] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:09:01.462457 24694 raft_consensus.cc:2468] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 1.
I20260709 13:09:01.463754 24844 tablet_bootstrap.cc:654] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.463471 24619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93156e55d4034667965228d19a646e50" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:01.464556 24587 leader_election.cc:304] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 3fc09a7ccf854d5da51529f58bd82889
I20260709 13:09:01.464883 24544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93156e55d4034667965228d19a646e50" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:01.465649 24544 raft_consensus.cc:3060] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.470305 24868 raft_consensus.cc:2804] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.471030 24868 raft_consensus.cc:697] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:01.472081 24868 consensus_queue.cc:237] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.484824 24869 raft_consensus.cc:493] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.485416 24869 raft_consensus.cc:515] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.486210 24862 leader_election.cc:290] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:01.489686 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5f0962f65146f3b7ee29957ffda06b" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:09:01.490893 24546 raft_consensus.cc:2468] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:09:01.495278 24586 leader_election.cc:304] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:09:01.495890 24842 tablet_bootstrap.cc:492] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:09:01.496799 24842 ts_tablet_manager.cc:1403] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.080s	user 0.008s	sys 0.007s
I20260709 13:09:01.498502 24870 raft_consensus.cc:2804] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.499191 24870 raft_consensus.cc:493] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.499646 24870 raft_consensus.cc:3060] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.501361 24869 leader_election.cc:290] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.503003 24694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5f0962f65146f3b7ee29957ffda06b" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:09:01.503897 24694 raft_consensus.cc:2468] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:09:01.503351 24869 raft_consensus.cc:493] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.504724 24869 raft_consensus.cc:515] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.509137 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ffb6ee5eb0c4379ac6d560539f4ed41" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:09:01.509950 24546 raft_consensus.cc:2468] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:09:01.509281 24842 raft_consensus.cc:359] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.511519 24586 leader_election.cc:304] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:09:01.511520 24842 raft_consensus.cc:385] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.514554 24842 raft_consensus.cc:740] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.515250 24869 leader_election.cc:290] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.516585 24863 raft_consensus.cc:697] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:01.515453 24842 consensus_queue.cc:260] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.517063 24694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ffb6ee5eb0c4379ac6d560539f4ed41" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:09:01.516829 24869 raft_consensus.cc:2804] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.521960 24544 raft_consensus.cc:2468] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 1.
I20260709 13:09:01.524029 24842 ts_tablet_manager.cc:1434] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.001s
I20260709 13:09:01.525467 24842 tablet_bootstrap.cc:492] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487: Bootstrap starting.
W20260709 13:09:01.533655 24587 leader_election.cc:343] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.534407 24844 tablet_bootstrap.cc:492] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:09:01.535247 24844 ts_tablet_manager.cc:1403] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.080s	user 0.006s	sys 0.014s
I20260709 13:09:01.538373 24863 consensus_queue.cc:237] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.539778 24444 catalog_manager.cc:5697] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:01.541428 24870 raft_consensus.cc:515] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.522991 24869 raft_consensus.cc:493] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.547837 24869 raft_consensus.cc:3060] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.514587 24441 catalog_manager.cc:5697] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:01.548250 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5f0962f65146f3b7ee29957ffda06b" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:01.549052 24546 raft_consensus.cc:3060] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.553135 24870 leader_election.cc:290] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.556320 24660 leader_election.cc:304] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:01.557636 24825 raft_consensus.cc:2804] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.558241 24825 raft_consensus.cc:697] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:01.559052 24837 raft_consensus.cc:515] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.559338 24825 consensus_queue.cc:237] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.560858 24619 raft_consensus.cc:2393] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bd09d620c95a44bcb4ac800897adc6f8 in current term 1: Already voted for candidate 76f7ff986dda425baf2cae8ca9b96487 in this term.
I20260709 13:09:01.563089 24694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5f0962f65146f3b7ee29957ffda06b" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:01.563846 24694 raft_consensus.cc:3060] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.564622 24844 raft_consensus.cc:359] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.565927 24844 raft_consensus.cc:385] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.566931 24547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93156e55d4034667965228d19a646e50" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:01.569895 24844 raft_consensus.cc:740] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.580307 24844 consensus_queue.cc:260] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.582363 24837 leader_election.cc:290] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.584619 24695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93156e55d4034667965228d19a646e50" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:01.585095 24546 raft_consensus.cc:2468] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 1.
I20260709 13:09:01.586470 24547 raft_consensus.cc:2393] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 76f7ff986dda425baf2cae8ca9b96487 in current term 1: Already voted for candidate bd09d620c95a44bcb4ac800897adc6f8 in this term.
I20260709 13:09:01.587929 24874 raft_consensus.cc:493] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.588672 24586 leader_election.cc:304] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:09:01.590735 24842 tablet_bootstrap.cc:654] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.592706 24874 raft_consensus.cc:515] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.591465 24586 leader_election.cc:304] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487; no voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:09:01.597941 24870 raft_consensus.cc:2749] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:01.599598 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf66736913843acae755cc3c5a97638" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:09:01.593580 24837 raft_consensus.cc:2804] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.600063 24874 leader_election.cc:290] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
W20260709 13:09:01.601603 24660 leader_election.cc:343] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.599436 24622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf66736913843acae755cc3c5a97638" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:09:01.603273 24837 raft_consensus.cc:697] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
W20260709 13:09:01.603731 24662 leader_election.cc:343] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.603492 24869 raft_consensus.cc:515] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.604254 24662 leader_election.cc:304] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd09d620c95a44bcb4ac800897adc6f8; no voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:09:01.604360 24837 consensus_queue.cc:237] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.605187 24874 raft_consensus.cc:2749] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:01.606030 24843 tablet_bootstrap.cc:492] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8: No bootstrap required, opened a new log
I20260709 13:09:01.606562 24869 leader_election.cc:290] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.606705 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ffb6ee5eb0c4379ac6d560539f4ed41" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:01.607578 24546 raft_consensus.cc:3060] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.607822 24695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ffb6ee5eb0c4379ac6d560539f4ed41" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
W20260709 13:09:01.609457 24587 leader_election.cc:343] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.610174 24843 ts_tablet_manager.cc:1403] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8: Time spent bootstrapping tablet: real 0.195s	user 0.033s	sys 0.031s
I20260709 13:09:01.610954 24694 raft_consensus.cc:2468] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 1.
I20260709 13:09:01.601595 24844 ts_tablet_manager.cc:1434] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.006s
I20260709 13:09:01.620160 24844 tablet_bootstrap.cc:492] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:09:01.616196 24843 raft_consensus.cc:359] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.616878 24842 tablet_bootstrap.cc:492] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487: No bootstrap required, opened a new log
I20260709 13:09:01.622695 24842 ts_tablet_manager.cc:1403] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487: Time spent bootstrapping tablet: real 0.098s	user 0.011s	sys 0.011s
I20260709 13:09:01.627043 24842 raft_consensus.cc:359] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.628037 24842 raft_consensus.cc:385] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.628422 24842 raft_consensus.cc:740] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.629355 24842 consensus_queue.cc:260] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.632429 24842 ts_tablet_manager.cc:1434] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:09:01.633179 24843 raft_consensus.cc:385] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.633610 24843 raft_consensus.cc:740] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.634166 24546 raft_consensus.cc:2468] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 1.
I20260709 13:09:01.635807 24586 leader_election.cc:304] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:09:01.637212 24837 raft_consensus.cc:2804] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.641418 24837 raft_consensus.cc:697] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:01.642493 24837 consensus_queue.cc:237] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.646620 24844 tablet_bootstrap.cc:654] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.634627 24843 consensus_queue.cc:260] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.649747 24874 raft_consensus.cc:493] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.650310 24874 raft_consensus.cc:515] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.659396 24547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ffb6ee5eb0c4379ac6d560539f4ed41" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:09:01.660669 24547 raft_consensus.cc:2393] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd09d620c95a44bcb4ac800897adc6f8 in current term 1: Already voted for candidate 76f7ff986dda425baf2cae8ca9b96487 in this term.
I20260709 13:09:01.659644 24843 ts_tablet_manager.cc:1434] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.000s
I20260709 13:09:01.670246 24844 tablet_bootstrap.cc:492] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:09:01.670872 24844 ts_tablet_manager.cc:1403] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.051s	user 0.020s	sys 0.007s
I20260709 13:09:01.671663 24874 leader_election.cc:290] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:01.672888 24622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ffb6ee5eb0c4379ac6d560539f4ed41" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:09:01.673897 24844 raft_consensus.cc:359] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.674873 24662 leader_election.cc:304] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd09d620c95a44bcb4ac800897adc6f8; no voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:09:01.675273 24844 raft_consensus.cc:385] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.675725 24844 raft_consensus.cc:740] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.676555 24844 consensus_queue.cc:260] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.678299 24874 raft_consensus.cc:3060] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.678835 24844 ts_tablet_manager.cc:1434] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:09:01.680023 24844 tablet_bootstrap.cc:492] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:09:01.683465 24441 catalog_manager.cc:5697] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:01.687152 24844 tablet_bootstrap.cc:654] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.687343 24874 raft_consensus.cc:2749] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:01.685556 24441 catalog_manager.cc:5697] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:01.690630 24441 catalog_manager.cc:5697] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 1 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:01.700423 24844 tablet_bootstrap.cc:492] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:09:01.701064 24844 ts_tablet_manager.cc:1403] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.015s
I20260709 13:09:01.704074 24844 raft_consensus.cc:359] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.704775 24844 raft_consensus.cc:385] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.705045 24844 raft_consensus.cc:740] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.704950 24444 catalog_manager.cc:5697] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:01.705926 24844 consensus_queue.cc:260] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.707340 24444 catalog_manager.cc:5697] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:01.708249 24844 ts_tablet_manager.cc:1434] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:01.709679 24844 tablet_bootstrap.cc:492] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889: Bootstrap starting.
I20260709 13:09:01.715564 24844 tablet_bootstrap.cc:654] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.721182 24844 tablet_bootstrap.cc:492] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889: No bootstrap required, opened a new log
I20260709 13:09:01.721746 24844 ts_tablet_manager.cc:1403] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:09:01.722453 24767 raft_consensus.cc:493] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.723078 24767 raft_consensus.cc:515] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.724627 24844 raft_consensus.cc:359] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.725407 24844 raft_consensus.cc:385] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.725589 24767 leader_election.cc:290] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.725687 24844 raft_consensus.cc:740] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.727790 24622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2f983a77dd426a80a8474fd87779db" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f7ff986dda425baf2cae8ca9b96487" is_pre_election: true
I20260709 13:09:01.728206 24844 consensus_queue.cc:260] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.728909 24694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2f983a77dd426a80a8474fd87779db" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:09:01.729825 24694 raft_consensus.cc:2393] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fc09a7ccf854d5da51529f58bd82889 in current term 1: Already voted for candidate 76f7ff986dda425baf2cae8ca9b96487 in this term.
I20260709 13:09:01.730949 24844 ts_tablet_manager.cc:1434] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:09:01.731240 24513 leader_election.cc:304] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889; no voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:09:01.732613 24867 raft_consensus.cc:3060] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.740096 24867 raft_consensus.cc:2749] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:01.769006 24837 raft_consensus.cc:493] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.769690 24837 raft_consensus.cc:515] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.771649 24837 leader_election.cc:290] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.772542 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d348aca96c4a3296d4d0b6b028c1fd" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:09:01.772780 24694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d348aca96c4a3296d4d0b6b028c1fd" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:09:01.773321 24546 raft_consensus.cc:2468] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:09:01.773509 24694 raft_consensus.cc:2468] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:09:01.774433 24586 leader_election.cc:304] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:09:01.775262 24837 raft_consensus.cc:2804] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.775635 24837 raft_consensus.cc:493] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.775976 24837 raft_consensus.cc:3060] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.782510 24837 raft_consensus.cc:515] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.784433 24837 leader_election.cc:290] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.785156 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d348aca96c4a3296d4d0b6b028c1fd" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:01.785506 24694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d348aca96c4a3296d4d0b6b028c1fd" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:01.785897 24546 raft_consensus.cc:3060] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.786176 24694 raft_consensus.cc:3060] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.793702 24546 raft_consensus.cc:2468] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 1.
I20260709 13:09:01.793705 24694 raft_consensus.cc:2468] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 1.
I20260709 13:09:01.795311 24587 leader_election.cc:304] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:01.796092 24837 raft_consensus.cc:2804] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.796583 24837 raft_consensus.cc:697] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:01.797362 24837 consensus_queue.cc:237] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.805387 24441 catalog_manager.cc:5697] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:01.807667 24837 raft_consensus.cc:493] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.808307 24837 raft_consensus.cc:515] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.810261 24837 leader_election.cc:290] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.811254 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf66736913843acae755cc3c5a97638" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889" is_pre_election: true
I20260709 13:09:01.811496 24694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf66736913843acae755cc3c5a97638" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8" is_pre_election: true
I20260709 13:09:01.812093 24546 raft_consensus.cc:2468] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:09:01.812237 24694 raft_consensus.cc:2468] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 0.
I20260709 13:09:01.813596 24586 leader_election.cc:304] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:09:01.814584 24837 raft_consensus.cc:2804] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.814963 24837 raft_consensus.cc:493] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.815302 24837 raft_consensus.cc:3060] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.821619 24837 raft_consensus.cc:515] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.823704 24837 leader_election.cc:290] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:01.824256 24874 consensus_queue.cc:1048] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:01.824651 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf66736913843acae755cc3c5a97638" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:01.824908 24694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf66736913843acae755cc3c5a97638" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:01.825434 24546 raft_consensus.cc:3060] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.825560 24694 raft_consensus.cc:3060] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.829757 24874 consensus_queue.cc:1048] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:01.835137 24546 raft_consensus.cc:2468] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 1.
I20260709 13:09:01.836709 24586 leader_election.cc:304] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:09:01.837699 24837 raft_consensus.cc:2804] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.838238 24837 raft_consensus.cc:697] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:01.839166 24837 consensus_queue.cc:237] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:01.842437 24694 raft_consensus.cc:2468] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 1.
W20260709 13:09:01.858289 24493 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:01.870504 24546 raft_consensus.cc:3060] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.879287 24445 catalog_manager.cc:5697] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:01.880688 24863 consensus_queue.cc:1048] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:01.885740 24863 consensus_queue.cc:1048] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:01.919327 24837 consensus_queue.cc:1048] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:01.931107 24837 consensus_queue.cc:1048] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:02.013589 24863 consensus_queue.cc:1048] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:02.024724 24825 consensus_queue.cc:1048] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:02.040055 24874 consensus_queue.cc:1048] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:02.040699 24694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93156e55d4034667965228d19a646e50"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
mode: GRACEFUL
new_leader_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:45626
I20260709 13:09:02.041471 24694 raft_consensus.cc:606] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Received request to transfer leadership to TS 3fc09a7ccf854d5da51529f58bd82889
I20260709 13:09:02.048507 24622 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee5f0962f65146f3b7ee29957ffda06b"
dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
mode: GRACEFUL
new_leader_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:38818
I20260709 13:09:02.049444 24622 raft_consensus.cc:606] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Received request to transfer leadership to TS 3fc09a7ccf854d5da51529f58bd82889
I20260709 13:09:02.057719 24622 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86d348aca96c4a3296d4d0b6b028c1fd"
dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
mode: GRACEFUL
new_leader_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:38818
I20260709 13:09:02.058409 24622 raft_consensus.cc:606] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Received request to transfer leadership to TS 3fc09a7ccf854d5da51529f58bd82889
I20260709 13:09:02.065587 24874 consensus_queue.cc:1048] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:02.066577 24880 consensus_queue.cc:1048] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:02.072762 24619 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bf66736913843acae755cc3c5a97638"
dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
mode: GRACEFUL
new_leader_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:38818
I20260709 13:09:02.073350 24619 raft_consensus.cc:606] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Received request to transfer leadership to TS 3fc09a7ccf854d5da51529f58bd82889
I20260709 13:09:02.080229 24694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3aabffc54154f92895ab62d0e23456f"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
mode: GRACEFUL
new_leader_uuid: "76f7ff986dda425baf2cae8ca9b96487"
 from {username='slave'} at 127.0.0.1:45626
I20260709 13:09:02.080926 24694 raft_consensus.cc:606] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Received request to transfer leadership to TS 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:09:02.085517 24873 raft_consensus.cc:993] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8: : Instructing follower 76f7ff986dda425baf2cae8ca9b96487 to start an election
I20260709 13:09:02.086287 24874 raft_consensus.cc:1081] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Signalling peer 76f7ff986dda425baf2cae8ca9b96487 to start an election
I20260709 13:09:02.087813 24880 consensus_queue.cc:1048] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:02.088338 24622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d3aabffc54154f92895ab62d0e23456f"
dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
 from {username='slave'} at 127.0.0.1:38802
I20260709 13:09:02.088579 24694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3aefcd1446da4fa0b24d4d5402d5c930"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
mode: GRACEFUL
new_leader_uuid: "76f7ff986dda425baf2cae8ca9b96487"
 from {username='slave'} at 127.0.0.1:45626
I20260709 13:09:02.090864 24622 raft_consensus.cc:493] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:02.091188 24694 raft_consensus.cc:606] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Received request to transfer leadership to TS 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:09:02.091452 24622 raft_consensus.cc:3060] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.100132 24622 raft_consensus.cc:515] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.100876 24869 consensus_queue.cc:1048] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:02.102648 24622 leader_election.cc:290] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 2 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:02.103574 24546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3aabffc54154f92895ab62d0e23456f" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:02.104375 24546 raft_consensus.cc:3060] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.107002 24693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3aabffc54154f92895ab62d0e23456f" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:02.107744 24693 raft_consensus.cc:3055] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:02.108017 24693 raft_consensus.cc:740] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:02.108888 24693 consensus_queue.cc:260] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.109826 24693 raft_consensus.cc:3060] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.115223 24546 raft_consensus.cc:2468] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 2.
I20260709 13:09:02.116724 24586 leader_election.cc:304] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:09:02.103519 24694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec03d81ef4cd465ea21e967d288c7faf"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
mode: GRACEFUL
new_leader_uuid: "76f7ff986dda425baf2cae8ca9b96487"
 from {username='slave'} at 127.0.0.1:45626
I20260709 13:09:02.117537 24694 raft_consensus.cc:606] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Received request to transfer leadership to TS 76f7ff986dda425baf2cae8ca9b96487
I20260709 13:09:02.118155 24880 raft_consensus.cc:2804] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:02.121688 24880 raft_consensus.cc:697] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:02.122710 24880 consensus_queue.cc:237] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.136111 24444 catalog_manager.cc:5697] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 1 to 2, leader changed from bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67) to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:02.146284 24693 raft_consensus.cc:2468] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 2.
I20260709 13:09:02.151386 24870 consensus_queue.cc:1048] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:02.152479 24622 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e2f983a77dd426a80a8474fd87779db"
dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
mode: GRACEFUL
new_leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
 from {username='slave'} at 127.0.0.1:38818
I20260709 13:09:02.153352 24622 raft_consensus.cc:606] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Received request to transfer leadership to TS bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:09:02.165899 24880 raft_consensus.cc:993] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487: : Instructing follower 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:09:02.166498 24870 raft_consensus.cc:1081] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Signalling peer 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:09:02.168430 24547 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee5f0962f65146f3b7ee29957ffda06b"
dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:48014
I20260709 13:09:02.169528 24547 raft_consensus.cc:493] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:02.169965 24547 raft_consensus.cc:3060] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.177217 24547 raft_consensus.cc:515] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.179729 24547 leader_election.cc:290] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Requested vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:02.180143 24622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5f0962f65146f3b7ee29957ffda06b" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:02.180431 24693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5f0962f65146f3b7ee29957ffda06b" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:02.181042 24622 raft_consensus.cc:3055] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:02.181756 24622 raft_consensus.cc:740] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:02.181771 24693 raft_consensus.cc:3060] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.182713 24622 consensus_queue.cc:260] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.184083 24622 raft_consensus.cc:3060] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.189555 24693 raft_consensus.cc:2468] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:09:02.190876 24512 leader_election.cc:304] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:02.191197 24622 raft_consensus.cc:2468] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:09:02.191713 24867 raft_consensus.cc:2804] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:02.192322 24867 raft_consensus.cc:697] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Becoming Leader. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:09:02.193243 24867 consensus_queue.cc:237] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.201140 24445 catalog_manager.cc:5697] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 reported cstate change: term changed from 1 to 2, leader changed from 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66) to 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "3fc09a7ccf854d5da51529f58bd82889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:02.271679 24874 raft_consensus.cc:993] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8: : Instructing follower 76f7ff986dda425baf2cae8ca9b96487 to start an election
I20260709 13:09:02.273131 24825 raft_consensus.cc:1081] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Signalling peer 76f7ff986dda425baf2cae8ca9b96487 to start an election
I20260709 13:09:02.276165 24622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3aefcd1446da4fa0b24d4d5402d5c930"
dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
 from {username='slave'} at 127.0.0.1:38802
I20260709 13:09:02.277235 24622 raft_consensus.cc:493] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:02.277765 24622 raft_consensus.cc:3060] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.292452 24880 consensus_queue.cc:1048] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:02.295100 24870 consensus_queue.cc:1048] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:02.298834 24622 raft_consensus.cc:515] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.308494 24622 leader_election.cc:290] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 2 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:02.309604 24547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aefcd1446da4fa0b24d4d5402d5c930" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:02.310936 24547 raft_consensus.cc:3060] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.314898 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aefcd1446da4fa0b24d4d5402d5c930" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:02.315896 24696 raft_consensus.cc:3055] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:02.316301 24696 raft_consensus.cc:740] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:02.318243 24696 consensus_queue.cc:260] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.323179 24696 raft_consensus.cc:3060] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.325292 24869 consensus_queue.cc:1048] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:02.334604 24880 consensus_queue.cc:1048] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:02.351019 24696 raft_consensus.cc:2468] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 2.
I20260709 13:09:02.351293 24547 raft_consensus.cc:2468] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 2.
I20260709 13:09:02.353154 24587 leader_election.cc:304] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:02.354472 24837 raft_consensus.cc:2804] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:02.354992 24837 raft_consensus.cc:697] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:02.356189 24869 raft_consensus.cc:993] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487: : Instructing follower bd09d620c95a44bcb4ac800897adc6f8 to start an election
I20260709 13:09:02.356489 24837 consensus_queue.cc:237] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.364840 24837 raft_consensus.cc:993] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487: : Instructing follower 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:09:02.365428 24837 raft_consensus.cc:1081] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Signalling peer 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:09:02.357023 24880 raft_consensus.cc:1081] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Signalling peer bd09d620c95a44bcb4ac800897adc6f8 to start an election
I20260709 13:09:02.372795 24825 raft_consensus.cc:993] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8: : Instructing follower 76f7ff986dda425baf2cae8ca9b96487 to start an election
I20260709 13:09:02.373718 24825 raft_consensus.cc:1081] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Signalling peer 76f7ff986dda425baf2cae8ca9b96487 to start an election
I20260709 13:09:02.372817 24546 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86d348aca96c4a3296d4d0b6b028c1fd"
dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:48014
I20260709 13:09:02.377636 24880 raft_consensus.cc:993] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487: : Instructing follower 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:09:02.377766 24546 raft_consensus.cc:493] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:02.378351 24870 raft_consensus.cc:1081] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Signalling peer 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:09:02.378372 24546 raft_consensus.cc:3060] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.380264 24696 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e2f983a77dd426a80a8474fd87779db"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
 from {username='slave'} at 127.0.0.1:45616
I20260709 13:09:02.381327 24696 raft_consensus.cc:493] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:02.381866 24696 raft_consensus.cc:3060] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.387233 24544 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6bf66736913843acae755cc3c5a97638"
dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:48014
I20260709 13:09:02.387324 24619 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec03d81ef4cd465ea21e967d288c7faf"
dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
 from {username='slave'} at 127.0.0.1:38802
I20260709 13:09:02.388298 24544 raft_consensus.cc:493] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:02.388357 24619 raft_consensus.cc:493] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:02.388880 24544 raft_consensus.cc:3060] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.388916 24619 raft_consensus.cc:3060] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.394610 24696 raft_consensus.cc:515] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.398006 24696 leader_election.cc:290] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 2 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:02.398968 24547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2f983a77dd426a80a8474fd87779db" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:02.399356 24622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2f983a77dd426a80a8474fd87779db" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:02.399991 24547 raft_consensus.cc:3060] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.400097 24622 raft_consensus.cc:3055] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:02.400555 24622 raft_consensus.cc:740] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:02.401665 24622 consensus_queue.cc:260] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.403580 24622 raft_consensus.cc:3060] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.408964 24619 raft_consensus.cc:515] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.412204 24619 leader_election.cc:290] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 2 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:02.414772 24545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec03d81ef4cd465ea21e967d288c7faf" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:02.415817 24545 raft_consensus.cc:3060] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.417434 24546 raft_consensus.cc:515] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.420994 24546 leader_election.cc:290] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Requested vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:02.422003 24619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d348aca96c4a3296d4d0b6b028c1fd" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:02.422885 24619 raft_consensus.cc:3055] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:02.423281 24619 raft_consensus.cc:740] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:02.423830 24547 raft_consensus.cc:2468] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 2.
I20260709 13:09:02.425449 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec03d81ef4cd465ea21e967d288c7faf" candidate_uuid: "76f7ff986dda425baf2cae8ca9b96487" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:02.425827 24693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d348aca96c4a3296d4d0b6b028c1fd" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:02.426208 24696 raft_consensus.cc:3055] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:02.426623 24693 raft_consensus.cc:3060] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.426733 24696 raft_consensus.cc:740] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:02.427939 24696 consensus_queue.cc:260] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.429589 24696 raft_consensus.cc:3060] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.426080 24660 leader_election.cc:304] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:02.431583 24863 raft_consensus.cc:2804] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:02.432150 24863 raft_consensus.cc:697] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 2 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:02.424598 24619 consensus_queue.cc:260] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.434007 24619 raft_consensus.cc:3060] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.433421 24863 consensus_queue.cc:237] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.437659 24693 raft_consensus.cc:2468] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:09:02.439622 24512 leader_election.cc:304] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:02.441839 24867 raft_consensus.cc:2804] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:02.442422 24867 raft_consensus.cc:697] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Becoming Leader. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:09:02.443665 24867 consensus_queue.cc:237] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.445684 24622 raft_consensus.cc:2468] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 2.
I20260709 13:09:02.445684 24545 raft_consensus.cc:2468] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 2.
I20260709 13:09:02.448508 24586 leader_election.cc:304] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, 76f7ff986dda425baf2cae8ca9b96487; no voters: 
I20260709 13:09:02.450248 24544 raft_consensus.cc:515] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.450707 24444 catalog_manager.cc:5697] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 1 to 2, leader changed from bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67) to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:02.453477 24544 leader_election.cc:290] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Requested vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:02.453526 24870 raft_consensus.cc:2804] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:02.457684 24870 raft_consensus.cc:697] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Becoming Leader. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:02.459103 24870 consensus_queue.cc:237] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.461587 24619 raft_consensus.cc:2468] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:09:02.454844 24622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf66736913843acae755cc3c5a97638" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:02.456475 24695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf66736913843acae755cc3c5a97638" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:02.463855 24622 raft_consensus.cc:3055] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:02.463989 24695 raft_consensus.cc:3060] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.464478 24445 catalog_manager.cc:5697] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 reported cstate change: term changed from 1 to 2, leader changed from 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66) to 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "3fc09a7ccf854d5da51529f58bd82889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:02.464313 24622 raft_consensus.cc:740] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:02.467000 24622 consensus_queue.cc:260] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.469504 24622 raft_consensus.cc:3060] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.474210 24441 catalog_manager.cc:5697] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 1 to 2, leader changed from 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66) to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:02.478379 24696 raft_consensus.cc:2468] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76f7ff986dda425baf2cae8ca9b96487 in term 2.
I20260709 13:09:02.479951 24695 raft_consensus.cc:2468] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:09:02.481519 24512 leader_election.cc:304] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:02.482607 24867 raft_consensus.cc:2804] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:02.483111 24867 raft_consensus.cc:697] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Becoming Leader. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:09:02.484153 24867 consensus_queue.cc:237] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.486585 24622 raft_consensus.cc:2468] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:09:02.491647 24441 catalog_manager.cc:5697] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 reported cstate change: term changed from 1 to 2, leader changed from bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67) to 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "76f7ff986dda425baf2cae8ca9b96487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:02.493927 24444 catalog_manager.cc:5697] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 reported cstate change: term changed from 1 to 2, leader changed from 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66) to 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "3fc09a7ccf854d5da51529f58bd82889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:02.595487 24544 raft_consensus.cc:1275] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Refusing update from remote peer 76f7ff986dda425baf2cae8ca9b96487: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.597599 24880 consensus_queue.cc:1048] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:02.611533 24695 raft_consensus.cc:1275] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Refusing update from remote peer 76f7ff986dda425baf2cae8ca9b96487: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.613142 24880 consensus_queue.cc:1048] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:02.615578 24825 raft_consensus.cc:993] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8: : Instructing follower 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:09:02.616266 24825 raft_consensus.cc:1081] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Signalling peer 3fc09a7ccf854d5da51529f58bd82889 to start an election
I20260709 13:09:02.618147 24547 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93156e55d4034667965228d19a646e50"
dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
 from {username='slave'} at 127.0.0.1:48012
I20260709 13:09:02.618762 24547 raft_consensus.cc:493] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:02.618947 24622 raft_consensus.cc:1275] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.619195 24547 raft_consensus.cc:3060] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.620906 24867 consensus_queue.cc:1048] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:02.641901 24547 raft_consensus.cc:515] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.649866 24547 leader_election.cc:290] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Requested vote from peers 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999), bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67:46355)
I20260709 13:09:02.651748 24696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93156e55d4034667965228d19a646e50" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
I20260709 13:09:02.652412 24695 raft_consensus.cc:1275] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.652593 24696 raft_consensus.cc:3055] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:02.652951 24696 raft_consensus.cc:740] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:02.654203 24696 consensus_queue.cc:260] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.658005 24696 raft_consensus.cc:3060] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.666296 24619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93156e55d4034667965228d19a646e50" candidate_uuid: "3fc09a7ccf854d5da51529f58bd82889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:02.666972 24619 raft_consensus.cc:3060] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:02.668419 24696 raft_consensus.cc:2468] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:09:02.671283 24512 leader_election.cc:304] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:02.673578 24922 consensus_queue.cc:1048] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:02.676723 24767 raft_consensus.cc:2804] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:02.677206 24619 raft_consensus.cc:2468] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc09a7ccf854d5da51529f58bd82889 in term 2.
I20260709 13:09:02.680111 24767 raft_consensus.cc:697] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Becoming Leader. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:09:02.682104 24767 consensus_queue.cc:237] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:02.690512 24441 catalog_manager.cc:5697] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 reported cstate change: term changed from 1 to 2, leader changed from bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67) to 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65). New cstate: current_term: 2 leader_uuid: "3fc09a7ccf854d5da51529f58bd82889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:02.791620 24696 raft_consensus.cc:1275] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Refusing update from remote peer 76f7ff986dda425baf2cae8ca9b96487: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.793164 24870 consensus_queue.cc:1048] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:02.803162 24544 raft_consensus.cc:1275] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Refusing update from remote peer 76f7ff986dda425baf2cae8ca9b96487: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.804828 24870 consensus_queue.cc:1048] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:02.851429 24544 raft_consensus.cc:1275] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Refusing update from remote peer bd09d620c95a44bcb4ac800897adc6f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.852851 24863 consensus_queue.cc:1048] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:02.863822 24619 raft_consensus.cc:1275] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Refusing update from remote peer bd09d620c95a44bcb4ac800897adc6f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.866458 24863 consensus_queue.cc:1048] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:02.873287 24622 raft_consensus.cc:1275] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.877158 24767 consensus_queue.cc:1048] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:02.878139 24619 raft_consensus.cc:1275] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.880206 24767 consensus_queue.cc:1048] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:02.892786 24696 raft_consensus.cc:1275] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.895238 24920 consensus_queue.cc:1048] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:02.898367 24696 raft_consensus.cc:1275] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.900730 24921 consensus_queue.cc:1048] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:03.026518 24696 raft_consensus.cc:1275] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Refusing update from remote peer 76f7ff986dda425baf2cae8ca9b96487: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:03.028093 24870 consensus_queue.cc:1048] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:03.038020 24544 raft_consensus.cc:1275] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Refusing update from remote peer 76f7ff986dda425baf2cae8ca9b96487: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:03.040259 24837 consensus_queue.cc:1048] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:03.135839 24696 raft_consensus.cc:1275] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:03.137107 24921 consensus_queue.cc:1048] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:03.147449 24619 raft_consensus.cc:1275] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Refusing update from remote peer 3fc09a7ccf854d5da51529f58bd82889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:03.150028 24923 consensus_queue.cc:1048] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:05.157810 24444 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38576:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:09:05.182125 24444 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38576:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:09:05.267066 12725 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:09:05.439159 24970 tablet.cc:1724] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:05.498734 24573 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:93156e55d4034667965228d19a646e50)
W20260709 13:09:05.512166 24573 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:93156e55d4034667965228d19a646e50)
W20260709 13:09:05.513321 24573 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:93156e55d4034667965228d19a646e50)
W20260709 13:09:05.525460 24513 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.66:38999, user_credentials={real_user=slave}} blocked reactor thread for 150872us
I20260709 13:09:05.479579 24975 tablet.cc:1724] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.588982 24979 tablet.cc:1724] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.599862 24984 tablet.cc:1724] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:05.610755 24586 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:39367, user_credentials={real_user=slave}} blocked reactor thread for 63405.7us
W20260709 13:09:05.621564 24981 log.cc:927] Time spent T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:09:05.629412 24986 tablet.cc:1724] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:05.674383 24586 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:39367, user_credentials={real_user=slave}} blocked reactor thread for 56672.2us
I20260709 13:09:05.715830 25003 tablet.cc:1724] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.730985 24970 tablet.cc:1724] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.732527 25001 tablet.cc:1724] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.730985 25004 tablet.cc:1724] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.750619 25000 tablet.cc:1724] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.757841 24975 tablet.cc:1724] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.766278 25002 tablet.cc:1724] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.771019 24984 tablet.cc:1724] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.775120 24621 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ffb6ee5eb0c4379ac6d560539f4ed41"
dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
mode: GRACEFUL
new_leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
 from {username='slave'} at 127.0.0.1:35438
I20260709 13:09:05.778949 24621 raft_consensus.cc:606] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Received request to transfer leadership to TS bd09d620c95a44bcb4ac800897adc6f8
W20260709 13:09:05.786397 24660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:39367, user_credentials={real_user=slave}} blocked reactor thread for 74992.4us
I20260709 13:09:05.807519 25008 tablet.cc:1724] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.798032 25004 tablet.cc:1724] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.824105 24970 tablet.cc:1724] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.870555 24975 tablet.cc:1724] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.887782 25009 tablet.cc:1724] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.896487 25001 tablet.cc:1724] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.896569 24979 tablet.cc:1724] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.901355 24543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93156e55d4034667965228d19a646e50"
dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
mode: GRACEFUL
new_leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
 from {username='slave'} at 127.0.0.1:41192
I20260709 13:09:05.902043 24543 raft_consensus.cc:606] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Received request to transfer leadership to TS bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:09:05.906508 12725 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:09:05.907289 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:05.906721 24986 tablet.cc:1724] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.867550 24984 tablet.cc:1724] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.918612 24621 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c"
dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
mode: GRACEFUL
new_leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
 from {username='slave'} at 127.0.0.1:35438
I20260709 13:09:05.919238 24621 raft_consensus.cc:606] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Received request to transfer leadership to TS bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:09:05.922887 25007 raft_consensus.cc:993] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487: : Instructing follower bd09d620c95a44bcb4ac800897adc6f8 to start an election
I20260709 13:09:05.923494 24996 raft_consensus.cc:1081] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Signalling peer bd09d620c95a44bcb4ac800897adc6f8 to start an election
I20260709 13:09:05.927855 24694 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ffb6ee5eb0c4379ac6d560539f4ed41"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
 from {username='slave'} at 127.0.0.1:45616
I20260709 13:09:05.928437 24694 raft_consensus.cc:493] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:05.928774 24694 raft_consensus.cc:3060] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:05.928889 25002 tablet.cc:1724] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.935994 24543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56"
dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
mode: GRACEFUL
new_leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
 from {username='slave'} at 127.0.0.1:41192
I20260709 13:09:05.938271 24694 raft_consensus.cc:515] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:05.938323 24543 raft_consensus.cc:606] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Received request to transfer leadership to TS bd09d620c95a44bcb4ac800897adc6f8
I20260709 13:09:05.943045 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:09:05.937888 24970 tablet.cc:1724] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.943811 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:05.950788 24543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ffb6ee5eb0c4379ac6d560539f4ed41" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:05.938454 25000 tablet.cc:1724] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.951604 24543 raft_consensus.cc:3060] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:05.958245 24694 leader_election.cc:290] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 2 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:05.959831 24618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ffb6ee5eb0c4379ac6d560539f4ed41" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:05.964023 24618 raft_consensus.cc:3055] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:05.964442 24618 raft_consensus.cc:740] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:05.965524 24618 consensus_queue.cc:260] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:05.934176 25009 tablet.cc:1724] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.970585 24543 raft_consensus.cc:2468] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 2.
I20260709 13:09:05.979794 24660 leader_election.cc:304] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:05.980170 24618 raft_consensus.cc:3060] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:05.979893 24984 tablet.cc:1724] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:05.981631 25006 raft_consensus.cc:2804] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:05.982193 25006 raft_consensus.cc:697] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:05.983230 25006 consensus_queue.cc:237] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:06.007323 24618 raft_consensus.cc:2468] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 2.
I20260709 13:09:06.015089 24444 catalog_manager.cc:6242] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2: soft_deleted_table [id=ef7efc38dca649eb93b507559fe50eaf] alter complete (version 1)
I20260709 13:09:06.025641 24445 catalog_manager.cc:5697] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 1 to 2, leader changed from 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66) to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 2 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:06.176401 24966 raft_consensus.cc:993] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889: : Instructing follower bd09d620c95a44bcb4ac800897adc6f8 to start an election
I20260709 13:09:06.177052 24998 raft_consensus.cc:1081] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Signalling peer bd09d620c95a44bcb4ac800897adc6f8 to start an election
I20260709 13:09:06.178996 24694 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93156e55d4034667965228d19a646e50"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
 from {username='slave'} at 127.0.0.1:45608
I20260709 13:09:06.179605 24694 raft_consensus.cc:493] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:06.179987 24694 raft_consensus.cc:3060] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:06.183285 24996 raft_consensus.cc:993] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487: : Instructing follower bd09d620c95a44bcb4ac800897adc6f8 to start an election
I20260709 13:09:06.183841 24869 raft_consensus.cc:1081] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Signalling peer bd09d620c95a44bcb4ac800897adc6f8 to start an election
I20260709 13:09:06.185415 24689 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
 from {username='slave'} at 127.0.0.1:45616
I20260709 13:09:06.185907 24689 raft_consensus.cc:493] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:06.186250 24689 raft_consensus.cc:3060] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:06.187074 24694 raft_consensus.cc:515] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:06.188730 24694 leader_election.cc:290] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 3 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:06.189743 24543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93156e55d4034667965228d19a646e50" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:06.189961 24618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93156e55d4034667965228d19a646e50" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:06.190454 24543 raft_consensus.cc:3055] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:06.190667 24618 raft_consensus.cc:3060] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:06.190820 24543 raft_consensus.cc:740] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:09:06.191787 24543 consensus_queue.cc:260] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:06.192830 24543 raft_consensus.cc:3060] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:06.194003 24689 raft_consensus.cc:515] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:06.196413 24689 leader_election.cc:290] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 3 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:06.197355 24544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:06.198027 24544 raft_consensus.cc:3060] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:06.199713 24618 raft_consensus.cc:2468] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 3.
I20260709 13:09:06.201028 24543 raft_consensus.cc:2468] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 3.
I20260709 13:09:06.201068 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a990da3933643cca63dbfbd7b18205c" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:06.201390 24662 leader_election.cc:304] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:06.201916 24621 raft_consensus.cc:3055] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:06.202337 24621 raft_consensus.cc:740] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 76f7ff986dda425baf2cae8ca9b96487, State: Running, Role: LEADER
I20260709 13:09:06.203344 25006 raft_consensus.cc:2804] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:06.203245 24621 consensus_queue.cc:260] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:06.203923 25006 raft_consensus.cc:697] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 3 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:06.204805 24621 raft_consensus.cc:3060] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:06.204873 25006 consensus_queue.cc:237] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:06.205250 24544 raft_consensus.cc:2468] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 3.
I20260709 13:09:06.206746 24660 leader_election.cc:304] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc09a7ccf854d5da51529f58bd82889, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:06.207762 24983 raft_consensus.cc:2804] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:06.208235 24983 raft_consensus.cc:697] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 3 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:06.209187 24983 consensus_queue.cc:237] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:06.217883 24621 raft_consensus.cc:2468] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 3.
I20260709 13:09:06.220184 24445 catalog_manager.cc:5697] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 2 to 3, leader changed from 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65) to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:06.221726 24445 catalog_manager.cc:5697] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 2 to 3, leader changed from 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66) to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:06.336777 24966 raft_consensus.cc:993] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889: : Instructing follower bd09d620c95a44bcb4ac800897adc6f8 to start an election
I20260709 13:09:06.337296 24998 raft_consensus.cc:1081] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Signalling peer bd09d620c95a44bcb4ac800897adc6f8 to start an election
I20260709 13:09:06.338888 24689 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56"
dest_uuid: "bd09d620c95a44bcb4ac800897adc6f8"
 from {username='slave'} at 127.0.0.1:45608
I20260709 13:09:06.339428 24689 raft_consensus.cc:493] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:06.339766 24689 raft_consensus.cc:3060] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:06.346403 24689 raft_consensus.cc:515] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:06.348340 24689 leader_election.cc:290] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 3 election: Requested vote from peers 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367), 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999)
I20260709 13:09:06.349231 24544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3fc09a7ccf854d5da51529f58bd82889"
I20260709 13:09:06.349391 24621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d8224ca1a24fafab3cb86dead9af56" candidate_uuid: "bd09d620c95a44bcb4ac800897adc6f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "76f7ff986dda425baf2cae8ca9b96487"
I20260709 13:09:06.350157 24544 raft_consensus.cc:3055] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:06.350307 24621 raft_consensus.cc:3060] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:06.350505 24544 raft_consensus.cc:740] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3fc09a7ccf854d5da51529f58bd82889, State: Running, Role: LEADER
I20260709 13:09:06.351421 24544 consensus_queue.cc:260] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:06.352523 24544 raft_consensus.cc:3060] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:06.356402 24621 raft_consensus.cc:2468] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 3.
I20260709 13:09:06.357812 24662 leader_election.cc:304] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76f7ff986dda425baf2cae8ca9b96487, bd09d620c95a44bcb4ac800897adc6f8; no voters: 
I20260709 13:09:06.358714 24983 raft_consensus.cc:2804] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:06.359412 24983 raft_consensus.cc:697] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 3 LEADER]: Becoming Leader. State: Replica: bd09d620c95a44bcb4ac800897adc6f8, State: Running, Role: LEADER
I20260709 13:09:06.360369 24983 consensus_queue.cc:237] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } }
I20260709 13:09:06.358491 24544 raft_consensus.cc:2468] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd09d620c95a44bcb4ac800897adc6f8 in term 3.
I20260709 13:09:06.376251 24445 catalog_manager.cc:5697] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 reported cstate change: term changed from 2 to 3, leader changed from 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65) to bd09d620c95a44bcb4ac800897adc6f8 (127.12.109.67). New cstate: current_term: 3 leader_uuid: "bd09d620c95a44bcb4ac800897adc6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd09d620c95a44bcb4ac800897adc6f8" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 46355 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:06.564533 24621 raft_consensus.cc:1275] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Refusing update from remote peer bd09d620c95a44bcb4ac800897adc6f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:06.565758 24983 consensus_queue.cc:1048] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:06.575558 24544 raft_consensus.cc:1275] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Refusing update from remote peer bd09d620c95a44bcb4ac800897adc6f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:06.577845 24930 consensus_queue.cc:1048] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:06.737385 24544 raft_consensus.cc:1275] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 3 FOLLOWER]: Refusing update from remote peer bd09d620c95a44bcb4ac800897adc6f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:06.738878 24930 consensus_queue.cc:1048] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:06.749770 24621 raft_consensus.cc:1275] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 3 FOLLOWER]: Refusing update from remote peer bd09d620c95a44bcb4ac800897adc6f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:06.751384 25006 consensus_queue.cc:1048] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:06.783596 24621 raft_consensus.cc:1275] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 3 FOLLOWER]: Refusing update from remote peer bd09d620c95a44bcb4ac800897adc6f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:09:06.784992 25006 consensus_queue.cc:1048] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:06.791850 24544 raft_consensus.cc:1275] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 3 FOLLOWER]: Refusing update from remote peer bd09d620c95a44bcb4ac800897adc6f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:06.794939 24543 raft_consensus.cc:1275] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 3 FOLLOWER]: Refusing update from remote peer bd09d620c95a44bcb4ac800897adc6f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:09:06.798035 24983 consensus_queue.cc:1048] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:06.798512 24930 consensus_queue.cc:1048] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc09a7ccf854d5da51529f58bd82889" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:06.813632 24621 raft_consensus.cc:1275] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 3 FOLLOWER]: Refusing update from remote peer bd09d620c95a44bcb4ac800897adc6f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:06.817133 24983 consensus_queue.cc:1048] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f7ff986dda425baf2cae8ca9b96487" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:07.012578 12725 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:09:07.017899 12725 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:09:07.018448 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:07.024488 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:09:07.025007 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:08.058849 12725 tablet_server.cc:179] TabletServer@127.12.109.65:0 shutting down...
W20260709 13:09:08.062036 24586 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 13:09:08.066112 24586 consensus_peers.cc:597] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 -> Peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Couldn't send request to peer 3fc09a7ccf854d5da51529f58bd82889. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:08.098047 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:08.099133 12725 tablet_replica.cc:333] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.100076 12725 raft_consensus.cc:2243] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.100656 12725 raft_consensus.cc:2272] T ec03d81ef4cd465ea21e967d288c7faf P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.103209 12725 tablet_replica.cc:333] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.103868 12725 raft_consensus.cc:2243] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.104342 12725 raft_consensus.cc:2272] T 6e2f983a77dd426a80a8474fd87779db P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.106690 12725 tablet_replica.cc:333] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.107380 12725 raft_consensus.cc:2243] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.108283 12725 raft_consensus.cc:2272] T 86d348aca96c4a3296d4d0b6b028c1fd P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.110880 12725 tablet_replica.cc:333] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.111538 12725 raft_consensus.cc:2243] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.112020 12725 raft_consensus.cc:2272] T 93156e55d4034667965228d19a646e50 P 3fc09a7ccf854d5da51529f58bd82889 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.114399 12725 tablet_replica.cc:333] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.115079 12725 raft_consensus.cc:2243] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.115548 12725 raft_consensus.cc:2272] T d3aabffc54154f92895ab62d0e23456f P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.117762 12725 tablet_replica.cc:333] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.118419 12725 raft_consensus.cc:2243] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.119329 12725 raft_consensus.cc:2272] T efc2d1bcdba24abca06cff7b82e9e10c P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.121858 12725 tablet_replica.cc:333] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.122557 12725 raft_consensus.cc:2243] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.123061 12725 raft_consensus.cc:2272] T f1d3461c689940d7a393a912f7774f94 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.126293 12725 tablet_replica.cc:333] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.127012 12725 raft_consensus.cc:2243] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.127650 12725 raft_consensus.cc:2272] T 3aefcd1446da4fa0b24d4d5402d5c930 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.130923 12725 tablet_replica.cc:333] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.131803 12725 raft_consensus.cc:2243] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.132433 12725 raft_consensus.cc:2272] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.135004 12725 tablet_replica.cc:333] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.135643 12725 raft_consensus.cc:2243] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.136474 12725 raft_consensus.cc:2272] T 6bf66736913843acae755cc3c5a97638 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.138656 12725 tablet_replica.cc:333] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.139256 12725 raft_consensus.cc:2243] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.139732 12725 raft_consensus.cc:2272] T 81d8224ca1a24fafab3cb86dead9af56 P 3fc09a7ccf854d5da51529f58bd82889 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.142105 12725 tablet_replica.cc:333] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.142714 12725 raft_consensus.cc:2243] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.143653 12725 raft_consensus.cc:2272] T f032c4d4202141c6b52d130b37bafbe8 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.146207 12725 tablet_replica.cc:333] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.146816 12725 raft_consensus.cc:2243] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.147740 12725 raft_consensus.cc:2272] T 5a99475fd525406090ecf4d7481216f6 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.150255 12725 tablet_replica.cc:333] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.150906 12725 raft_consensus.cc:2243] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.151350 12725 raft_consensus.cc:2272] T 62426280cfa2407189d6a8ad14efea38 P 3fc09a7ccf854d5da51529f58bd82889 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.154026 12725 tablet_replica.cc:333] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.154713 12725 raft_consensus.cc:2243] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.155256 12725 raft_consensus.cc:2272] T 3c896072f0f34ec2b5d9a2c0ee528cde P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.158332 12725 tablet_replica.cc:333] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.159204 12725 raft_consensus.cc:2243] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.159785 12725 raft_consensus.cc:2272] T 281735360a8f4634b84dc1e9262ea6b2 P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.162729 12725 tablet_replica.cc:333] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
W20260709 13:09:08.163021 24586 consensus_peers.cc:597] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 -> Peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Couldn't send request to peer 3fc09a7ccf854d5da51529f58bd82889. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:08.163354 12725 raft_consensus.cc:2243] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.164436 12725 raft_consensus.cc:2272] T ee5f0962f65146f3b7ee29957ffda06b P 3fc09a7ccf854d5da51529f58bd82889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.166750 12725 tablet_replica.cc:333] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889: stopping tablet replica
I20260709 13:09:08.167341 12725 raft_consensus.cc:2243] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.167928 12725 raft_consensus.cc:2272] T 6a990da3933643cca63dbfbd7b18205c P 3fc09a7ccf854d5da51529f58bd82889 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:08.185833 24586 consensus_peers.cc:597] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 -> Peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Couldn't send request to peer 3fc09a7ccf854d5da51529f58bd82889. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:08.218860 12725 tablet_server.cc:196] TabletServer@127.12.109.65:0 shutdown complete.
W20260709 13:09:08.223284 24660 consensus_peers.cc:597] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 -> Peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Couldn't send request to peer 3fc09a7ccf854d5da51529f58bd82889. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:39367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:08.226485 24660 consensus_peers.cc:597] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 -> Peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Couldn't send request to peer 3fc09a7ccf854d5da51529f58bd82889. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:39367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:08.237265 24586 consensus_peers.cc:597] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 -> Peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Couldn't send request to peer 3fc09a7ccf854d5da51529f58bd82889. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:39367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:08.237470 24660 consensus_peers.cc:597] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 -> Peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Couldn't send request to peer 3fc09a7ccf854d5da51529f58bd82889. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:39367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:08.266489 12725 tablet_server.cc:179] TabletServer@127.12.109.66:0 shutting down...
W20260709 13:09:08.273456 24662 consensus_peers.cc:597] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 -> Peer 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999): Couldn't send request to peer 76f7ff986dda425baf2cae8ca9b96487. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:08.295765 24586 consensus_peers.cc:597] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 -> Peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Couldn't send request to peer 3fc09a7ccf854d5da51529f58bd82889. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:39367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:08.311143 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:08.312166 12725 tablet_replica.cc:333] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.313372 12725 raft_consensus.cc:2243] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.313993 12725 raft_consensus.cc:2272] T 86d348aca96c4a3296d4d0b6b028c1fd P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.316623 12725 tablet_replica.cc:333] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.317265 12725 raft_consensus.cc:2243] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.317655 12725 raft_consensus.cc:2272] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.319859 12725 tablet_replica.cc:333] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.320429 12725 raft_consensus.cc:2243] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.321254 12725 raft_consensus.cc:2272] T 3aefcd1446da4fa0b24d4d5402d5c930 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.323661 12725 tablet_replica.cc:333] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.324188 12725 raft_consensus.cc:2243] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.324615 12725 raft_consensus.cc:2272] T 6e2f983a77dd426a80a8474fd87779db P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.326804 12725 tablet_replica.cc:333] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.327499 12725 raft_consensus.cc:2243] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.328545 12725 raft_consensus.cc:2272] T ec03d81ef4cd465ea21e967d288c7faf P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.331269 12725 tablet_replica.cc:333] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.332012 12725 raft_consensus.cc:2243] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.332661 12725 raft_consensus.cc:2272] T 93156e55d4034667965228d19a646e50 P 76f7ff986dda425baf2cae8ca9b96487 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.335134 12725 tablet_replica.cc:333] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.335632 12725 raft_consensus.cc:2243] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.336424 12725 raft_consensus.cc:2272] T d3aabffc54154f92895ab62d0e23456f P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.338615 12725 tablet_replica.cc:333] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.339146 12725 raft_consensus.cc:2243] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.339594 12725 raft_consensus.cc:2272] T ee5f0962f65146f3b7ee29957ffda06b P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.341703 12725 tablet_replica.cc:333] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.342244 12725 raft_consensus.cc:2243] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.342638 12725 raft_consensus.cc:2272] T 6a990da3933643cca63dbfbd7b18205c P 76f7ff986dda425baf2cae8ca9b96487 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.344736 12725 tablet_replica.cc:333] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.345264 12725 raft_consensus.cc:2243] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.346100 12725 raft_consensus.cc:2272] T f1d3461c689940d7a393a912f7774f94 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.348299 12725 tablet_replica.cc:333] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.348883 12725 raft_consensus.cc:2243] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.349272 12725 raft_consensus.cc:2272] T f032c4d4202141c6b52d130b37bafbe8 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.351697 12725 tablet_replica.cc:333] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.352209 12725 raft_consensus.cc:2243] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.353080 12725 raft_consensus.cc:2272] T 62426280cfa2407189d6a8ad14efea38 P 76f7ff986dda425baf2cae8ca9b96487 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.355626 12725 tablet_replica.cc:333] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.356218 12725 raft_consensus.cc:2243] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.356694 12725 raft_consensus.cc:2272] T 5a99475fd525406090ecf4d7481216f6 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.359220 12725 tablet_replica.cc:333] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.359896 12725 raft_consensus.cc:2243] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.360409 12725 raft_consensus.cc:2272] T 6bf66736913843acae755cc3c5a97638 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.362725 12725 tablet_replica.cc:333] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.363260 12725 raft_consensus.cc:2243] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.363786 12725 raft_consensus.cc:2272] T 81d8224ca1a24fafab3cb86dead9af56 P 76f7ff986dda425baf2cae8ca9b96487 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.365922 12725 tablet_replica.cc:333] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.366768 12725 raft_consensus.cc:2243] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.367290 12725 raft_consensus.cc:2272] T 3c896072f0f34ec2b5d9a2c0ee528cde P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.370338 12725 tablet_replica.cc:333] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.370862 12725 raft_consensus.cc:2243] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.371732 12725 raft_consensus.cc:2272] T 281735360a8f4634b84dc1e9262ea6b2 P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.373943 12725 tablet_replica.cc:333] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487: stopping tablet replica
I20260709 13:09:08.374516 12725 raft_consensus.cc:2243] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.374936 12725 raft_consensus.cc:2272] T efc2d1bcdba24abca06cff7b82e9e10c P 76f7ff986dda425baf2cae8ca9b96487 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:08.407475 24660 consensus_peers.cc:597] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 -> Peer 3fc09a7ccf854d5da51529f58bd82889 (127.12.109.65:39367): Couldn't send request to peer 3fc09a7ccf854d5da51529f58bd82889. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:39367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:08.422734 12725 tablet_server.cc:196] TabletServer@127.12.109.66:0 shutdown complete.
I20260709 13:09:08.465366 12725 tablet_server.cc:179] TabletServer@127.12.109.67:0 shutting down...
I20260709 13:09:08.499812 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:08.500669 12725 tablet_replica.cc:333] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.501464 12725 raft_consensus.cc:2243] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.502151 12725 raft_consensus.cc:2272] T 86d348aca96c4a3296d4d0b6b028c1fd P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.504427 12725 tablet_replica.cc:333] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.505067 12725 raft_consensus.cc:2243] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.505559 12725 raft_consensus.cc:2272] T d3aabffc54154f92895ab62d0e23456f P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.507894 12725 tablet_replica.cc:333] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.508479 12725 raft_consensus.cc:2243] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.509406 12725 raft_consensus.cc:2272] T 6e2f983a77dd426a80a8474fd87779db P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.511673 12725 tablet_replica.cc:333] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.512255 12725 raft_consensus.cc:2243] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.512712 12725 raft_consensus.cc:2272] T ec03d81ef4cd465ea21e967d288c7faf P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.514853 12725 tablet_replica.cc:333] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.515422 12725 raft_consensus.cc:2243] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.516244 12725 raft_consensus.cc:2272] T 93156e55d4034667965228d19a646e50 P bd09d620c95a44bcb4ac800897adc6f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.518482 12725 tablet_replica.cc:333] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.519054 12725 raft_consensus.cc:2243] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.519517 12725 raft_consensus.cc:2272] T 6bf66736913843acae755cc3c5a97638 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.522418 12725 tablet_replica.cc:333] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.523185 12725 raft_consensus.cc:2243] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.524197 12725 raft_consensus.cc:2272] T 81d8224ca1a24fafab3cb86dead9af56 P bd09d620c95a44bcb4ac800897adc6f8 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:08.525557 24662 consensus_peers.cc:597] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 -> Peer 76f7ff986dda425baf2cae8ca9b96487 (127.12.109.66:38999): Couldn't send request to peer 76f7ff986dda425baf2cae8ca9b96487. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.66:38999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:08.527012 12725 tablet_replica.cc:333] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.527714 12725 raft_consensus.cc:2243] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.528533 12725 raft_consensus.cc:2272] T 3c896072f0f34ec2b5d9a2c0ee528cde P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.530735 12725 tablet_replica.cc:333] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.531301 12725 raft_consensus.cc:2243] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.531711 12725 raft_consensus.cc:2272] T f032c4d4202141c6b52d130b37bafbe8 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.534158 12725 tablet_replica.cc:333] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.534725 12725 raft_consensus.cc:2243] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.535133 12725 raft_consensus.cc:2272] T 5a99475fd525406090ecf4d7481216f6 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.537292 12725 tablet_replica.cc:333] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.537839 12725 raft_consensus.cc:2243] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.538408 12725 raft_consensus.cc:2272] T 62426280cfa2407189d6a8ad14efea38 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.540518 12725 tablet_replica.cc:333] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.541008 12725 raft_consensus.cc:2243] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.541406 12725 raft_consensus.cc:2272] T ee5f0962f65146f3b7ee29957ffda06b P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.543601 12725 tablet_replica.cc:333] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.544162 12725 raft_consensus.cc:2243] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.544991 12725 raft_consensus.cc:2272] T 6a990da3933643cca63dbfbd7b18205c P bd09d620c95a44bcb4ac800897adc6f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.547222 12725 tablet_replica.cc:333] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.547801 12725 raft_consensus.cc:2243] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.548632 12725 raft_consensus.cc:2272] T 3ffb6ee5eb0c4379ac6d560539f4ed41 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.550673 12725 tablet_replica.cc:333] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.551204 12725 raft_consensus.cc:2243] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.551689 12725 raft_consensus.cc:2272] T 3aefcd1446da4fa0b24d4d5402d5c930 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.553797 12725 tablet_replica.cc:333] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.554364 12725 raft_consensus.cc:2243] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.554826 12725 raft_consensus.cc:2272] T efc2d1bcdba24abca06cff7b82e9e10c P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.556928 12725 tablet_replica.cc:333] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.557466 12725 raft_consensus.cc:2243] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.557919 12725 raft_consensus.cc:2272] T 281735360a8f4634b84dc1e9262ea6b2 P bd09d620c95a44bcb4ac800897adc6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.560088 12725 tablet_replica.cc:333] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8: stopping tablet replica
I20260709 13:09:08.560580 12725 raft_consensus.cc:2243] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:08.561008 12725 raft_consensus.cc:2272] T f1d3461c689940d7a393a912f7774f94 P bd09d620c95a44bcb4ac800897adc6f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.604351 12725 tablet_server.cc:196] TabletServer@127.12.109.67:0 shutdown complete.
I20260709 13:09:08.647161 12725 master.cc:562] Master@127.12.109.126:38639 shutting down...
I20260709 13:09:08.671145 12725 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:08.671769 12725 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:08.672106 12725 tablet_replica.cc:333] T 00000000000000000000000000000000 P d8bfccc8a5774a3e96866061556447b2: stopping tablet replica
I20260709 13:09:08.693701 12725 master.cc:584] Master@127.12.109.126:38639 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (13078 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:09:08.754782 12725 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.109.126:43651
I20260709 13:09:08.756119 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:08.763355 25032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:08.763470 25031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:08.764237 25034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:08.765066 12725 server_base.cc:1061] running on GCE node
I20260709 13:09:08.766160 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:08.766352 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:08.766472 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602548766459 us; error 0 us; skew 500 ppm
I20260709 13:09:08.766959 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:08.769706 12725 webserver.cc:533] Webserver started at http://127.12.109.126:42081/ using document root <none> and password file <none>
I20260709 13:09:08.770627 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:08.770829 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:08.771070 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:08.772305 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/master-0-root/instance:
uuid: "f047b5858163488e82044de7758dcc34"
format_stamp: "Formatted at 2026-07-09 13:09:08 on dist-test-slave-zhdc"
I20260709 13:09:08.777204 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:09:08.780939 25039 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:08.781805 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:09:08.782181 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/master-0-root
uuid: "f047b5858163488e82044de7758dcc34"
format_stamp: "Formatted at 2026-07-09 13:09:08 on dist-test-slave-zhdc"
I20260709 13:09:08.782526 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:08.795322 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:08.796846 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:08.841317 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.126:43651
I20260709 13:09:08.841466 25090 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.126:43651 every 8 connection(s)
I20260709 13:09:08.845949 25091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.858518 25091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34: Bootstrap starting.
I20260709 13:09:08.864244 25091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.869256 25091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34: No bootstrap required, opened a new log
I20260709 13:09:08.872138 25091 raft_consensus.cc:359] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f047b5858163488e82044de7758dcc34" member_type: VOTER }
I20260709 13:09:08.872653 25091 raft_consensus.cc:385] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.872844 25091 raft_consensus.cc:740] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f047b5858163488e82044de7758dcc34, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.873422 25091 consensus_queue.cc:260] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f047b5858163488e82044de7758dcc34" member_type: VOTER }
I20260709 13:09:08.873847 25091 raft_consensus.cc:399] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:09:08.874127 25091 raft_consensus.cc:493] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:09:08.874380 25091 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.879899 25091 raft_consensus.cc:515] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f047b5858163488e82044de7758dcc34" member_type: VOTER }
I20260709 13:09:08.880406 25091 leader_election.cc:304] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f047b5858163488e82044de7758dcc34; no voters: 
I20260709 13:09:08.881623 25091 leader_election.cc:290] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:09:08.882028 25094 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:08.883740 25094 raft_consensus.cc:697] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [term 1 LEADER]: Becoming Leader. State: Replica: f047b5858163488e82044de7758dcc34, State: Running, Role: LEADER
I20260709 13:09:08.884495 25094 consensus_queue.cc:237] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f047b5858163488e82044de7758dcc34" member_type: VOTER }
I20260709 13:09:08.885062 25091 sys_catalog.cc:565] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:09:08.889912 25096 sys_catalog.cc:455] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f047b5858163488e82044de7758dcc34. Latest consensus state: current_term: 1 leader_uuid: "f047b5858163488e82044de7758dcc34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f047b5858163488e82044de7758dcc34" member_type: VOTER } }
I20260709 13:09:08.889957 25095 sys_catalog.cc:455] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f047b5858163488e82044de7758dcc34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f047b5858163488e82044de7758dcc34" member_type: VOTER } }
I20260709 13:09:08.890801 25096 sys_catalog.cc:458] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:08.890825 25095 sys_catalog.cc:458] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:08.900089 25101 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:09:08.905803 25101 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:09:08.907163 12725 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:09:08.915808 25101 catalog_manager.cc:1383] Generated new cluster ID: db7e241bfc45464f875b46643f094c39
I20260709 13:09:08.916355 25101 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:09:08.938135 25101 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:09:08.939682 25101 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:09:08.958765 25101 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34: Generated new TSK 0
I20260709 13:09:08.959503 25101 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:09:08.975142 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:08.981961 25112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:08.983418 25113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:08.984936 25115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:08.985859 12725 server_base.cc:1061] running on GCE node
I20260709 13:09:08.987054 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:08.987287 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:08.987471 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602548987451 us; error 0 us; skew 500 ppm
I20260709 13:09:08.988021 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:08.990978 12725 webserver.cc:533] Webserver started at http://127.12.109.65:43659/ using document root <none> and password file <none>
I20260709 13:09:08.991562 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:08.991788 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:08.992091 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:08.993331 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-0-root/instance:
uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
format_stamp: "Formatted at 2026-07-09 13:09:08 on dist-test-slave-zhdc"
I20260709 13:09:08.999404 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 13:09:09.003329 25120 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:09.004298 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:09:09.004575 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-0-root
uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
format_stamp: "Formatted at 2026-07-09 13:09:08 on dist-test-slave-zhdc"
I20260709 13:09:09.004830 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:09.023161 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:09.024632 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:09.026515 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:09.029215 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:09.029415 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:09.029752 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:09.029933 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:09.078521 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.65:37649
I20260709 13:09:09.078660 25182 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.65:37649 every 8 connection(s)
I20260709 13:09:09.084440 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:09.092519 25187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:09.095140 25188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:09.095989 25190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:09.096978 12725 server_base.cc:1061] running on GCE node
I20260709 13:09:09.099165 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:09.099453 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:09.099717 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602549099696 us; error 0 us; skew 500 ppm
I20260709 13:09:09.099954 25183 heartbeater.cc:344] Connected to a master server at 127.12.109.126:43651
I20260709 13:09:09.100381 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:09.100445 25183 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:09.101662 25183 heartbeater.cc:507] Master 127.12.109.126:43651 requested a full tablet report, sending...
I20260709 13:09:09.103937 12725 webserver.cc:533] Webserver started at http://127.12.109.66:43247/ using document root <none> and password file <none>
I20260709 13:09:09.104732 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:09.104584 25056 ts_manager.cc:194] Registered new tserver with Master: bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649)
I20260709 13:09:09.105093 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:09.105489 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:09.107342 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-1-root/instance:
uuid: "c3395c72252c4138b3d77bf5f0d7d630"
format_stamp: "Formatted at 2026-07-09 13:09:09 on dist-test-slave-zhdc"
I20260709 13:09:09.107555 25056 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55802
I20260709 13:09:09.113075 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:09:09.117039 25195 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:09.117937 12725 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 13:09:09.118330 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-1-root
uuid: "c3395c72252c4138b3d77bf5f0d7d630"
format_stamp: "Formatted at 2026-07-09 13:09:09 on dist-test-slave-zhdc"
I20260709 13:09:09.118636 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:09.129554 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:09.131004 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:09.132717 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:09.135214 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:09.135454 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:09.135705 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:09.135916 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:09.182224 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.66:36153
I20260709 13:09:09.182276 25257 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.66:36153 every 8 connection(s)
I20260709 13:09:09.188196 12725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:09.197639 25262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:09.197690 25261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:09.201716 25264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:09.202986 25258 heartbeater.cc:344] Connected to a master server at 127.12.109.126:43651
I20260709 13:09:09.203473 25258 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:09.204285 25258 heartbeater.cc:507] Master 127.12.109.126:43651 requested a full tablet report, sending...
I20260709 13:09:09.204629 12725 server_base.cc:1061] running on GCE node
I20260709 13:09:09.205751 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:09.205996 12725 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:09.206290 12725 hybrid_clock.cc:648] HybridClock initialized: now 1783602549206266 us; error 0 us; skew 500 ppm
I20260709 13:09:09.206934 12725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:09.206847 25056 ts_manager.cc:194] Registered new tserver with Master: c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:09.209105 25056 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55816
I20260709 13:09:09.210091 12725 webserver.cc:533] Webserver started at http://127.12.109.67:34563/ using document root <none> and password file <none>
I20260709 13:09:09.210839 12725 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:09.211108 12725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:09.211459 12725 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:09.212985 12725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-2-root/instance:
uuid: "ac08c897ed7e478bbd7852f67f11caff"
format_stamp: "Formatted at 2026-07-09 13:09:09 on dist-test-slave-zhdc"
I20260709 13:09:09.217674 12725 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:09:09.221387 25269 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:09.222261 12725 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:09:09.222574 12725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-2-root
uuid: "ac08c897ed7e478bbd7852f67f11caff"
format_stamp: "Formatted at 2026-07-09 13:09:09 on dist-test-slave-zhdc"
I20260709 13:09:09.222859 12725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskmaBG8u/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602188851028-12725-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:09.240631 12725 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:09.242023 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:09.243744 12725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:09.246250 12725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:09.246465 12725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:09.246752 12725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:09.246899 12725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:09.292599 12725 rpc_server.cc:307] RPC server started. Bound to: 127.12.109.67:41425
I20260709 13:09:09.292712 25331 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.109.67:41425 every 8 connection(s)
I20260709 13:09:09.308365 25332 heartbeater.cc:344] Connected to a master server at 127.12.109.126:43651
I20260709 13:09:09.308928 25332 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:09.309790 25332 heartbeater.cc:507] Master 127.12.109.126:43651 requested a full tablet report, sending...
I20260709 13:09:09.312098 25056 ts_manager.cc:194] Registered new tserver with Master: ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:09.313185 12725 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017255546s
I20260709 13:09:09.313809 25056 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55822
I20260709 13:09:09.342785 25056 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55836:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:09.345695 25056 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:09.431809 25223 tablet_service.cc:1511] Processing CreateTablet for tablet e64433f5d83d40a2be74255a152a160a (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:09.433528 25223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e64433f5d83d40a2be74255a152a160a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.434965 25148 tablet_service.cc:1511] Processing CreateTablet for tablet e64433f5d83d40a2be74255a152a160a (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:09.436652 25148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e64433f5d83d40a2be74255a152a160a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.439267 25222 tablet_service.cc:1511] Processing CreateTablet for tablet afc98daf1f6a48d7a59b85bf860351e4 (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:09.440834 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afc98daf1f6a48d7a59b85bf860351e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.443774 25221 tablet_service.cc:1511] Processing CreateTablet for tablet 33e80f79ae7547a7a8e6bd6fd8d95dcb (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:09.445345 25221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e80f79ae7547a7a8e6bd6fd8d95dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.448601 25220 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa95aeebb5246e49558118198cf4d01 (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:09.449851 25146 tablet_service.cc:1511] Processing CreateTablet for tablet 33e80f79ae7547a7a8e6bd6fd8d95dcb (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:09.448601 25147 tablet_service.cc:1511] Processing CreateTablet for tablet afc98daf1f6a48d7a59b85bf860351e4 (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:09.451126 25220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa95aeebb5246e49558118198cf4d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.451457 25146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e80f79ae7547a7a8e6bd6fd8d95dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.454586 25145 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa95aeebb5246e49558118198cf4d01 (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:09.456354 25145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa95aeebb5246e49558118198cf4d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.462728 25147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afc98daf1f6a48d7a59b85bf860351e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.471858 25297 tablet_service.cc:1511] Processing CreateTablet for tablet e64433f5d83d40a2be74255a152a160a (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:09.473541 25297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e64433f5d83d40a2be74255a152a160a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.473176 25294 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa95aeebb5246e49558118198cf4d01 (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:09.474723 25294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa95aeebb5246e49558118198cf4d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.475061 25289 tablet_service.cc:1511] Processing CreateTablet for tablet dd30bf4abddf45c38bd41a6dbfbed3ac (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:09.475368 25144 tablet_service.cc:1511] Processing CreateTablet for tablet f79486558b854673aa41d16a1494868e (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:09.476552 25289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd30bf4abddf45c38bd41a6dbfbed3ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.476881 25144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f79486558b854673aa41d16a1494868e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.479466 25296 tablet_service.cc:1511] Processing CreateTablet for tablet afc98daf1f6a48d7a59b85bf860351e4 (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:09.481119 25296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afc98daf1f6a48d7a59b85bf860351e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.480743 25292 tablet_service.cc:1511] Processing CreateTablet for tablet 161bbc3aba054bcdabb265faed76e09e (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:09.483219 25295 tablet_service.cc:1511] Processing CreateTablet for tablet 33e80f79ae7547a7a8e6bd6fd8d95dcb (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:09.484814 25295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e80f79ae7547a7a8e6bd6fd8d95dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.484521 25290 tablet_service.cc:1511] Processing CreateTablet for tablet 58b3849e07f54c6ab1df11556d0c7212 (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:09.486029 25290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58b3849e07f54c6ab1df11556d0c7212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.473268 25219 tablet_service.cc:1511] Processing CreateTablet for tablet f79486558b854673aa41d16a1494868e (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:09.491623 25219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f79486558b854673aa41d16a1494868e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.479470 25293 tablet_service.cc:1511] Processing CreateTablet for tablet f79486558b854673aa41d16a1494868e (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:09.495734 25293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f79486558b854673aa41d16a1494868e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.483224 25291 tablet_service.cc:1511] Processing CreateTablet for tablet 034d7d105c1b4cfbbe230cfd8011cec9 (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:09.498975 25291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034d7d105c1b4cfbbe230cfd8011cec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.502529 25292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 161bbc3aba054bcdabb265faed76e09e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.511086 25347 tablet_bootstrap.cc:492] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:09.521693 25347 tablet_bootstrap.cc:654] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.530238 25143 tablet_service.cc:1511] Processing CreateTablet for tablet 161bbc3aba054bcdabb265faed76e09e (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:09.532531 25143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 161bbc3aba054bcdabb265faed76e09e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.542591 25222 tablet_service.cc:1511] Processing CreateTablet for tablet 161bbc3aba054bcdabb265faed76e09e (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:09.543656 25142 tablet_service.cc:1511] Processing CreateTablet for tablet 034d7d105c1b4cfbbe230cfd8011cec9 (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:09.544802 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 161bbc3aba054bcdabb265faed76e09e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.545091 25142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034d7d105c1b4cfbbe230cfd8011cec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.588274 25221 tablet_service.cc:1511] Processing CreateTablet for tablet 034d7d105c1b4cfbbe230cfd8011cec9 (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:09.590560 25141 tablet_service.cc:1511] Processing CreateTablet for tablet 58b3849e07f54c6ab1df11556d0c7212 (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:09.592101 25141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58b3849e07f54c6ab1df11556d0c7212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.589986 25221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034d7d105c1b4cfbbe230cfd8011cec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.603685 25347 tablet_bootstrap.cc:492] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:09.604348 25347 ts_tablet_manager.cc:1403] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.094s	user 0.014s	sys 0.003s
I20260709 13:09:09.606181 25223 tablet_service.cc:1511] Processing CreateTablet for tablet 58b3849e07f54c6ab1df11556d0c7212 (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:09.606678 25146 tablet_service.cc:1511] Processing CreateTablet for tablet dd30bf4abddf45c38bd41a6dbfbed3ac (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:09.609154 25146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd30bf4abddf45c38bd41a6dbfbed3ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.609174 25218 tablet_service.cc:1511] Processing CreateTablet for tablet dd30bf4abddf45c38bd41a6dbfbed3ac (DEFAULT_TABLE table=test-workload [id=d48ce7a89bdf4664b3940980fa97ce83]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:09.607731 25223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58b3849e07f54c6ab1df11556d0c7212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.608016 25348 tablet_bootstrap.cc:492] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:09.613775 25218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd30bf4abddf45c38bd41a6dbfbed3ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.620353 25347 raft_consensus.cc:359] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.621448 25347 raft_consensus.cc:385] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.621750 25347 raft_consensus.cc:740] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.622776 25347 consensus_queue.cc:260] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.655087 25348 tablet_bootstrap.cc:654] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.665692 25351 tablet_bootstrap.cc:492] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:09.667995 25347 ts_tablet_manager.cc:1434] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.063s	user 0.020s	sys 0.012s
I20260709 13:09:09.669042 25258 heartbeater.cc:499] Master 127.12.109.126:43651 was elected leader, sending a full tablet report...
I20260709 13:09:09.669353 25347 tablet_bootstrap.cc:492] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:09.674175 25348 tablet_bootstrap.cc:492] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:09.675730 25348 ts_tablet_manager.cc:1403] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.068s	user 0.009s	sys 0.006s
I20260709 13:09:09.677373 25347 tablet_bootstrap.cc:654] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.678606 25348 raft_consensus.cc:359] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.679503 25348 raft_consensus.cc:385] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.679819 25348 raft_consensus.cc:740] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.680892 25348 consensus_queue.cc:260] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.687489 25351 tablet_bootstrap.cc:654] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.694533 25348 ts_tablet_manager.cc:1434] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.011s
I20260709 13:09:09.696333 25348 tablet_bootstrap.cc:492] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:09.696369 25332 heartbeater.cc:499] Master 127.12.109.126:43651 was elected leader, sending a full tablet report...
I20260709 13:09:09.699764 25351 tablet_bootstrap.cc:492] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:09.700564 25351 ts_tablet_manager.cc:1403] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.006s
I20260709 13:09:09.704102 25348 tablet_bootstrap.cc:654] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.704612 25351 raft_consensus.cc:359] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.705572 25351 raft_consensus.cc:385] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.705969 25351 raft_consensus.cc:740] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.706915 25351 consensus_queue.cc:260] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.716288 25183 heartbeater.cc:499] Master 127.12.109.126:43651 was elected leader, sending a full tablet report...
I20260709 13:09:09.717623 25351 ts_tablet_manager.cc:1434] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.007s
I20260709 13:09:09.719085 25351 tablet_bootstrap.cc:492] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:09.727061 25351 tablet_bootstrap.cc:654] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.727833 25347 tablet_bootstrap.cc:492] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:09.727690 25357 raft_consensus.cc:493] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.728937 25347 ts_tablet_manager.cc:1403] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.060s	user 0.019s	sys 0.037s
I20260709 13:09:09.728925 25357 raft_consensus.cc:515] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.734767 25347 raft_consensus.cc:359] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.735618 25347 raft_consensus.cc:385] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.735988 25347 raft_consensus.cc:740] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.737048 25347 consensus_queue.cc:260] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.742399 25347 ts_tablet_manager.cc:1434] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.003s
I20260709 13:09:09.743554 25347 tablet_bootstrap.cc:492] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:09.751390 25347 tablet_bootstrap.cc:654] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.751622 25357 leader_election.cc:290] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:09.763000 25348 tablet_bootstrap.cc:492] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:09.763818 25348 ts_tablet_manager.cc:1403] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.068s	user 0.035s	sys 0.020s
I20260709 13:09:09.769352 25348 raft_consensus.cc:359] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.770282 25348 raft_consensus.cc:385] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.770623 25348 raft_consensus.cc:740] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.771521 25348 consensus_queue.cc:260] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.771962 25347 tablet_bootstrap.cc:492] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:09.773036 25347 ts_tablet_manager.cc:1403] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.003s
I20260709 13:09:09.775092 25348 ts_tablet_manager.cc:1434] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 13:09:09.776276 25348 tablet_bootstrap.cc:492] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:09.776448 25347 raft_consensus.cc:359] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.777292 25347 raft_consensus.cc:385] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.777614 25347 raft_consensus.cc:740] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.782972 25347 consensus_queue.cc:260] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.784314 25348 tablet_bootstrap.cc:654] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.786374 25351 tablet_bootstrap.cc:492] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:09.787133 25351 ts_tablet_manager.cc:1403] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.068s	user 0.015s	sys 0.006s
I20260709 13:09:09.790737 25351 raft_consensus.cc:359] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.791710 25347 ts_tablet_manager.cc:1434] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:09:09.791764 25351 raft_consensus.cc:385] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.792678 25351 raft_consensus.cc:740] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.793622 25347 tablet_bootstrap.cc:492] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:09.800642 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa95aeebb5246e49558118198cf4d01" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:09.793705 25351 consensus_queue.cc:260] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.811203 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa95aeebb5246e49558118198cf4d01" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
W20260709 13:09:09.812562 25121 leader_election.cc:343] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:09.813472 25124 leader_election.cc:343] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.813998 25351 ts_tablet_manager.cc:1434] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.026s	user 0.020s	sys 0.001s
I20260709 13:09:09.813959 25124 leader_election.cc:304] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:09.815349 25351 tablet_bootstrap.cc:492] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:09.815442 25357 raft_consensus.cc:2749] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:09.819669 25348 tablet_bootstrap.cc:492] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:09.820390 25348 ts_tablet_manager.cc:1403] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.021s
I20260709 13:09:09.822513 25351 tablet_bootstrap.cc:654] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.822513 25347 tablet_bootstrap.cc:654] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.824338 25348 raft_consensus.cc:359] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.825311 25348 raft_consensus.cc:385] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.825624 25348 raft_consensus.cc:740] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.826789 25348 consensus_queue.cc:260] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.833141 25348 ts_tablet_manager.cc:1434] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:09:09.834918 25348 tablet_bootstrap.cc:492] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:09.838918 25351 tablet_bootstrap.cc:492] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:09.839308 25347 tablet_bootstrap.cc:492] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:09.839737 25351 ts_tablet_manager.cc:1403] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.004s
I20260709 13:09:09.839956 25347 ts_tablet_manager.cc:1403] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.011s
I20260709 13:09:09.842586 25348 tablet_bootstrap.cc:654] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.843420 25347 raft_consensus.cc:359] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.843667 25351 raft_consensus.cc:359] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.844293 25347 raft_consensus.cc:385] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.844580 25351 raft_consensus.cc:385] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.844686 25347 raft_consensus.cc:740] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.845082 25351 raft_consensus.cc:740] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.845883 25347 consensus_queue.cc:260] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.846136 25351 consensus_queue.cc:260] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.849865 25348 tablet_bootstrap.cc:492] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:09.850508 25348 ts_tablet_manager.cc:1403] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.005s
I20260709 13:09:09.851801 25347 ts_tablet_manager.cc:1434] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:09:09.852021 25351 ts_tablet_manager.cc:1434] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:09:09.852986 25347 tablet_bootstrap.cc:492] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:09.853240 25351 tablet_bootstrap.cc:492] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:09.853814 25348 raft_consensus.cc:359] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.854733 25348 raft_consensus.cc:385] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.855067 25348 raft_consensus.cc:740] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.856034 25348 consensus_queue.cc:260] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.861304 25351 tablet_bootstrap.cc:654] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.861304 25347 tablet_bootstrap.cc:654] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.861553 25348 ts_tablet_manager.cc:1434] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:09:09.866190 25348 tablet_bootstrap.cc:492] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:09.873835 25348 tablet_bootstrap.cc:654] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.877310 25347 tablet_bootstrap.cc:492] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:09.878212 25347 ts_tablet_manager.cc:1403] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.000s
I20260709 13:09:09.882833 25347 raft_consensus.cc:359] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.883545 25347 raft_consensus.cc:385] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.883863 25347 raft_consensus.cc:740] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.885237 25348 tablet_bootstrap.cc:492] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:09.884792 25347 consensus_queue.cc:260] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.885248 25351 tablet_bootstrap.cc:492] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:09.885934 25348 ts_tablet_manager.cc:1403] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.003s
I20260709 13:09:09.886050 25351 ts_tablet_manager.cc:1403] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.009s
I20260709 13:09:09.888029 25354 raft_consensus.cc:493] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.888612 25354 raft_consensus.cc:515] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.889744 25348 raft_consensus.cc:359] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.890491 25351 raft_consensus.cc:359] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.891310 25351 raft_consensus.cc:385] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.891481 25347 ts_tablet_manager.cc:1434] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:09:09.891669 25351 raft_consensus.cc:740] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.893138 25347 tablet_bootstrap.cc:492] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:09.892725 25351 consensus_queue.cc:260] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.890898 25348 raft_consensus.cc:385] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.893690 25348 raft_consensus.cc:740] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.894774 25354 leader_election.cc:290] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:09.894738 25348 consensus_queue.cc:260] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.897908 25348 ts_tablet_manager.cc:1434] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:09:09.899049 25348 tablet_bootstrap.cc:492] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
W20260709 13:09:09.902225 25104 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:09.907781 25351 ts_tablet_manager.cc:1434] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:09:09.909036 25351 tablet_bootstrap.cc:492] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:09.913030 25347 tablet_bootstrap.cc:654] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.916123 25351 tablet_bootstrap.cc:654] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.918560 25348 tablet_bootstrap.cc:654] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.935401 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd30bf4abddf45c38bd41a6dbfbed3ac" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:09.938850 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd30bf4abddf45c38bd41a6dbfbed3ac" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:09.939759 25233 raft_consensus.cc:2468] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:09.940565 25348 tablet_bootstrap.cc:492] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:09.941184 25348 ts_tablet_manager.cc:1403] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.005s
I20260709 13:09:09.941792 25270 leader_election.cc:304] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:09.942251 25347 tablet_bootstrap.cc:492] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:09.942883 25354 raft_consensus.cc:2804] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:09:09.943019 25270 leader_election.cc:343] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.943399 25354 raft_consensus.cc:493] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.943802 25354 raft_consensus.cc:3060] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.944778 25348 raft_consensus.cc:359] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.945240 25351 tablet_bootstrap.cc:492] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:09.945364 25348 raft_consensus.cc:385] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.945746 25348 raft_consensus.cc:740] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.945938 25351 ts_tablet_manager.cc:1403] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.037s	user 0.005s	sys 0.013s
I20260709 13:09:09.946518 25348 consensus_queue.cc:260] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.949507 25348 ts_tablet_manager.cc:1434] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 13:09:09.949435 25351 raft_consensus.cc:359] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.950544 25351 raft_consensus.cc:385] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.950843 25348 tablet_bootstrap.cc:492] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:09.950920 25351 raft_consensus.cc:740] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.951989 25351 consensus_queue.cc:260] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.942914 25347 ts_tablet_manager.cc:1403] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.050s	user 0.020s	sys 0.013s
I20260709 13:09:09.954445 25351 ts_tablet_manager.cc:1434] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:09:09.955710 25351 tablet_bootstrap.cc:492] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:09.954502 25354 raft_consensus.cc:515] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.959410 25348 tablet_bootstrap.cc:654] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.962564 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd30bf4abddf45c38bd41a6dbfbed3ac" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:09.962879 25351 tablet_bootstrap.cc:654] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:09.964417 25270 leader_election.cc:343] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.966832 25354 leader_election.cc:290] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:09.958880 25347 raft_consensus.cc:359] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.967921 25347 raft_consensus.cc:385] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.968479 25347 raft_consensus.cc:740] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.967152 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd30bf4abddf45c38bd41a6dbfbed3ac" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:09.969449 25233 raft_consensus.cc:3060] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.970135 25347 consensus_queue.cc:260] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.979106 25357 raft_consensus.cc:493] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.979825 25357 raft_consensus.cc:515] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.980188 25347 ts_tablet_manager.cc:1434] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.006s
I20260709 13:09:09.981851 25347 tablet_bootstrap.cc:492] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:09.983459 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc98daf1f6a48d7a59b85bf860351e4" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:09.984232 25232 raft_consensus.cc:2468] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 0.
I20260709 13:09:09.985663 25121 leader_election.cc:304] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:09.985934 25233 raft_consensus.cc:2468] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:09.988938 25348 tablet_bootstrap.cc:492] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:09.989149 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc98daf1f6a48d7a59b85bf860351e4" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
I20260709 13:09:09.989717 25348 ts_tablet_manager.cc:1403] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.005s
I20260709 13:09:09.989887 25347 tablet_bootstrap.cc:654] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:09.990912 25124 leader_election.cc:343] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.991381 25351 tablet_bootstrap.cc:492] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:09.991626 25357 leader_election.cc:290] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:09.991997 25351 ts_tablet_manager.cc:1403] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.012s
I20260709 13:09:09.992447 25357 raft_consensus.cc:2804] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.992908 25357 raft_consensus.cc:493] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.993264 25357 raft_consensus.cc:3060] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.993412 25348 raft_consensus.cc:359] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.994486 25348 raft_consensus.cc:385] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.994860 25348 raft_consensus.cc:740] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.995596 25351 raft_consensus.cc:359] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.996449 25351 raft_consensus.cc:385] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.995910 25348 consensus_queue.cc:260] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.996862 25351 raft_consensus.cc:740] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.997779 25351 consensus_queue.cc:260] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:09.999447 25348 ts_tablet_manager.cc:1434] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:09:10.001212 25348 tablet_bootstrap.cc:492] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:10.007014 25270 leader_election.cc:304] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: bca68f82aedd42f5bf4c4fdfcb0c72db
I20260709 13:09:10.008211 25350 raft_consensus.cc:493] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.008764 25354 raft_consensus.cc:2804] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.008869 25350 raft_consensus.cc:515] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.009455 25354 raft_consensus.cc:697] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Leader. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:10.010368 25348 tablet_bootstrap.cc:654] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.010475 25354 consensus_queue.cc:237] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.015735 25350 leader_election.cc:290] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:10.023077 25377 raft_consensus.cc:493] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.023895 25377 raft_consensus.cc:515] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.029896 25357 raft_consensus.cc:515] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.033783 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc98daf1f6a48d7a59b85bf860351e4" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:10.034545 25233 raft_consensus.cc:3060] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.039505 25350 raft_consensus.cc:493] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.040190 25350 raft_consensus.cc:515] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.043128 25350 leader_election.cc:290] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:10.044175 25233 raft_consensus.cc:2468] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 1.
I20260709 13:09:10.045730 25121 leader_election.cc:304] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:10.050853 25384 raft_consensus.cc:2804] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.058744 25357 leader_election.cc:290] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 election: Requested vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:10.060096 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc98daf1f6a48d7a59b85bf860351e4" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:10.060597 25351 ts_tablet_manager.cc:1434] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.068s	user 0.018s	sys 0.005s
I20260709 13:09:10.062999 25351 tablet_bootstrap.cc:492] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:10.064085 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e64433f5d83d40a2be74255a152a160a" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
I20260709 13:09:10.065032 25306 raft_consensus.cc:2468] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 0.
I20260709 13:09:10.065196 25348 tablet_bootstrap.cc:492] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:10.065891 25348 ts_tablet_manager.cc:1403] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.065s	user 0.022s	sys 0.025s
I20260709 13:09:10.066398 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
W20260709 13:09:10.066761 25124 leader_election.cc:343] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:10.068025 25121 leader_election.cc:343] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.068264 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e80f79ae7547a7a8e6bd6fd8d95dcb" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
I20260709 13:09:10.064126 25377 leader_election.cc:290] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:10.069427 25348 raft_consensus.cc:359] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.070238 25199 leader_election.cc:304] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:10.070577 25348 raft_consensus.cc:385] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.071058 25348 raft_consensus.cc:740] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.071431 25350 raft_consensus.cc:2804] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.072557 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
I20260709 13:09:10.073874 25350 raft_consensus.cc:493] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.074136 25351 tablet_bootstrap.cc:654] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.074273 25306 raft_consensus.cc:2468] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 0.
I20260709 13:09:10.074414 25350 raft_consensus.cc:3060] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:10.073071 25199 leader_election.cc:343] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.072055 25348 consensus_queue.cc:260] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.076920 25354 raft_consensus.cc:493] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.077521 25354 raft_consensus.cc:515] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.078310 25374 raft_consensus.cc:493] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.078804 25374 raft_consensus.cc:515] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.079370 25348 ts_tablet_manager.cc:1434] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:09:10.080756 25348 tablet_bootstrap.cc:492] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:10.080931 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f79486558b854673aa41d16a1494868e" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:10.081230 25374 leader_election.cc:290] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
W20260709 13:09:10.082716 25270 leader_election.cc:343] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.082882 25124 leader_election.cc:304] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:10.083086 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc98daf1f6a48d7a59b85bf860351e4" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:10.084088 25377 raft_consensus.cc:2804] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.084484 25377 raft_consensus.cc:493] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.084861 25377 raft_consensus.cc:3060] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.086550 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f79486558b854673aa41d16a1494868e" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:10.090224 25354 leader_election.cc:290] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:10.090338 25156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e80f79ae7547a7a8e6bd6fd8d95dcb" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:10.091084 25232 raft_consensus.cc:2468] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:10.091369 25156 raft_consensus.cc:2468] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 0.
I20260709 13:09:10.089802 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e64433f5d83d40a2be74255a152a160a" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:10.092657 25157 raft_consensus.cc:2468] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 0.
I20260709 13:09:10.093036 25348 tablet_bootstrap.cc:654] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.092948 25196 leader_election.cc:304] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: ac08c897ed7e478bbd7852f67f11caff
I20260709 13:09:10.086582 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc98daf1f6a48d7a59b85bf860351e4" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:10.094969 25377 raft_consensus.cc:515] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.095543 25270 leader_election.cc:304] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: bca68f82aedd42f5bf4c4fdfcb0c72db
I20260709 13:09:10.095769 25233 raft_consensus.cc:2393] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ac08c897ed7e478bbd7852f67f11caff in current term 1: Already voted for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in this term.
I20260709 13:09:10.098733 25347 tablet_bootstrap.cc:492] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:10.099409 25347 ts_tablet_manager.cc:1403] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.118s	user 0.017s	sys 0.025s
I20260709 13:09:10.086031 25056 catalog_manager.cc:5697] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff reported cstate change: term changed from 0 to 1, leader changed from <none> to ac08c897ed7e478bbd7852f67f11caff (127.12.109.67). New cstate: current_term: 1 leader_uuid: "ac08c897ed7e478bbd7852f67f11caff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:10.104074 25354 raft_consensus.cc:2804] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.104588 25354 raft_consensus.cc:493] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.105005 25354 raft_consensus.cc:3060] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.107141 25350 raft_consensus.cc:515] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.109666 25350 leader_election.cc:290] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:10.109848 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:10.109851 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:10.110102 25377 leader_election.cc:290] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 election: Requested vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:10.114459 25384 raft_consensus.cc:697] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Becoming Leader. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:10.116387 25374 raft_consensus.cc:493] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.118094 25350 raft_consensus.cc:2804] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.119113 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e64433f5d83d40a2be74255a152a160a" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:10.122188 25306 raft_consensus.cc:3060] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.122618 25374 raft_consensus.cc:515] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
W20260709 13:09:10.125144 25121 leader_election.cc:343] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.125415 25384 consensus_queue.cc:237] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.123354 25350 raft_consensus.cc:493] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.126690 25350 raft_consensus.cc:3060] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.124548 25307 raft_consensus.cc:3060] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.129431 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa95aeebb5246e49558118198cf4d01" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:10.130385 25157 raft_consensus.cc:2468] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:10.131923 25270 leader_election.cc:304] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: 
I20260709 13:09:10.132787 25348 tablet_bootstrap.cc:492] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:10.132768 25351 tablet_bootstrap.cc:492] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:10.133589 25348 ts_tablet_manager.cc:1403] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.053s	user 0.011s	sys 0.006s
I20260709 13:09:10.133998 25351 ts_tablet_manager.cc:1403] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.071s	user 0.010s	sys 0.010s
I20260709 13:09:10.133872 25347 raft_consensus.cc:359] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.134703 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e64433f5d83d40a2be74255a152a160a" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:10.135443 25157 raft_consensus.cc:3060] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.134928 25347 raft_consensus.cc:385] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.137025 25348 raft_consensus.cc:359] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.137586 25347 raft_consensus.cc:740] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.137801 25348 raft_consensus.cc:385] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.138221 25348 raft_consensus.cc:740] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.138723 25347 consensus_queue.cc:260] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.139247 25348 consensus_queue.cc:260] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.138962 25351 raft_consensus.cc:359] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.146188 25347 ts_tablet_manager.cc:1434] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.004s
I20260709 13:09:10.140527 25351 raft_consensus.cc:385] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.147043 25351 raft_consensus.cc:740] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.147586 25347 tablet_bootstrap.cc:492] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:10.148066 25351 consensus_queue.cc:260] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.161342 25347 tablet_bootstrap.cc:654] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.162781 25374 leader_election.cc:290] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:10.163756 25374 raft_consensus.cc:2804] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.164237 25374 raft_consensus.cc:493] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.164695 25374 raft_consensus.cc:3060] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.172253 25348 ts_tablet_manager.cc:1434] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 13:09:10.174558 25391 raft_consensus.cc:493] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.175007 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa95aeebb5246e49558118198cf4d01" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:10.175315 25391 raft_consensus.cc:515] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
W20260709 13:09:10.176556 25270 leader_election.cc:343] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.173302 25354 raft_consensus.cc:515] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.179277 25391 leader_election.cc:290] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:10.180441 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034d7d105c1b4cfbbe230cfd8011cec9" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:10.180785 25354 leader_election.cc:290] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
W20260709 13:09:10.182350 25270 leader_election.cc:343] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.182920 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f79486558b854673aa41d16a1494868e" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:10.183364 25351 ts_tablet_manager.cc:1434] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.049s	user 0.003s	sys 0.005s
I20260709 13:09:10.183533 25233 raft_consensus.cc:3060] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.184924 25351 tablet_bootstrap.cc:492] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:10.185706 25156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034d7d105c1b4cfbbe230cfd8011cec9" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:10.184791 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f79486558b854673aa41d16a1494868e" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:10.188710 25270 leader_election.cc:304] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff; no voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630
W20260709 13:09:10.190467 25270 leader_election.cc:343] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.193357 25233 raft_consensus.cc:2468] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:10.191221 25270 leader_election.cc:304] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff; no voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:10.197502 25354 raft_consensus.cc:3060] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.201046 25391 raft_consensus.cc:2749] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:09:10.203841 25270 leader_election.cc:343] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.190846 25350 raft_consensus.cc:515] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.209513 25350 leader_election.cc:290] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:10.210142 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e80f79ae7547a7a8e6bd6fd8d95dcb" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:10.211057 25158 raft_consensus.cc:3060] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.212004 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e80f79ae7547a7a8e6bd6fd8d95dcb" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:10.212774 25305 raft_consensus.cc:3060] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.213004 25351 tablet_bootstrap.cc:654] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.215734 25391 raft_consensus.cc:493] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.215811 25306 raft_consensus.cc:2468] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 1.
I20260709 13:09:10.216418 25391 raft_consensus.cc:515] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.218956 25124 leader_election.cc:304] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:10.220315 25384 raft_consensus.cc:2804] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.204959 25270 leader_election.cc:304] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: bca68f82aedd42f5bf4c4fdfcb0c72db
I20260709 13:09:10.221118 25384 raft_consensus.cc:697] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Becoming Leader. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:10.222559 25384 consensus_queue.cc:237] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.224690 25390 raft_consensus.cc:2804] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.220681 25158 raft_consensus.cc:2468] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 1.
I20260709 13:09:10.231745 25196 leader_election.cc:304] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:10.232759 25350 raft_consensus.cc:2804] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.223160 25157 raft_consensus.cc:2468] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 1.
I20260709 13:09:10.233767 25391 leader_election.cc:290] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:10.234859 25196 leader_election.cc:304] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:10.235981 25386 raft_consensus.cc:2804] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.236478 25350 raft_consensus.cc:697] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Becoming Leader. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:10.237083 25307 raft_consensus.cc:2468] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 1.
I20260709 13:09:10.261037 25374 raft_consensus.cc:515] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.272863 25374 leader_election.cc:290] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:10.275527 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa95aeebb5246e49558118198cf4d01" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:10.276057 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa95aeebb5246e49558118198cf4d01" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:10.276265 25158 raft_consensus.cc:3060] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.280328 25347 tablet_bootstrap.cc:492] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:10.280961 25347 ts_tablet_manager.cc:1403] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.134s	user 0.014s	sys 0.007s
I20260709 13:09:10.277073 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b3849e07f54c6ab1df11556d0c7212" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:10.282727 25233 raft_consensus.cc:2468] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:10.282859 25354 raft_consensus.cc:2749] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:10.266395 25390 raft_consensus.cc:697] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Leader. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:10.295635 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b3849e07f54c6ab1df11556d0c7212" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:10.296060 25390 consensus_queue.cc:237] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.296721 25157 raft_consensus.cc:2468] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:10.297531 25350 consensus_queue.cc:237] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.298633 25270 leader_election.cc:304] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: 
I20260709 13:09:10.300768 25354 raft_consensus.cc:2804] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.301242 25354 raft_consensus.cc:493] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.301710 25354 raft_consensus.cc:3060] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.312284 25158 raft_consensus.cc:2468] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:10.296726 25386 raft_consensus.cc:697] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Becoming Leader. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:10.316020 25351 tablet_bootstrap.cc:492] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:10.316655 25351 ts_tablet_manager.cc:1403] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.132s	user 0.022s	sys 0.007s
W20260709 13:09:10.320271 25270 leader_election.cc:343] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.321727 25347 raft_consensus.cc:359] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.322562 25347 raft_consensus.cc:385] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.322604 25386 consensus_queue.cc:237] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.322945 25347 raft_consensus.cc:740] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.322944 25270 leader_election.cc:304] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:10.324290 25347 consensus_queue.cc:260] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.325553 25394 raft_consensus.cc:2804] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.326287 25394 raft_consensus.cc:697] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Leader. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:10.327486 25394 consensus_queue.cc:237] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.337123 25305 raft_consensus.cc:2468] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 1.
I20260709 13:09:10.339088 25354 raft_consensus.cc:515] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.340478 25054 catalog_manager.cc:5697] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:10.341758 25354 leader_election.cc:290] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:10.343714 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b3849e07f54c6ab1df11556d0c7212" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:10.344435 25233 raft_consensus.cc:3060] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.344741 25351 raft_consensus.cc:359] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.345551 25351 raft_consensus.cc:385] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.345851 25351 raft_consensus.cc:740] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.346590 25351 consensus_queue.cc:260] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.346662 25055 catalog_manager.cc:5697] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db reported cstate change: term changed from 0 to 1, leader changed from <none> to bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65). New cstate: current_term: 1 leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:10.348485 25351 ts_tablet_manager.cc:1434] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.000s
I20260709 13:09:10.349593 25351 tablet_bootstrap.cc:492] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:10.355556 25347 ts_tablet_manager.cc:1434] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.074s	user 0.010s	sys 0.007s
I20260709 13:09:10.356787 25347 tablet_bootstrap.cc:492] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:10.363250 25347 tablet_bootstrap.cc:654] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.365106 25233 raft_consensus.cc:2468] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:10.371060 25351 tablet_bootstrap.cc:654] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.373209 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b3849e07f54c6ab1df11556d0c7212" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:10.374276 25158 raft_consensus.cc:3060] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.375761 25270 leader_election.cc:304] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:10.383277 25347 tablet_bootstrap.cc:492] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:10.384121 25347 ts_tablet_manager.cc:1403] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.028s	user 0.022s	sys 0.003s
I20260709 13:09:10.383342 25390 raft_consensus.cc:2804] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.386684 25390 raft_consensus.cc:697] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Leader. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:10.388381 25390 consensus_queue.cc:237] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.400267 25347 raft_consensus.cc:359] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.401639 25347 raft_consensus.cc:385] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.401952 25347 raft_consensus.cc:740] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.403029 25347 consensus_queue.cc:260] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.403729 25056 catalog_manager.cc:5697] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff reported cstate change: term changed from 0 to 1, leader changed from <none> to ac08c897ed7e478bbd7852f67f11caff (127.12.109.67). New cstate: current_term: 1 leader_uuid: "ac08c897ed7e478bbd7852f67f11caff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:10.405232 25347 ts_tablet_manager.cc:1434] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 13:09:10.406476 25056 catalog_manager.cc:5697] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff reported cstate change: term changed from 0 to 1, leader changed from <none> to ac08c897ed7e478bbd7852f67f11caff (127.12.109.67). New cstate: current_term: 1 leader_uuid: "ac08c897ed7e478bbd7852f67f11caff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:10.416849 25351 tablet_bootstrap.cc:492] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:10.416806 25158 raft_consensus.cc:2468] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:10.417718 25351 ts_tablet_manager.cc:1403] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.068s	user 0.022s	sys 0.007s
I20260709 13:09:10.420969 25351 raft_consensus.cc:359] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.421685 25351 raft_consensus.cc:385] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.421952 25351 raft_consensus.cc:740] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.422791 25351 consensus_queue.cc:260] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.425110 25351 ts_tablet_manager.cc:1434] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:09:10.428920 25055 catalog_manager.cc:5697] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db reported cstate change: term changed from 0 to 1, leader changed from <none> to bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65). New cstate: current_term: 1 leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:10.429020 25054 catalog_manager.cc:5697] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:10.442031 25384 raft_consensus.cc:493] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.442926 25384 raft_consensus.cc:515] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.442155 25056 catalog_manager.cc:5697] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff reported cstate change: term changed from 0 to 1, leader changed from <none> to ac08c897ed7e478bbd7852f67f11caff (127.12.109.67). New cstate: current_term: 1 leader_uuid: "ac08c897ed7e478bbd7852f67f11caff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:10.446173 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f79486558b854673aa41d16a1494868e" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:10.447033 25233 raft_consensus.cc:2393] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bca68f82aedd42f5bf4c4fdfcb0c72db in current term 1: Already voted for candidate ac08c897ed7e478bbd7852f67f11caff in this term.
I20260709 13:09:10.448398 25384 leader_election.cc:290] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:10.449364 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f79486558b854673aa41d16a1494868e" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
I20260709 13:09:10.451051 25124 leader_election.cc:304] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:10.451892 25384 raft_consensus.cc:3060] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.458990 25384 raft_consensus.cc:2749] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:10.482775 25384 raft_consensus.cc:493] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.483433 25384 raft_consensus.cc:515] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.485538 25384 leader_election.cc:290] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:10.486858 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd30bf4abddf45c38bd41a6dbfbed3ac" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:10.487021 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd30bf4abddf45c38bd41a6dbfbed3ac" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
I20260709 13:09:10.487965 25233 raft_consensus.cc:2393] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bca68f82aedd42f5bf4c4fdfcb0c72db in current term 1: Already voted for candidate ac08c897ed7e478bbd7852f67f11caff in this term.
I20260709 13:09:10.489084 25121 leader_election.cc:304] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:10.489895 25384 raft_consensus.cc:3060] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.496088 25384 raft_consensus.cc:2749] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:10.537721 25390 consensus_queue.cc:1048] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.551184 25390 consensus_queue.cc:1048] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.586022 25386 raft_consensus.cc:493] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.586802 25386 raft_consensus.cc:515] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.589026 25386 leader_election.cc:290] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:10.590097 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:10.590185 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
I20260709 13:09:10.591212 25305 raft_consensus.cc:2393] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c3395c72252c4138b3d77bf5f0d7d630 in current term 1: Already voted for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in this term.
I20260709 13:09:10.592530 25199 leader_election.cc:304] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3395c72252c4138b3d77bf5f0d7d630; no voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db
I20260709 13:09:10.593331 25386 raft_consensus.cc:3060] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.594558 25384 consensus_queue.cc:1048] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.601218 25386 raft_consensus.cc:2749] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:10.608034 25385 consensus_queue.cc:1048] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.640259 25386 raft_consensus.cc:493] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.640826 25386 raft_consensus.cc:515] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.642802 25386 leader_election.cc:290] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:10.643821 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034d7d105c1b4cfbbe230cfd8011cec9" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:10.644018 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034d7d105c1b4cfbbe230cfd8011cec9" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
I20260709 13:09:10.644580 25158 raft_consensus.cc:2468] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 0.
I20260709 13:09:10.644706 25305 raft_consensus.cc:2468] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 0.
I20260709 13:09:10.645664 25196 leader_election.cc:304] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:10.646528 25386 raft_consensus.cc:2804] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.646843 25386 raft_consensus.cc:493] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.647123 25386 raft_consensus.cc:3060] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.653044 25386 raft_consensus.cc:515] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.655097 25386 leader_election.cc:290] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:10.655916 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034d7d105c1b4cfbbe230cfd8011cec9" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:10.656128 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034d7d105c1b4cfbbe230cfd8011cec9" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:10.656633 25158 raft_consensus.cc:3060] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.656734 25305 raft_consensus.cc:3060] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.662740 25158 raft_consensus.cc:2468] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 1.
I20260709 13:09:10.662779 25305 raft_consensus.cc:2468] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 1.
I20260709 13:09:10.664134 25199 leader_election.cc:304] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:10.665048 25386 raft_consensus.cc:2804] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.665083 25385 consensus_queue.cc:1048] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.665594 25386 raft_consensus.cc:697] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Becoming Leader. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:10.666579 25386 consensus_queue.cc:237] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:10.676124 25056 catalog_manager.cc:5697] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:10.683653 25377 consensus_queue.cc:1048] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.717711 25394 consensus_queue.cc:1048] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.729606 25390 consensus_queue.cc:1048] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.741958 25386 consensus_queue.cc:1048] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.754547 25350 consensus_queue.cc:1048] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.773451 25233 raft_consensus.cc:3060] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.781153 25390 consensus_queue.cc:1048] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.792757 25354 consensus_queue.cc:1048] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.826716 25350 consensus_queue.cc:1048] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.839115 25398 consensus_queue.cc:1048] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.921854 25233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e64433f5d83d40a2be74255a152a160a"
dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
mode: GRACEFUL
new_leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:52808
I20260709 13:09:10.922578 25233 raft_consensus.cc:606] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Received request to transfer leadership to TS bca68f82aedd42f5bf4c4fdfcb0c72db
I20260709 13:09:10.931170 25233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33e80f79ae7547a7a8e6bd6fd8d95dcb"
dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
mode: GRACEFUL
new_leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:52808
I20260709 13:09:10.931751 25233 raft_consensus.cc:606] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Received request to transfer leadership to TS bca68f82aedd42f5bf4c4fdfcb0c72db
I20260709 13:09:10.946553 25307 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9aa95aeebb5246e49558118198cf4d01"
dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
mode: GRACEFUL
new_leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:58264
I20260709 13:09:10.947180 25307 raft_consensus.cc:606] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Received request to transfer leadership to TS bca68f82aedd42f5bf4c4fdfcb0c72db
I20260709 13:09:10.952713 25307 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f79486558b854673aa41d16a1494868e"
dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
mode: GRACEFUL
new_leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:58264
I20260709 13:09:10.953286 25307 raft_consensus.cc:606] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Received request to transfer leadership to TS c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:10.967437 25158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e"
dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
mode: GRACEFUL
new_leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:51882
I20260709 13:09:10.968062 25158 raft_consensus.cc:606] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Received request to transfer leadership to TS c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:10.976708 25307 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58b3849e07f54c6ab1df11556d0c7212"
dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
mode: GRACEFUL
new_leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:58264
I20260709 13:09:10.977324 25307 raft_consensus.cc:606] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Received request to transfer leadership to TS c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:10.989077 25354 consensus_queue.cc:1048] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.996851 25394 consensus_queue.cc:1048] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:11.008201 25354 raft_consensus.cc:993] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff: : Instructing follower c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:11.008809 25390 raft_consensus.cc:1081] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Signalling peer c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:11.010491 25233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58b3849e07f54c6ab1df11556d0c7212"
dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:52796
I20260709 13:09:11.011024 25233 raft_consensus.cc:493] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:11.011432 25233 raft_consensus.cc:3060] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.018416 25233 raft_consensus.cc:515] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.020780 25233 leader_election.cc:290] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:11.021106 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b3849e07f54c6ab1df11556d0c7212" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:11.021608 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b3849e07f54c6ab1df11556d0c7212" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:11.021888 25158 raft_consensus.cc:3060] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.022369 25307 raft_consensus.cc:3055] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:11.022723 25307 raft_consensus.cc:740] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:11.023615 25307 consensus_queue.cc:260] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.024998 25307 raft_consensus.cc:3060] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.030027 25158 raft_consensus.cc:2468] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:11.031441 25196 leader_election.cc:304] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:11.032209 25398 raft_consensus.cc:2804] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:11.032721 25398 raft_consensus.cc:697] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Becoming Leader. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:11.032871 25307 raft_consensus.cc:2468] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:11.033681 25398 consensus_queue.cc:237] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.041105 25056 catalog_manager.cc:5697] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 reported cstate change: term changed from 1 to 2, leader changed from ac08c897ed7e478bbd7852f67f11caff (127.12.109.67) to c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:11.099186 25398 consensus_queue.cc:1048] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:11.110666 25398 consensus_queue.cc:1048] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:11.211966 25384 raft_consensus.cc:993] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db: : Instructing follower c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:11.212600 25384 raft_consensus.cc:1081] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Signalling peer c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:11.215027 25233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e"
dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:52790
I20260709 13:09:11.215740 25233 raft_consensus.cc:493] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:11.216142 25233 raft_consensus.cc:3060] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.217532 25394 raft_consensus.cc:993] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff: : Instructing follower bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:11.218145 25390 raft_consensus.cc:1081] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Signalling peer bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:11.219939 25158 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9aa95aeebb5246e49558118198cf4d01"
dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:51860
I20260709 13:09:11.220575 25158 raft_consensus.cc:493] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:11.220999 25158 raft_consensus.cc:3060] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.224850 25233 raft_consensus.cc:515] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.227425 25233 leader_election.cc:290] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:11.228161 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:11.228662 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:11.228904 25157 raft_consensus.cc:3055] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:11.229346 25157 raft_consensus.cc:740] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:11.229346 25307 raft_consensus.cc:3060] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.229931 25158 raft_consensus.cc:515] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.230263 25157 consensus_queue.cc:260] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.231540 25157 raft_consensus.cc:3060] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.232270 25158 leader_election.cc:290] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Requested vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:11.233445 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa95aeebb5246e49558118198cf4d01" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:11.233445 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa95aeebb5246e49558118198cf4d01" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:11.234400 25305 raft_consensus.cc:3055] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:11.234488 25233 raft_consensus.cc:3060] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.234828 25305 raft_consensus.cc:740] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:11.235760 25305 consensus_queue.cc:260] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.236799 25307 raft_consensus.cc:2468] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:11.237128 25305 raft_consensus.cc:3060] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.238227 25199 leader_election.cc:304] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:11.239261 25398 raft_consensus.cc:2804] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:11.239758 25398 raft_consensus.cc:697] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Becoming Leader. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:11.240628 25398 consensus_queue.cc:237] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.242362 25233 raft_consensus.cc:2468] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:11.243961 25121 leader_election.cc:304] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:11.245019 25384 raft_consensus.cc:2804] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:11.245549 25384 raft_consensus.cc:697] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Becoming Leader. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:11.246560 25305 raft_consensus.cc:2468] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:11.246404 25384 consensus_queue.cc:237] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.246384 25157 raft_consensus.cc:2468] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:11.250720 25056 catalog_manager.cc:5697] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 reported cstate change: term changed from 1 to 2, leader changed from bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65) to c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:11.254725 25386 raft_consensus.cc:993] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630: : Instructing follower bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:11.255398 25386 raft_consensus.cc:1081] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Signalling peer bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:11.254912 25055 catalog_manager.cc:5697] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db reported cstate change: term changed from 1 to 2, leader changed from ac08c897ed7e478bbd7852f67f11caff (127.12.109.67) to bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65). New cstate: current_term: 2 leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:11.257185 25157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33e80f79ae7547a7a8e6bd6fd8d95dcb"
dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:51874
I20260709 13:09:11.257710 25157 raft_consensus.cc:493] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:11.258111 25157 raft_consensus.cc:3060] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.265437 25157 raft_consensus.cc:515] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.268738 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e80f79ae7547a7a8e6bd6fd8d95dcb" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:11.269225 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e80f79ae7547a7a8e6bd6fd8d95dcb" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:11.269490 25390 raft_consensus.cc:993] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff: : Instructing follower c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:11.269593 25305 raft_consensus.cc:3060] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.269966 25233 raft_consensus.cc:3055] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:11.270191 25390 raft_consensus.cc:1081] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Signalling peer c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:11.270430 25233 raft_consensus.cc:740] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:11.271479 25233 consensus_queue.cc:260] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.272373 25232 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f79486558b854673aa41d16a1494868e"
dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:52796
I20260709 13:09:11.272656 25233 raft_consensus.cc:3060] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.273016 25232 raft_consensus.cc:493] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:11.273311 25232 raft_consensus.cc:3060] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.268132 25157 leader_election.cc:290] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Requested vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:11.279021 25305 raft_consensus.cc:2468] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:11.279878 25232 raft_consensus.cc:515] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.280470 25124 leader_election.cc:304] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: 
I20260709 13:09:11.281121 25233 raft_consensus.cc:2468] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:11.281525 25384 raft_consensus.cc:2804] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:11.282125 25384 raft_consensus.cc:697] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Becoming Leader. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:11.282714 25232 leader_election.cc:290] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:11.283438 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f79486558b854673aa41d16a1494868e" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:11.284147 25305 raft_consensus.cc:3055] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:11.283600 25384 consensus_queue.cc:237] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.284472 25305 raft_consensus.cc:740] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:11.284754 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f79486558b854673aa41d16a1494868e" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:11.285377 25157 raft_consensus.cc:3060] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.285503 25305 consensus_queue.cc:260] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.287009 25305 raft_consensus.cc:3060] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.292757 25157 raft_consensus.cc:2468] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:11.293819 25305 raft_consensus.cc:2468] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:11.293037 25055 catalog_manager.cc:5697] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db reported cstate change: term changed from 1 to 2, leader changed from c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66) to bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65). New cstate: current_term: 2 leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:11.294157 25196 leader_election.cc:304] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:11.295164 25386 raft_consensus.cc:2804] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:11.295650 25386 raft_consensus.cc:697] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Becoming Leader. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:11.296528 25386 consensus_queue.cc:237] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.303921 25056 catalog_manager.cc:5697] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 reported cstate change: term changed from 1 to 2, leader changed from ac08c897ed7e478bbd7852f67f11caff (127.12.109.67) to c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:11.306376 25398 raft_consensus.cc:993] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630: : Instructing follower bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:11.307055 25386 raft_consensus.cc:1081] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Signalling peer bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:11.308804 25157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e64433f5d83d40a2be74255a152a160a"
dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:51874
I20260709 13:09:11.309350 25157 raft_consensus.cc:493] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:11.309684 25157 raft_consensus.cc:3060] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.316651 25157 raft_consensus.cc:515] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.319375 25157 leader_election.cc:290] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Requested vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:11.320720 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e64433f5d83d40a2be74255a152a160a" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:11.321494 25307 raft_consensus.cc:3060] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.323571 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e64433f5d83d40a2be74255a152a160a" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:11.324306 25232 raft_consensus.cc:3055] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:11.324643 25232 raft_consensus.cc:740] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:11.325491 25232 consensus_queue.cc:260] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.326887 25232 raft_consensus.cc:3060] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:11.328816 25307 raft_consensus.cc:2468] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:11.330018 25124 leader_election.cc:304] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: 
I20260709 13:09:11.330866 25384 raft_consensus.cc:2804] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:11.331396 25384 raft_consensus.cc:697] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Becoming Leader. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:11.332319 25384 consensus_queue.cc:237] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:11.335078 25232 raft_consensus.cc:2468] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:11.340554 25056 catalog_manager.cc:5697] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db reported cstate change: term changed from 1 to 2, leader changed from c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66) to bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65). New cstate: current_term: 2 leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:11.542656 25157 raft_consensus.cc:1275] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:11.544240 25386 consensus_queue.cc:1048] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:11.552646 25307 raft_consensus.cc:1275] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:11.554399 25398 consensus_queue.cc:1048] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:11.698957 25305 raft_consensus.cc:1275] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:11.700398 25350 consensus_queue.cc:1048] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:11.708508 25157 raft_consensus.cc:1275] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:11.710919 25350 consensus_queue.cc:1048] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:11.736972 25232 raft_consensus.cc:1275] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:11.738701 25384 consensus_queue.cc:1048] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:11.749516 25305 raft_consensus.cc:1275] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:11.751539 25435 consensus_queue.cc:1048] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:11.761217 25157 raft_consensus.cc:1275] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:11.762605 25350 consensus_queue.cc:1048] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:11.772809 25305 raft_consensus.cc:1275] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:11.775043 25350 consensus_queue.cc:1048] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:11.795032 25232 raft_consensus.cc:1275] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:11.796471 25435 consensus_queue.cc:1048] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:11.805962 25305 raft_consensus.cc:1275] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:11.808053 25436 consensus_queue.cc:1048] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:11.855288 25305 raft_consensus.cc:1275] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:11.856758 25436 consensus_queue.cc:1048] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:11.866955 25232 raft_consensus.cc:1275] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:11.868552 25385 consensus_queue.cc:1048] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:14.017572 25056 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55842:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:14.020751 25056 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:14.079777 25142 tablet_service.cc:1511] Processing CreateTablet for tablet 0ff976e9e6d84f4e9708af71a410e275 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:14.081387 25221 tablet_service.cc:1511] Processing CreateTablet for tablet 0ff976e9e6d84f4e9708af71a410e275 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:14.082124 25142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff976e9e6d84f4e9708af71a410e275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.084779 25221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff976e9e6d84f4e9708af71a410e275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.084717 25144 tablet_service.cc:1511] Processing CreateTablet for tablet 7dbe909446ac4c1b8e329828cc311a52 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:14.084726 25295 tablet_service.cc:1511] Processing CreateTablet for tablet 0ff976e9e6d84f4e9708af71a410e275 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:14.086647 25144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dbe909446ac4c1b8e329828cc311a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.087883 25295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff976e9e6d84f4e9708af71a410e275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.088565 25296 tablet_service.cc:1511] Processing CreateTablet for tablet 7dbe909446ac4c1b8e329828cc311a52 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:14.092015 25145 tablet_service.cc:1511] Processing CreateTablet for tablet 76ad5adfa4824e669ae7414ebd491691 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:14.092594 25294 tablet_service.cc:1511] Processing CreateTablet for tablet 76ad5adfa4824e669ae7414ebd491691 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:14.093825 25145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76ad5adfa4824e669ae7414ebd491691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.096807 25290 tablet_service.cc:1511] Processing CreateTablet for tablet 8a417912724d4279b7b1d1e1ddfcec1c (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:14.097312 25222 tablet_service.cc:1511] Processing CreateTablet for tablet 76ad5adfa4824e669ae7414ebd491691 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:14.107549 25291 tablet_service.cc:1511] Processing CreateTablet for tablet 304954defc094f77bd6356c5b30b2d56 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:14.108491 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76ad5adfa4824e669ae7414ebd491691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.109027 25218 tablet_service.cc:1511] Processing CreateTablet for tablet 25090e4349ea43b5a76c9295ae73eaab (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:14.100397 25220 tablet_service.cc:1511] Processing CreateTablet for tablet 7dbe909446ac4c1b8e329828cc311a52 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:14.100836 25141 tablet_service.cc:1511] Processing CreateTablet for tablet 8a417912724d4279b7b1d1e1ddfcec1c (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:14.112839 25289 tablet_service.cc:1511] Processing CreateTablet for tablet 97e2ed0f2196447f997303c0889538af (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:14.114619 25292 tablet_service.cc:1511] Processing CreateTablet for tablet 7f19f2e70f5e4b189c52a968eeb7c61b (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:14.115487 25148 tablet_service.cc:1511] Processing CreateTablet for tablet 304954defc094f77bd6356c5b30b2d56 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:14.102592 25297 tablet_service.cc:1511] Processing CreateTablet for tablet 25090e4349ea43b5a76c9295ae73eaab (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:14.103322 25223 tablet_service.cc:1511] Processing CreateTablet for tablet 8a417912724d4279b7b1d1e1ddfcec1c (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:14.104503 25146 tablet_service.cc:1511] Processing CreateTablet for tablet 25090e4349ea43b5a76c9295ae73eaab (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:14.119156 25146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25090e4349ea43b5a76c9295ae73eaab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.118863 25293 tablet_service.cc:1511] Processing CreateTablet for tablet 34d2357c56f641e48b3aba13247ec1d1 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:14.122423 25141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a417912724d4279b7b1d1e1ddfcec1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.123214 25148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304954defc094f77bd6356c5b30b2d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.106348 25296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dbe909446ac4c1b8e329828cc311a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.124521 25219 tablet_service.cc:1511] Processing CreateTablet for tablet 304954defc094f77bd6356c5b30b2d56 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:14.126221 25290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a417912724d4279b7b1d1e1ddfcec1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.127317 25294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76ad5adfa4824e669ae7414ebd491691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.128832 25292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f19f2e70f5e4b189c52a968eeb7c61b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.130347 25293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d2357c56f641e48b3aba13247ec1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.132767 25289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97e2ed0f2196447f997303c0889538af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.137115 25297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25090e4349ea43b5a76c9295ae73eaab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.142505 25291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304954defc094f77bd6356c5b30b2d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.150620 25220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dbe909446ac4c1b8e329828cc311a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.151748 25218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25090e4349ea43b5a76c9295ae73eaab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.152540 25223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a417912724d4279b7b1d1e1ddfcec1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.153174 25219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304954defc094f77bd6356c5b30b2d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.164856 25147 tablet_service.cc:1511] Processing CreateTablet for tablet 97e2ed0f2196447f997303c0889538af (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:14.166684 25147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97e2ed0f2196447f997303c0889538af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.180100 25217 tablet_service.cc:1511] Processing CreateTablet for tablet 97e2ed0f2196447f997303c0889538af (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:14.181931 25217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97e2ed0f2196447f997303c0889538af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.207827 25143 tablet_service.cc:1511] Processing CreateTablet for tablet 7f19f2e70f5e4b189c52a968eeb7c61b (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:14.208844 25216 tablet_service.cc:1511] Processing CreateTablet for tablet 7f19f2e70f5e4b189c52a968eeb7c61b (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:14.209591 25143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f19f2e70f5e4b189c52a968eeb7c61b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.210530 25216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f19f2e70f5e4b189c52a968eeb7c61b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.242007 25215 tablet_service.cc:1511] Processing CreateTablet for tablet 34d2357c56f641e48b3aba13247ec1d1 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:14.248421 25215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d2357c56f641e48b3aba13247ec1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.265722 25456 tablet_bootstrap.cc:492] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:14.271126 25143 tablet_service.cc:1511] Processing CreateTablet for tablet 34d2357c56f641e48b3aba13247ec1d1 (DEFAULT_TABLE table=soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:14.275158 25143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d2357c56f641e48b3aba13247ec1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.327816 25457 tablet_bootstrap.cc:492] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:14.341980 25457 tablet_bootstrap.cc:654] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.351397 25458 tablet_bootstrap.cc:492] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:14.362246 25458 tablet_bootstrap.cc:654] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.373601 25456 tablet_bootstrap.cc:654] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.376184 25458 tablet_bootstrap.cc:492] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:14.376937 25458 ts_tablet_manager.cc:1403] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.026s	user 0.005s	sys 0.017s
I20260709 13:09:14.377581 25457 tablet_bootstrap.cc:492] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:14.378266 25457 ts_tablet_manager.cc:1403] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.051s	user 0.018s	sys 0.009s
I20260709 13:09:14.380012 25456 tablet_bootstrap.cc:492] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:14.380540 25456 ts_tablet_manager.cc:1403] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.115s	user 0.014s	sys 0.000s
I20260709 13:09:14.380859 25458 raft_consensus.cc:359] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.381922 25458 raft_consensus.cc:385] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.381742 25457 raft_consensus.cc:359] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.382308 25458 raft_consensus.cc:740] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.382675 25457 raft_consensus.cc:385] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.382975 25457 raft_consensus.cc:740] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.383227 25458 consensus_queue.cc:260] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.383759 25456 raft_consensus.cc:359] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.384755 25456 raft_consensus.cc:385] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.385100 25456 raft_consensus.cc:740] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.386384 25458 ts_tablet_manager.cc:1434] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.002s
I20260709 13:09:14.383777 25457 consensus_queue.cc:260] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.387650 25458 tablet_bootstrap.cc:492] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:14.389851 25457 ts_tablet_manager.cc:1434] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:09:14.391070 25457 tablet_bootstrap.cc:492] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:14.385932 25456 consensus_queue.cc:260] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.399492 25457 tablet_bootstrap.cc:654] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.399583 25456 ts_tablet_manager.cc:1434] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.004s
I20260709 13:09:14.400965 25456 tablet_bootstrap.cc:492] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:14.402745 25458 tablet_bootstrap.cc:654] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.410815 25456 tablet_bootstrap.cc:654] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:14.425520 25184 tablet.cc:2406] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:09:14.435547 25394 raft_consensus.cc:493] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.436518 25394 raft_consensus.cc:515] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.441341 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff976e9e6d84f4e9708af71a410e275" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
W20260709 13:09:14.444162 25270 leader_election.cc:343] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.447670 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff976e9e6d84f4e9708af71a410e275" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:14.448637 25232 raft_consensus.cc:2468] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:14.451930 25270 leader_election.cc:304] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: bca68f82aedd42f5bf4c4fdfcb0c72db
I20260709 13:09:14.453729 25394 leader_election.cc:290] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:14.454427 25465 raft_consensus.cc:2804] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.455006 25465 raft_consensus.cc:493] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.454094 25458 tablet_bootstrap.cc:492] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:14.455410 25465 raft_consensus.cc:3060] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.455840 25458 ts_tablet_manager.cc:1403] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.068s	user 0.028s	sys 0.016s
I20260709 13:09:14.457383 25457 tablet_bootstrap.cc:492] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:14.458010 25457 ts_tablet_manager.cc:1403] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.067s	user 0.024s	sys 0.020s
I20260709 13:09:14.459163 25458 raft_consensus.cc:359] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.459997 25458 raft_consensus.cc:385] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.460280 25458 raft_consensus.cc:740] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.461062 25458 consensus_queue.cc:260] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.461440 25457 raft_consensus.cc:359] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.462316 25457 raft_consensus.cc:385] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.462644 25457 raft_consensus.cc:740] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.463392 25457 consensus_queue.cc:260] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.463749 25458 ts_tablet_manager.cc:1434] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:09:14.466778 25456 tablet_bootstrap.cc:492] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:14.466956 25458 tablet_bootstrap.cc:492] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:14.467448 25456 ts_tablet_manager.cc:1403] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.067s	user 0.026s	sys 0.020s
I20260709 13:09:14.471045 25456 raft_consensus.cc:359] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.471863 25456 raft_consensus.cc:385] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.472169 25456 raft_consensus.cc:740] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.472925 25456 consensus_queue.cc:260] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.475459 25458 tablet_bootstrap.cc:654] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.475984 25456 ts_tablet_manager.cc:1434] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:09:14.477193 25456 tablet_bootstrap.cc:492] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:14.465934 25457 ts_tablet_manager.cc:1434] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:09:14.482448 25457 tablet_bootstrap.cc:492] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:14.483741 25458 tablet_bootstrap.cc:492] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:14.484511 25458 ts_tablet_manager.cc:1403] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 13:09:14.485396 25456 tablet_bootstrap.cc:654] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.486944 25465 raft_consensus.cc:515] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.488405 25458 raft_consensus.cc:359] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.489261 25458 raft_consensus.cc:385] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.489465 25465 leader_election.cc:290] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:14.489586 25458 raft_consensus.cc:740] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.490793 25458 consensus_queue.cc:260] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.491407 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff976e9e6d84f4e9708af71a410e275" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:14.492117 25158 raft_consensus.cc:3060] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.493920 25457 tablet_bootstrap.cc:654] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.494489 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff976e9e6d84f4e9708af71a410e275" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:14.495169 25232 raft_consensus.cc:3060] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.500535 25458 ts_tablet_manager.cc:1434] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:09:14.501900 25458 tablet_bootstrap.cc:492] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:14.511557 25458 tablet_bootstrap.cc:654] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.521744 25457 tablet_bootstrap.cc:492] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:14.521747 25232 raft_consensus.cc:2468] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:14.522459 25457 ts_tablet_manager.cc:1403] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.040s	user 0.025s	sys 0.011s
I20260709 13:09:14.523855 25458 tablet_bootstrap.cc:492] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:14.524509 25458 ts_tablet_manager.cc:1403] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.003s
I20260709 13:09:14.525830 25270 leader_election.cc:304] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:14.526976 25465 raft_consensus.cc:2804] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.527472 25465 raft_consensus.cc:697] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Leader. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:14.527199 25457 raft_consensus.cc:359] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.527964 25458 raft_consensus.cc:359] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.528867 25458 raft_consensus.cc:385] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.528638 25465 consensus_queue.cc:237] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.529189 25458 raft_consensus.cc:740] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.530481 25158 raft_consensus.cc:2468] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:14.528396 25457 raft_consensus.cc:385] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.533223 25457 raft_consensus.cc:740] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.534186 25457 consensus_queue.cc:260] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.531410 25458 consensus_queue.cc:260] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.537091 25458 ts_tablet_manager.cc:1434] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:09:14.537146 25457 ts_tablet_manager.cc:1434] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.004s
I20260709 13:09:14.538385 25457 tablet_bootstrap.cc:492] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:14.538440 25458 tablet_bootstrap.cc:492] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:14.532717 25456 tablet_bootstrap.cc:492] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:14.542172 25456 ts_tablet_manager.cc:1403] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.065s	user 0.011s	sys 0.010s
I20260709 13:09:14.547107 25458 tablet_bootstrap.cc:654] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.547112 25457 tablet_bootstrap.cc:654] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.551093 25436 raft_consensus.cc:493] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.551685 25436 raft_consensus.cc:515] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.555972 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25090e4349ea43b5a76c9295ae73eaab" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
I20260709 13:09:14.556380 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25090e4349ea43b5a76c9295ae73eaab" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
W20260709 13:09:14.558012 25124 leader_election.cc:343] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:14.559698 25121 leader_election.cc:343] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.560799 25121 leader_election.cc:304] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:14.560282 25436 leader_election.cc:290] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:14.566497 25436 raft_consensus.cc:2749] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:14.570840 25458 tablet_bootstrap.cc:492] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:14.571699 25465 raft_consensus.cc:493] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.571501 25458 ts_tablet_manager.cc:1403] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.004s
I20260709 13:09:14.572373 25465 raft_consensus.cc:515] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.575129 25456 raft_consensus.cc:359] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.570835 25457 tablet_bootstrap.cc:492] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:14.576058 25456 raft_consensus.cc:385] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.576224 25465 leader_election.cc:290] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:14.576532 25456 raft_consensus.cc:740] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.576231 25458 raft_consensus.cc:359] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.576682 25457 ts_tablet_manager.cc:1403] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.039s	user 0.019s	sys 0.007s
I20260709 13:09:14.577131 25056 catalog_manager.cc:5697] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff reported cstate change: term changed from 0 to 1, leader changed from <none> to ac08c897ed7e478bbd7852f67f11caff (127.12.109.67). New cstate: current_term: 1 leader_uuid: "ac08c897ed7e478bbd7852f67f11caff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:14.577845 25458 raft_consensus.cc:385] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.578130 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbe909446ac4c1b8e329828cc311a52" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:14.577673 25456 consensus_queue.cc:260] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.581332 25458 raft_consensus.cc:740] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.581116 25457 raft_consensus.cc:359] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.581938 25457 raft_consensus.cc:385] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.579952 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbe909446ac4c1b8e329828cc311a52" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:14.582785 25158 raft_consensus.cc:2468] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:14.582376 25458 consensus_queue.cc:260] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.583541 25456 ts_tablet_manager.cc:1434] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 13:09:14.584438 25270 leader_election.cc:304] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: 
I20260709 13:09:14.584864 25456 tablet_bootstrap.cc:492] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:14.582355 25457 raft_consensus.cc:740] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.586107 25465 raft_consensus.cc:2804] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.586704 25465 raft_consensus.cc:493] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.587158 25465 raft_consensus.cc:3060] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.590758 25458 ts_tablet_manager.cc:1434] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:09:14.592106 25458 tablet_bootstrap.cc:492] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:14.586174 25457 consensus_queue.cc:260] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.594585 25233 raft_consensus.cc:2468] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:14.596275 25457 ts_tablet_manager.cc:1434] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:09:14.597479 25457 tablet_bootstrap.cc:492] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:14.593794 25456 tablet_bootstrap.cc:654] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.607909 25458 tablet_bootstrap.cc:654] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.608383 25465 raft_consensus.cc:515] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.610677 25456 tablet_bootstrap.cc:492] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:14.616451 25456 ts_tablet_manager.cc:1403] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.000s
I20260709 13:09:14.618940 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbe909446ac4c1b8e329828cc311a52" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:14.619696 25158 raft_consensus.cc:3060] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.620085 25456 raft_consensus.cc:359] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.620895 25456 raft_consensus.cc:385] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.621168 25456 raft_consensus.cc:740] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.622006 25456 consensus_queue.cc:260] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.626281 25456 ts_tablet_manager.cc:1434] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:09:14.626746 25465 leader_election.cc:290] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:14.627434 25456 tablet_bootstrap.cc:492] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:14.628060 25458 tablet_bootstrap.cc:492] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:14.628587 25457 tablet_bootstrap.cc:654] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.629208 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbe909446ac4c1b8e329828cc311a52" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:14.629896 25233 raft_consensus.cc:3060] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.628641 25458 ts_tablet_manager.cc:1403] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.012s
I20260709 13:09:14.634385 25458 raft_consensus.cc:359] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.635396 25458 raft_consensus.cc:385] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.635777 25458 raft_consensus.cc:740] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.636585 25456 tablet_bootstrap.cc:654] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.636791 25458 consensus_queue.cc:260] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.645190 25458 ts_tablet_manager.cc:1434] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:09:14.652323 25458 tablet_bootstrap.cc:492] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:14.664414 25474 raft_consensus.cc:493] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.665066 25474 raft_consensus.cc:515] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.665441 25233 raft_consensus.cc:2468] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:14.666347 25456 tablet_bootstrap.cc:492] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:14.662215 25158 raft_consensus.cc:2468] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:14.667050 25456 ts_tablet_manager.cc:1403] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.040s	user 0.026s	sys 0.011s
I20260709 13:09:14.668363 25270 leader_election.cc:304] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:14.669066 25457 tablet_bootstrap.cc:492] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:14.669441 25465 raft_consensus.cc:2804] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.669664 25457 ts_tablet_manager.cc:1403] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.072s	user 0.022s	sys 0.005s
I20260709 13:09:14.670081 25465 raft_consensus.cc:697] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Leader. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:14.670833 25458 tablet_bootstrap.cc:654] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.670718 25456 raft_consensus.cc:359] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.671072 25465 consensus_queue.cc:237] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.673180 25457 raft_consensus.cc:359] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.673945 25457 raft_consensus.cc:385] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.674293 25457 raft_consensus.cc:740] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.674315 25474 leader_election.cc:290] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:14.675113 25457 consensus_queue.cc:260] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.675926 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304954defc094f77bd6356c5b30b2d56" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
I20260709 13:09:14.671618 25456 raft_consensus.cc:385] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.676911 25456 raft_consensus.cc:740] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.677840 25456 consensus_queue.cc:260] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.679812 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304954defc094f77bd6356c5b30b2d56" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:14.680595 25456 ts_tablet_manager.cc:1434] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:09:14.681758 25456 tablet_bootstrap.cc:492] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
W20260709 13:09:14.682310 25124 leader_election.cc:343] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.677834 25457 ts_tablet_manager.cc:1434] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:09:14.685843 25457 tablet_bootstrap.cc:492] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:14.680540 25233 raft_consensus.cc:2468] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 0.
I20260709 13:09:14.693727 25457 tablet_bootstrap.cc:654] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.694757 25121 leader_election.cc:304] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: ac08c897ed7e478bbd7852f67f11caff
I20260709 13:09:14.696461 25474 raft_consensus.cc:2804] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.697072 25474 raft_consensus.cc:493] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.697505 25474 raft_consensus.cc:3060] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.701555 25456 tablet_bootstrap.cc:654] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.710603 25474 raft_consensus.cc:515] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.711109 25457 tablet_bootstrap.cc:492] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:14.711802 25457 ts_tablet_manager.cc:1403] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.026s	user 0.006s	sys 0.008s
I20260709 13:09:14.712868 25456 tablet_bootstrap.cc:492] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:14.713428 25456 ts_tablet_manager.cc:1403] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.003s
I20260709 13:09:14.714179 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304954defc094f77bd6356c5b30b2d56" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:14.714640 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304954defc094f77bd6356c5b30b2d56" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:14.715293 25233 raft_consensus.cc:3060] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:14.715826 25124 leader_election.cc:343] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.716506 25474 leader_election.cc:290] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 election: Requested vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:14.715541 25457 raft_consensus.cc:359] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.717317 25457 raft_consensus.cc:385] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.716832 25456 raft_consensus.cc:359] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.717677 25457 raft_consensus.cc:740] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.717762 25456 raft_consensus.cc:385] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.718180 25456 raft_consensus.cc:740] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.718637 25457 consensus_queue.cc:260] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.721392 25457 ts_tablet_manager.cc:1434] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:09:14.721103 25056 catalog_manager.cc:5697] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff reported cstate change: term changed from 0 to 1, leader changed from <none> to ac08c897ed7e478bbd7852f67f11caff (127.12.109.67). New cstate: current_term: 1 leader_uuid: "ac08c897ed7e478bbd7852f67f11caff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:14.722723 25457 tablet_bootstrap.cc:492] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:14.725077 25233 raft_consensus.cc:2468] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 1.
I20260709 13:09:14.726609 25121 leader_election.cc:304] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: ac08c897ed7e478bbd7852f67f11caff
I20260709 13:09:14.728986 25474 raft_consensus.cc:2804] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.731204 25457 tablet_bootstrap.cc:654] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.732954 25474 raft_consensus.cc:697] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Becoming Leader. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:14.738328 25458 tablet_bootstrap.cc:492] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:14.738986 25458 ts_tablet_manager.cc:1403] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.093s	user 0.024s	sys 0.024s
I20260709 13:09:14.741065 25473 raft_consensus.cc:493] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.741863 25473 raft_consensus.cc:515] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.742857 25458 raft_consensus.cc:359] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.743741 25398 raft_consensus.cc:493] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.744077 25458 raft_consensus.cc:385] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.744438 25458 raft_consensus.cc:740] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.744304 25398 raft_consensus.cc:515] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.745298 25458 consensus_queue.cc:260] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.748070 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a417912724d4279b7b1d1e1ddfcec1c" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:14.748234 25473 leader_election.cc:290] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:14.748121 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f19f2e70f5e4b189c52a968eeb7c61b" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
I20260709 13:09:14.748920 25158 raft_consensus.cc:2468] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 0.
I20260709 13:09:14.749442 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f19f2e70f5e4b189c52a968eeb7c61b" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:14.750115 25398 leader_election.cc:290] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:14.750664 25196 leader_election.cc:304] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:14.751852 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a417912724d4279b7b1d1e1ddfcec1c" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff" is_pre_election: true
I20260709 13:09:14.748157 25458 ts_tablet_manager.cc:1434] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:09:14.753468 25233 raft_consensus.cc:2468] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 0.
I20260709 13:09:14.753579 25398 raft_consensus.cc:2804] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.754034 25398 raft_consensus.cc:493] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.754436 25398 raft_consensus.cc:3060] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.754709 25458 tablet_bootstrap.cc:492] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:14.754848 25121 leader_election.cc:304] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
W20260709 13:09:14.755086 25199 leader_election.cc:343] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.752291 25474 consensus_queue.cc:237] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.755947 25473 raft_consensus.cc:2804] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.756364 25473 raft_consensus.cc:493] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.756755 25473 raft_consensus.cc:3060] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.719204 25456 consensus_queue.cc:260] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.759673 25307 raft_consensus.cc:2468] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 0.
I20260709 13:09:14.761363 25456 ts_tablet_manager.cc:1434] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.000s
I20260709 13:09:14.763271 25456 tablet_bootstrap.cc:492] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:14.764096 25398 raft_consensus.cc:515] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.766644 25398 leader_election.cc:290] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:14.770503 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a417912724d4279b7b1d1e1ddfcec1c" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:14.771250 25158 raft_consensus.cc:3060] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.775028 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a417912724d4279b7b1d1e1ddfcec1c" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:14.776788 25473 raft_consensus.cc:515] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.778465 25458 tablet_bootstrap.cc:654] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:14.778528 25199 leader_election.cc:343] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.780093 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f19f2e70f5e4b189c52a968eeb7c61b" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:14.780687 25233 raft_consensus.cc:3060] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.783864 25457 tablet_bootstrap.cc:492] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:14.784633 25457 ts_tablet_manager.cc:1403] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.062s	user 0.029s	sys 0.004s
I20260709 13:09:14.788538 25457 raft_consensus.cc:359] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.789381 25457 raft_consensus.cc:385] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.789716 25457 raft_consensus.cc:740] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.790624 25457 consensus_queue.cc:260] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.792559 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f19f2e70f5e4b189c52a968eeb7c61b" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:14.793576 25457 ts_tablet_manager.cc:1434] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:09:14.794890 25457 tablet_bootstrap.cc:492] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:14.793522 25307 raft_consensus.cc:3060] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.797314 25456 tablet_bootstrap.cc:654] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.795848 25473 leader_election.cc:290] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 election: Requested vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:14.799494 25465 raft_consensus.cc:493] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.800082 25465 raft_consensus.cc:515] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.801971 25233 raft_consensus.cc:2468] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 1.
I20260709 13:09:14.802418 25465 leader_election.cc:290] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:14.803293 25121 leader_election.cc:304] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:14.803795 25158 raft_consensus.cc:2468] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 1.
I20260709 13:09:14.805084 25196 leader_election.cc:304] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: ac08c897ed7e478bbd7852f67f11caff
I20260709 13:09:14.806411 25398 raft_consensus.cc:2804] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.808248 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97e2ed0f2196447f997303c0889538af" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:14.808499 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97e2ed0f2196447f997303c0889538af" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:14.809264 25157 raft_consensus.cc:2468] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
W20260709 13:09:14.809930 25270 leader_election.cc:343] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.811684 25270 leader_election.cc:304] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:14.811889 25474 raft_consensus.cc:2804] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.812542 25474 raft_consensus.cc:697] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Becoming Leader. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:14.813158 25457 tablet_bootstrap.cc:654] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.813131 25465 raft_consensus.cc:2804] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.813774 25465 raft_consensus.cc:493] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.814180 25465 raft_consensus.cc:3060] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.815801 25056 catalog_manager.cc:5697] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db reported cstate change: term changed from 0 to 1, leader changed from <none> to bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65). New cstate: current_term: 1 leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:14.813436 25474 consensus_queue.cc:237] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.830359 25457 tablet_bootstrap.cc:492] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:14.830969 25457 ts_tablet_manager.cc:1403] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.003s
I20260709 13:09:14.832840 25307 raft_consensus.cc:2468] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 1.
I20260709 13:09:14.833493 25456 tablet_bootstrap.cc:492] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:14.834264 25456 ts_tablet_manager.cc:1403] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.071s	user 0.017s	sys 0.000s
I20260709 13:09:14.836066 25398 raft_consensus.cc:697] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Becoming Leader. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:14.836293 25465 raft_consensus.cc:515] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.841316 25458 tablet_bootstrap.cc:492] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:14.842871 25458 ts_tablet_manager.cc:1403] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.088s	user 0.021s	sys 0.010s
I20260709 13:09:14.842937 25398 consensus_queue.cc:237] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.834653 25457 raft_consensus.cc:359] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.845327 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97e2ed0f2196447f997303c0889538af" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:14.838436 25456 raft_consensus.cc:359] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.846350 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97e2ed0f2196447f997303c0889538af" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:14.847667 25457 raft_consensus.cc:385] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.846216 25458 raft_consensus.cc:359] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.847026 25465 leader_election.cc:290] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:14.848042 25158 raft_consensus.cc:3060] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.848052 25457 raft_consensus.cc:740] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.848389 25458 raft_consensus.cc:385] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.849160 25458 raft_consensus.cc:740] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.849443 25457 consensus_queue.cc:260] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.850589 25233 raft_consensus.cc:3060] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.852023 25457 ts_tablet_manager.cc:1434] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:09:14.853195 25457 tablet_bootstrap.cc:492] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630: Bootstrap starting.
I20260709 13:09:14.860004 25457 tablet_bootstrap.cc:654] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.848129 25456 raft_consensus.cc:385] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.860702 25456 raft_consensus.cc:740] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.861533 25233 raft_consensus.cc:2468] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:14.863092 25456 consensus_queue.cc:260] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.864814 25270 leader_election.cc:304] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:14.866446 25456 ts_tablet_manager.cc:1434] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.031s	user 0.003s	sys 0.006s
I20260709 13:09:14.866353 25465 raft_consensus.cc:2804] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.867269 25465 raft_consensus.cc:697] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Leader. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:14.846306 25056 catalog_manager.cc:5697] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db reported cstate change: term changed from 0 to 1, leader changed from <none> to bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65). New cstate: current_term: 1 leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:14.850042 25458 consensus_queue.cc:260] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.875392 25458 ts_tablet_manager.cc:1434] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.032s	user 0.004s	sys 0.004s
I20260709 13:09:14.875964 25158 raft_consensus.cc:2468] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:14.877147 25458 tablet_bootstrap.cc:492] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db: Bootstrap starting.
I20260709 13:09:14.879207 25456 tablet_bootstrap.cc:492] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:14.868930 25465 consensus_queue.cc:237] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.891644 25458 tablet_bootstrap.cc:654] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.892868 25456 tablet_bootstrap.cc:654] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.899163 25457 tablet_bootstrap.cc:492] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630: No bootstrap required, opened a new log
I20260709 13:09:14.899715 25457 ts_tablet_manager.cc:1403] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.002s
I20260709 13:09:14.900514 25456 tablet_bootstrap.cc:492] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:14.901140 25456 ts_tablet_manager.cc:1403] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.004s
I20260709 13:09:14.902376 25056 catalog_manager.cc:5697] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66). New cstate: current_term: 1 leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:14.907972 25456 raft_consensus.cc:359] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.910521 25456 raft_consensus.cc:385] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.910900 25456 raft_consensus.cc:740] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.911783 25456 consensus_queue.cc:260] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.913564 25456 ts_tablet_manager.cc:1434] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:09:14.914726 25456 tablet_bootstrap.cc:492] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff: Bootstrap starting.
I20260709 13:09:14.909610 25457 raft_consensus.cc:359] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.915151 25458 tablet_bootstrap.cc:492] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db: No bootstrap required, opened a new log
I20260709 13:09:14.915750 25458 ts_tablet_manager.cc:1403] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.005s
I20260709 13:09:14.917512 25465 raft_consensus.cc:493] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.918097 25465 raft_consensus.cc:515] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.920383 25465 leader_election.cc:290] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:14.920336 25054 catalog_manager.cc:5697] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff reported cstate change: term changed from 0 to 1, leader changed from <none> to ac08c897ed7e478bbd7852f67f11caff (127.12.109.67). New cstate: current_term: 1 leader_uuid: "ac08c897ed7e478bbd7852f67f11caff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:14.922401 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76ad5adfa4824e669ae7414ebd491691" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:14.923013 25158 raft_consensus.cc:2468] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:14.923204 25456 tablet_bootstrap.cc:654] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.925411 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76ad5adfa4824e669ae7414ebd491691" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:14.926942 25270 leader_election.cc:304] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: 
I20260709 13:09:14.915169 25457 raft_consensus.cc:385] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.929075 25457 raft_consensus.cc:740] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.930802 25233 raft_consensus.cc:2468] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:14.929661 25394 raft_consensus.cc:2804] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.932401 25394 raft_consensus.cc:493] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.929838 25457 consensus_queue.cc:260] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.932653 25456 tablet_bootstrap.cc:492] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff: No bootstrap required, opened a new log
I20260709 13:09:14.932763 25394 raft_consensus.cc:3060] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.933599 25456 ts_tablet_manager.cc:1403] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 13:09:14.934710 25457 ts_tablet_manager.cc:1434] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:09:14.936805 25456 raft_consensus.cc:359] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.936971 25458 raft_consensus.cc:359] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.937561 25456 raft_consensus.cc:385] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.937664 25458 raft_consensus.cc:385] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.937898 25456 raft_consensus.cc:740] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.937978 25458 raft_consensus.cc:740] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.938936 25458 consensus_queue.cc:260] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.938936 25456 consensus_queue.cc:260] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.941359 25456 ts_tablet_manager.cc:1434] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:14.941447 25458 ts_tablet_manager.cc:1434] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:09:14.944257 25394 raft_consensus.cc:515] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.946673 25394 leader_election.cc:290] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:14.947755 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76ad5adfa4824e669ae7414ebd491691" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:14.948328 25158 raft_consensus.cc:3060] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.950402 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76ad5adfa4824e669ae7414ebd491691" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:14.951049 25233 raft_consensus.cc:3060] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.956646 25158 raft_consensus.cc:2468] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:14.958081 25270 leader_election.cc:304] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: 
I20260709 13:09:14.959686 25394 raft_consensus.cc:2804] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.960287 25394 raft_consensus.cc:697] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Leader. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:14.961194 25394 consensus_queue.cc:237] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.959484 25233 raft_consensus.cc:2468] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
W20260709 13:09:14.964138 25104 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:14.970894 25054 catalog_manager.cc:5697] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff reported cstate change: term changed from 0 to 1, leader changed from <none> to ac08c897ed7e478bbd7852f67f11caff (127.12.109.67). New cstate: current_term: 1 leader_uuid: "ac08c897ed7e478bbd7852f67f11caff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:14.989477 25394 raft_consensus.cc:493] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.990154 25394 raft_consensus.cc:515] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:14.992246 25394 leader_election.cc:290] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:14.993237 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a417912724d4279b7b1d1e1ddfcec1c" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:14.993366 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a417912724d4279b7b1d1e1ddfcec1c" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:14.994172 25158 raft_consensus.cc:2393] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ac08c897ed7e478bbd7852f67f11caff in current term 1: Already voted for candidate c3395c72252c4138b3d77bf5f0d7d630 in this term.
I20260709 13:09:14.995241 25270 leader_election.cc:304] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff; no voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:14.996038 25394 raft_consensus.cc:3060] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:15.002136 25394 raft_consensus.cc:2749] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:15.077719 25394 consensus_queue.cc:1048] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:15.091008 25394 consensus_queue.cc:1048] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.103853 25394 raft_consensus.cc:493] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.104348 25394 raft_consensus.cc:515] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.106355 25394 leader_election.cc:290] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:15.107086 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25090e4349ea43b5a76c9295ae73eaab" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:15.107872 25158 raft_consensus.cc:2468] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:15.108852 25270 leader_election.cc:304] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: 
I20260709 13:09:15.108505 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25090e4349ea43b5a76c9295ae73eaab" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:15.109782 25394 raft_consensus.cc:2804] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:15.110343 25233 raft_consensus.cc:2468] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:15.110354 25394 raft_consensus.cc:493] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:15.110898 25394 raft_consensus.cc:3060] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:15.117705 25394 raft_consensus.cc:515] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.119633 25394 leader_election.cc:290] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:15.120545 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25090e4349ea43b5a76c9295ae73eaab" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:15.120770 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25090e4349ea43b5a76c9295ae73eaab" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:15.121146 25158 raft_consensus.cc:3060] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:15.121390 25233 raft_consensus.cc:3060] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:15.127058 25233 raft_consensus.cc:2468] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:15.127285 25158 raft_consensus.cc:2468] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:15.128139 25270 leader_election.cc:304] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:15.128971 25394 raft_consensus.cc:2804] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:15.129417 25394 raft_consensus.cc:697] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Leader. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:15.130180 25394 consensus_queue.cc:237] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.139590 25055 catalog_manager.cc:5697] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff reported cstate change: term changed from 0 to 1, leader changed from <none> to ac08c897ed7e478bbd7852f67f11caff (127.12.109.67). New cstate: current_term: 1 leader_uuid: "ac08c897ed7e478bbd7852f67f11caff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:15.168668 25474 consensus_queue.cc:1048] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.179764 25307 raft_consensus.cc:3060] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:15.190613 25503 raft_consensus.cc:493] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.190377 25394 raft_consensus.cc:493] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.191272 25503 raft_consensus.cc:515] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.191607 25509 consensus_queue.cc:1048] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:15.191689 25394 raft_consensus.cc:515] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.193754 25503 leader_election.cc:290] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:15.194963 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d2357c56f641e48b3aba13247ec1d1" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:15.195778 25158 raft_consensus.cc:2468] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:15.196141 25394 leader_election.cc:290] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:15.195119 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d2357c56f641e48b3aba13247ec1d1" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:15.197815 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304954defc094f77bd6356c5b30b2d56" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" is_pre_election: true
I20260709 13:09:15.198239 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304954defc094f77bd6356c5b30b2d56" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630" is_pre_election: true
I20260709 13:09:15.197883 25270 leader_election.cc:304] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: 
I20260709 13:09:15.199919 25233 raft_consensus.cc:2468] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 0.
I20260709 13:09:15.203990 25270 leader_election.cc:304] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff; no voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:15.203887 25509 consensus_queue.cc:1048] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.206282 25394 raft_consensus.cc:2804] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:15.206688 25394 raft_consensus.cc:493] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:15.207119 25394 raft_consensus.cc:3060] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:15.213052 25503 raft_consensus.cc:2749] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:15.224979 25394 raft_consensus.cc:515] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.227398 25394 leader_election.cc:290] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:15.228606 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d2357c56f641e48b3aba13247ec1d1" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:15.229070 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d2357c56f641e48b3aba13247ec1d1" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:15.229262 25233 raft_consensus.cc:3060] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:15.229725 25158 raft_consensus.cc:3060] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:15.230981 25436 consensus_queue.cc:1048] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.238999 25233 raft_consensus.cc:2468] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:15.240706 25270 leader_election.cc:304] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:15.241281 25158 raft_consensus.cc:2468] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 1.
I20260709 13:09:15.241911 25503 raft_consensus.cc:2804] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:15.242619 25503 raft_consensus.cc:697] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Leader. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:15.243490 25503 consensus_queue.cc:237] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.255002 25054 catalog_manager.cc:5697] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff reported cstate change: term changed from 0 to 1, leader changed from <none> to ac08c897ed7e478bbd7852f67f11caff (127.12.109.67). New cstate: current_term: 1 leader_uuid: "ac08c897ed7e478bbd7852f67f11caff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:15.271971 25503 consensus_queue.cc:1048] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.285175 25465 consensus_queue.cc:1048] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.288578 25398 consensus_queue.cc:1048] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.307090 25492 consensus_queue.cc:1048] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:15.402604 25503 consensus_queue.cc:1048] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.414211 25465 consensus_queue.cc:1048] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.430128 25306 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ff976e9e6d84f4e9708af71a410e275"
dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
mode: GRACEFUL
new_leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:58264
I20260709 13:09:15.433964 25306 raft_consensus.cc:606] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Received request to transfer leadership to TS bca68f82aedd42f5bf4c4fdfcb0c72db
I20260709 13:09:15.443830 25306 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7dbe909446ac4c1b8e329828cc311a52"
dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
mode: GRACEFUL
new_leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:58264
I20260709 13:09:15.444648 25306 raft_consensus.cc:606] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Received request to transfer leadership to TS bca68f82aedd42f5bf4c4fdfcb0c72db
I20260709 13:09:15.451264 25306 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76ad5adfa4824e669ae7414ebd491691"
dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
mode: GRACEFUL
new_leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:58264
I20260709 13:09:15.451843 25306 raft_consensus.cc:606] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Received request to transfer leadership to TS bca68f82aedd42f5bf4c4fdfcb0c72db
I20260709 13:09:15.456933 25233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a417912724d4279b7b1d1e1ddfcec1c"
dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
mode: GRACEFUL
new_leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:52808
I20260709 13:09:15.457460 25233 raft_consensus.cc:606] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Received request to transfer leadership to TS bca68f82aedd42f5bf4c4fdfcb0c72db
I20260709 13:09:15.462541 25307 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25090e4349ea43b5a76c9295ae73eaab"
dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
mode: GRACEFUL
new_leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:58264
I20260709 13:09:15.463197 25307 raft_consensus.cc:606] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Received request to transfer leadership to TS c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:15.468370 25157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "304954defc094f77bd6356c5b30b2d56"
dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
mode: GRACEFUL
new_leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:51882
I20260709 13:09:15.468884 25157 raft_consensus.cc:606] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Received request to transfer leadership to TS c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:15.474267 25394 consensus_queue.cc:1048] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.474951 25306 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97e2ed0f2196447f997303c0889538af"
dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
mode: GRACEFUL
new_leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:58264
I20260709 13:09:15.475745 25306 raft_consensus.cc:606] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Received request to transfer leadership to TS c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:15.482280 25158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f19f2e70f5e4b189c52a968eeb7c61b"
dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
mode: GRACEFUL
new_leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:51882
I20260709 13:09:15.482815 25158 raft_consensus.cc:606] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Received request to transfer leadership to TS c3395c72252c4138b3d77bf5f0d7d630
I20260709 13:09:15.491200 25394 consensus_queue.cc:1048] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.499078 25503 raft_consensus.cc:993] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff: : Instructing follower c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:15.499611 25394 raft_consensus.cc:1081] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Signalling peer c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:15.501322 25233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97e2ed0f2196447f997303c0889538af"
dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:52796
I20260709 13:09:15.502617 25233 raft_consensus.cc:493] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:15.502975 25233 raft_consensus.cc:3060] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.508884 25233 raft_consensus.cc:515] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.511246 25233 leader_election.cc:290] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:15.511617 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97e2ed0f2196447f997303c0889538af" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:15.512349 25158 raft_consensus.cc:3060] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.512557 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97e2ed0f2196447f997303c0889538af" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:15.513182 25306 raft_consensus.cc:3055] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:15.513412 25306 raft_consensus.cc:740] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:15.514204 25306 consensus_queue.cc:260] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.515199 25306 raft_consensus.cc:3060] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.518151 25158 raft_consensus.cc:2468] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:15.519413 25196 leader_election.cc:304] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:15.520378 25398 raft_consensus.cc:2804] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:15.520887 25398 raft_consensus.cc:697] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Becoming Leader. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:15.521936 25306 raft_consensus.cc:2468] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:15.521723 25398 consensus_queue.cc:237] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.529556 25055 catalog_manager.cc:5697] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 reported cstate change: term changed from 1 to 2, leader changed from ac08c897ed7e478bbd7852f67f11caff (127.12.109.67) to c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.545519 25503 raft_consensus.cc:993] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff: : Instructing follower bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:15.546087 25394 raft_consensus.cc:1081] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Signalling peer bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:15.547977 25158 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ff976e9e6d84f4e9708af71a410e275"
dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:51860
I20260709 13:09:15.548584 25158 raft_consensus.cc:493] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:15.548952 25158 raft_consensus.cc:3060] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.556406 25158 raft_consensus.cc:515] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.558120 25158 leader_election.cc:290] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Requested vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:15.559136 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff976e9e6d84f4e9708af71a410e275" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:15.559302 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff976e9e6d84f4e9708af71a410e275" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:15.560027 25233 raft_consensus.cc:3060] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.560134 25306 raft_consensus.cc:3055] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:15.560606 25306 raft_consensus.cc:740] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:15.561474 25306 consensus_queue.cc:260] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.562945 25306 raft_consensus.cc:3060] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.565690 25509 raft_consensus.cc:993] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db: : Instructing follower c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:15.566298 25509 raft_consensus.cc:1081] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Signalling peer c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:15.566210 25394 consensus_queue.cc:1048] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:15.568643 25231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "304954defc094f77bd6356c5b30b2d56"
dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:52790
I20260709 13:09:15.569217 25231 raft_consensus.cc:493] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:15.569607 25231 raft_consensus.cc:3060] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.569720 25233 raft_consensus.cc:2468] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:15.571357 25121 leader_election.cc:304] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:15.572407 25509 raft_consensus.cc:2804] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:15.573717 25509 raft_consensus.cc:697] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Becoming Leader. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:15.572359 25306 raft_consensus.cc:2468] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:15.575035 25509 consensus_queue.cc:237] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.578359 25503 raft_consensus.cc:993] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff: : Instructing follower c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:15.578912 25519 raft_consensus.cc:1081] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Signalling peer c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:15.580998 25231 raft_consensus.cc:515] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.583892 25231 leader_election.cc:290] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:15.584604 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304954defc094f77bd6356c5b30b2d56" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:15.585261 25158 raft_consensus.cc:3055] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:15.585566 25158 raft_consensus.cc:740] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:15.586122 25394 consensus_queue.cc:1048] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:15.587918 25158 consensus_queue.cc:260] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.589335 25158 raft_consensus.cc:3060] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.591971 25231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25090e4349ea43b5a76c9295ae73eaab"
dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:52796
I20260709 13:09:15.592029 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304954defc094f77bd6356c5b30b2d56" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:15.592978 25307 raft_consensus.cc:3060] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.594168 25231 raft_consensus.cc:493] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:15.594815 25231 raft_consensus.cc:3060] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.587476 25055 catalog_manager.cc:5697] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db reported cstate change: term changed from 1 to 2, leader changed from ac08c897ed7e478bbd7852f67f11caff (127.12.109.67) to bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65). New cstate: current_term: 2 leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.603374 25158 raft_consensus.cc:2468] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:15.603404 25231 raft_consensus.cc:515] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.605127 25196 leader_election.cc:304] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:15.606022 25398 raft_consensus.cc:2804] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:15.606612 25398 raft_consensus.cc:697] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Becoming Leader. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:15.607501 25398 consensus_queue.cc:237] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.608812 25231 leader_election.cc:290] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:15.611833 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25090e4349ea43b5a76c9295ae73eaab" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:15.612464 25306 raft_consensus.cc:3055] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:15.612792 25306 raft_consensus.cc:740] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:15.613639 25306 consensus_queue.cc:260] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.614774 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25090e4349ea43b5a76c9295ae73eaab" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:15.615132 25306 raft_consensus.cc:3060] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.617493 25158 raft_consensus.cc:3060] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.616768 25307 raft_consensus.cc:2468] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:15.623473 25055 catalog_manager.cc:5697] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 reported cstate change: term changed from 1 to 2, leader changed from bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65) to c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.625022 25158 raft_consensus.cc:2468] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:15.626391 25196 leader_election.cc:304] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:15.627238 25398 raft_consensus.cc:2804] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:15.627449 25306 raft_consensus.cc:2468] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:15.627784 25398 raft_consensus.cc:697] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Becoming Leader. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:15.628731 25398 consensus_queue.cc:237] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.639134 25055 catalog_manager.cc:5697] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 reported cstate change: term changed from 1 to 2, leader changed from ac08c897ed7e478bbd7852f67f11caff (127.12.109.67) to c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.676532 25519 raft_consensus.cc:993] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff: : Instructing follower bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:15.677028 25394 raft_consensus.cc:1081] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Signalling peer bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:15.678673 25158 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7dbe909446ac4c1b8e329828cc311a52"
dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:51860
I20260709 13:09:15.679227 25158 raft_consensus.cc:493] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:15.679533 25158 raft_consensus.cc:3060] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.685036 25158 raft_consensus.cc:515] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.686867 25158 leader_election.cc:290] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Requested vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:15.687933 25231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbe909446ac4c1b8e329828cc311a52" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:15.687947 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbe909446ac4c1b8e329828cc311a52" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:15.688726 25231 raft_consensus.cc:3060] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.688804 25306 raft_consensus.cc:3055] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:15.689252 25306 raft_consensus.cc:740] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:15.689975 25306 consensus_queue.cc:260] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.691154 25306 raft_consensus.cc:3060] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.695170 25231 raft_consensus.cc:2468] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:15.696530 25121 leader_election.cc:304] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:15.697347 25509 raft_consensus.cc:2804] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:15.697896 25306 raft_consensus.cc:2468] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:15.698244 25509 raft_consensus.cc:697] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Becoming Leader. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:15.699802 25509 consensus_queue.cc:237] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.707854 25055 catalog_manager.cc:5697] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db reported cstate change: term changed from 1 to 2, leader changed from ac08c897ed7e478bbd7852f67f11caff (127.12.109.67) to bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65). New cstate: current_term: 2 leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.760735 25394 consensus_queue.cc:1048] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.770138 25398 raft_consensus.cc:993] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630: : Instructing follower bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:15.770836 25492 raft_consensus.cc:1081] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Signalling peer bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:15.773094 25158 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a417912724d4279b7b1d1e1ddfcec1c"
dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:51874
I20260709 13:09:15.773682 25158 raft_consensus.cc:493] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:15.774117 25158 raft_consensus.cc:3060] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.775405 25394 consensus_queue.cc:1048] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:15.782223 25158 raft_consensus.cc:515] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.785369 25231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a417912724d4279b7b1d1e1ddfcec1c" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:15.786087 25231 raft_consensus.cc:3055] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:15.786407 25231 raft_consensus.cc:740] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:15.787256 25158 leader_election.cc:290] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Requested vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:15.787358 25231 consensus_queue.cc:260] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.788941 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a417912724d4279b7b1d1e1ddfcec1c" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:15.789343 25231 raft_consensus.cc:3060] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.789601 25306 raft_consensus.cc:3060] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.799105 25306 raft_consensus.cc:2468] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:15.800630 25231 raft_consensus.cc:2468] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:15.800792 25124 leader_election.cc:304] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: 
I20260709 13:09:15.801834 25509 raft_consensus.cc:2804] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:15.802331 25509 raft_consensus.cc:697] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Becoming Leader. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:15.803220 25509 consensus_queue.cc:237] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.812414 25055 catalog_manager.cc:5697] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db reported cstate change: term changed from 1 to 2, leader changed from c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66) to bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65). New cstate: current_term: 2 leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.821971 25436 raft_consensus.cc:993] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db: : Instructing follower c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:15.822570 25509 raft_consensus.cc:1081] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Signalling peer c3395c72252c4138b3d77bf5f0d7d630 to start an election
I20260709 13:09:15.824113 25231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f19f2e70f5e4b189c52a968eeb7c61b"
dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
 from {username='slave'} at 127.0.0.1:52790
I20260709 13:09:15.824787 25231 raft_consensus.cc:493] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:15.825101 25231 raft_consensus.cc:3060] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.831341 25231 raft_consensus.cc:515] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.833287 25231 leader_election.cc:290] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:15.834208 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f19f2e70f5e4b189c52a968eeb7c61b" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:15.834825 25158 raft_consensus.cc:3055] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:15.834823 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f19f2e70f5e4b189c52a968eeb7c61b" candidate_uuid: "c3395c72252c4138b3d77bf5f0d7d630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:15.835346 25158 raft_consensus.cc:740] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:15.835587 25306 raft_consensus.cc:3060] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.836194 25158 consensus_queue.cc:260] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.837201 25158 raft_consensus.cc:3060] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.843025 25158 raft_consensus.cc:2468] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:15.842854 25306 raft_consensus.cc:2468] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3395c72252c4138b3d77bf5f0d7d630 in term 2.
I20260709 13:09:15.844380 25196 leader_election.cc:304] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bca68f82aedd42f5bf4c4fdfcb0c72db, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:15.845326 25398 raft_consensus.cc:2804] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:15.845888 25398 raft_consensus.cc:697] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Becoming Leader. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:15.846812 25398 consensus_queue.cc:237] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.854561 25055 catalog_manager.cc:5697] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 reported cstate change: term changed from 1 to 2, leader changed from bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65) to c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66). New cstate: current_term: 2 leader_uuid: "c3395c72252c4138b3d77bf5f0d7d630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.916901 25306 raft_consensus.cc:1275] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:15.918352 25398 consensus_queue.cc:1048] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:15.926333 25158 raft_consensus.cc:1275] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:15.928964 25492 consensus_queue.cc:1048] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:15.936786 25503 raft_consensus.cc:993] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff: : Instructing follower bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:15.937274 25394 raft_consensus.cc:1081] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Signalling peer bca68f82aedd42f5bf4c4fdfcb0c72db to start an election
I20260709 13:09:15.938903 25157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76ad5adfa4824e669ae7414ebd491691"
dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
 from {username='slave'} at 127.0.0.1:51860
I20260709 13:09:15.939515 25157 raft_consensus.cc:493] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:15.939879 25157 raft_consensus.cc:3060] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.947851 25157 raft_consensus.cc:515] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.949453 25157 leader_election.cc:290] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Requested vote from peers c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153), ac08c897ed7e478bbd7852f67f11caff (127.12.109.67:41425)
I20260709 13:09:15.950367 25231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76ad5adfa4824e669ae7414ebd491691" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:15.950579 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76ad5adfa4824e669ae7414ebd491691" candidate_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
I20260709 13:09:15.951303 25306 raft_consensus.cc:3055] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:15.951303 25231 raft_consensus.cc:3060] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.951763 25306 raft_consensus.cc:740] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:15.952781 25306 consensus_queue.cc:260] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.954187 25306 raft_consensus.cc:3060] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:15.961692 25306 raft_consensus.cc:2468] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:15.963027 25124 leader_election.cc:304] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: 
I20260709 13:09:15.963914 25509 raft_consensus.cc:2804] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:15.964354 25509 raft_consensus.cc:697] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Becoming Leader. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:15.965145 25509 consensus_queue.cc:237] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:15.966768 25231 raft_consensus.cc:2468] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca68f82aedd42f5bf4c4fdfcb0c72db in term 2.
I20260709 13:09:15.972648 25306 raft_consensus.cc:1275] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:15.972831 25055 catalog_manager.cc:5697] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db reported cstate change: term changed from 1 to 2, leader changed from ac08c897ed7e478bbd7852f67f11caff (127.12.109.67) to bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65). New cstate: current_term: 2 leader_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.974259 25509 consensus_queue.cc:1048] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:15.990080 25231 raft_consensus.cc:1275] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:15.991472 25509 consensus_queue.cc:1048] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.033746 25306 raft_consensus.cc:1275] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:16.035331 25490 consensus_queue.cc:1048] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.044076 25157 raft_consensus.cc:1275] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:16.046275 25492 consensus_queue.cc:1048] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.106397 25306 raft_consensus.cc:1275] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:16.107941 25527 consensus_queue.cc:1048] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.116073 25231 raft_consensus.cc:1275] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:16.118356 25436 consensus_queue.cc:1048] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.125017 25157 raft_consensus.cc:1275] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:16.126534 25490 consensus_queue.cc:1048] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.135840 25306 raft_consensus.cc:1275] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:16.137312 25492 consensus_queue.cc:1048] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.224128 25307 raft_consensus.cc:1275] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:16.225630 25527 consensus_queue.cc:1048] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.234441 25232 raft_consensus.cc:1275] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:16.236114 25527 consensus_queue.cc:1048] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.356778 25157 raft_consensus.cc:1275] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:16.358450 25398 consensus_queue.cc:1048] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.359921 25307 raft_consensus.cc:1275] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer c3395c72252c4138b3d77bf5f0d7d630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:16.361392 25398 consensus_queue.cc:1048] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.565696 25231 raft_consensus.cc:1275] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:16.567267 25436 consensus_queue.cc:1048] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.577471 25306 raft_consensus.cc:1275] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Refusing update from remote peer bca68f82aedd42f5bf4c4fdfcb0c72db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:16.579746 25473 consensus_queue.cc:1048] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:18.491020 25055 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55842:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:09:18.514281 25055 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55842:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:09:18.541085 12725 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:09:18.901803 25569 log.cc:927] Time spent T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:09:18.883663 25564 log.cc:927] Time spent T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:09:18.913403 25270 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:37649, user_credentials={real_user=slave}} blocked reactor thread for 65093.4us
I20260709 13:09:19.000330 25586 tablet.cc:1724] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.015132 25584 tablet.cc:1724] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.047093 25582 tablet.cc:1724] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:19.090782 25579 log.cc:927] Time spent T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:09:19.102257 25589 tablet.cc:1724] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.085896 25585 tablet.cc:1724] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.107306 25588 tablet.cc:1724] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.016027 25583 tablet.cc:1724] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:19.113235 25580 log.cc:927] Time spent T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff: Append to log took a long time: real 0.127s	user 0.003s	sys 0.000s
I20260709 13:09:19.116146 25593 tablet.cc:1724] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.127736 25596 tablet.cc:1724] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.140085 25590 tablet.cc:1724] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:19.154778 25199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.67:41425, user_credentials={real_user=slave}} blocked reactor thread for 71768us
I20260709 13:09:19.185316 25582 tablet.cc:1724] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.190948 25584 tablet.cc:1724] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.197849 25585 tablet.cc:1724] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.204209 25586 tablet.cc:1724] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.206475 25594 tablet.cc:1724] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.218161 25596 tablet.cc:1724] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.218434 25597 tablet.cc:1724] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.227288 25593 tablet.cc:1724] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:19.239882 25595 log.cc:927] Time spent T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:09:19.191686 25589 tablet.cc:1724] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:19.241686 25196 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.109.65:37649, user_credentials={real_user=slave}} blocked reactor thread for 156151us
I20260709 13:09:19.257187 25590 tablet.cc:1724] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.267861 25584 tablet.cc:1724] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.260507 25585 tablet.cc:1724] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.291853 25583 tablet.cc:1724] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.297544 25229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e"
dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
mode: GRACEFUL
new_leader_uuid: "ac08c897ed7e478bbd7852f67f11caff"
 from {username='slave'} at 127.0.0.1:34984
I20260709 13:09:19.298400 25229 raft_consensus.cc:606] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Received request to transfer leadership to TS ac08c897ed7e478bbd7852f67f11caff
I20260709 13:09:19.301139 25594 tablet.cc:1724] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.307315 25588 tablet.cc:1724] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.325002 25598 raft_consensus.cc:993] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630: : Instructing follower ac08c897ed7e478bbd7852f67f11caff to start an election
I20260709 13:09:19.325678 25490 raft_consensus.cc:1081] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Signalling peer ac08c897ed7e478bbd7852f67f11caff to start an election
I20260709 13:09:19.328671 25302 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e"
dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
 from {username='slave'} at 127.0.0.1:58258
I20260709 13:09:19.329300 25302 raft_consensus.cc:493] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:19.329708 25302 raft_consensus.cc:3060] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:19.330679 25588 tablet.cc:1724] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.321079 25593 tablet.cc:1724] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:19.346004 25302 raft_consensus.cc:515] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:19.351239 25157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33e80f79ae7547a7a8e6bd6fd8d95dcb"
dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
mode: GRACEFUL
new_leader_uuid: "ac08c897ed7e478bbd7852f67f11caff"
 from {username='slave'} at 127.0.0.1:38406
I20260709 13:09:19.352022 25157 raft_consensus.cc:606] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Received request to transfer leadership to TS ac08c897ed7e478bbd7852f67f11caff
I20260709 13:09:19.352454 25302 leader_election.cc:290] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 3 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:19.353016 25156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:19.353729 25156 raft_consensus.cc:3060] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:19.355225 25229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161bbc3aba054bcdabb265faed76e09e" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:19.355908 25229 raft_consensus.cc:3055] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:19.356245 25229 raft_consensus.cc:740] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c3395c72252c4138b3d77bf5f0d7d630, State: Running, Role: LEADER
I20260709 13:09:19.357167 25229 consensus_queue.cc:260] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:19.361747 25229 raft_consensus.cc:3060] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:19.363122 12725 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:09:19.364003 12725 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:19.369450 25156 raft_consensus.cc:2468] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 3.
I20260709 13:09:19.371184 25270 leader_election.cc:304] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, bca68f82aedd42f5bf4c4fdfcb0c72db; no voters: 
I20260709 13:09:19.372404 25581 raft_consensus.cc:2804] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:19.372730 25229 raft_consensus.cc:2468] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 3.
I20260709 13:09:19.372990 25581 raft_consensus.cc:697] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 3 LEADER]: Becoming Leader. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:19.374192 25581 consensus_queue.cc:237] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:19.376116 25054 catalog_manager.cc:6242] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34: soft_deleted_table [id=6e8ed4a8e8964071ace4409752b5c43a] alter complete (version 1)
I20260709 13:09:19.388972 25054 catalog_manager.cc:5697] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff reported cstate change: term changed from 2 to 3, leader changed from c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66) to ac08c897ed7e478bbd7852f67f11caff (127.12.109.67). New cstate: current_term: 3 leader_uuid: "ac08c897ed7e478bbd7852f67f11caff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:19.506021 25559 raft_consensus.cc:993] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db: : Instructing follower ac08c897ed7e478bbd7852f67f11caff to start an election
I20260709 13:09:19.506714 25556 raft_consensus.cc:1081] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Signalling peer ac08c897ed7e478bbd7852f67f11caff to start an election
I20260709 13:09:19.508234 25302 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33e80f79ae7547a7a8e6bd6fd8d95dcb"
dest_uuid: "ac08c897ed7e478bbd7852f67f11caff"
 from {username='slave'} at 127.0.0.1:58254
I20260709 13:09:19.508761 25302 raft_consensus.cc:493] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:19.509130 25302 raft_consensus.cc:3060] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:19.515107 25302 raft_consensus.cc:515] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:19.516966 25302 leader_election.cc:290] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 3 election: Requested vote from peers bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649), c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153)
I20260709 13:09:19.517972 25156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e80f79ae7547a7a8e6bd6fd8d95dcb" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db"
I20260709 13:09:19.518385 25230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e80f79ae7547a7a8e6bd6fd8d95dcb" candidate_uuid: "ac08c897ed7e478bbd7852f67f11caff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c3395c72252c4138b3d77bf5f0d7d630"
I20260709 13:09:19.518723 25156 raft_consensus.cc:3055] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:19.519083 25156 raft_consensus.cc:740] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bca68f82aedd42f5bf4c4fdfcb0c72db, State: Running, Role: LEADER
I20260709 13:09:19.519080 25230 raft_consensus.cc:3060] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:19.519902 25156 consensus_queue.cc:260] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:19.521152 25156 raft_consensus.cc:3060] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:19.526576 25230 raft_consensus.cc:2468] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 3.
I20260709 13:09:19.527797 25270 leader_election.cc:304] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac08c897ed7e478bbd7852f67f11caff, c3395c72252c4138b3d77bf5f0d7d630; no voters: 
I20260709 13:09:19.527889 25156 raft_consensus.cc:2468] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac08c897ed7e478bbd7852f67f11caff in term 3.
I20260709 13:09:19.528636 25394 raft_consensus.cc:2804] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:19.529109 25394 raft_consensus.cc:697] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 3 LEADER]: Becoming Leader. State: Replica: ac08c897ed7e478bbd7852f67f11caff, State: Running, Role: LEADER
I20260709 13:09:19.529825 25394 consensus_queue.cc:237] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } }
I20260709 13:09:19.540915 25054 catalog_manager.cc:5697] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff reported cstate change: term changed from 2 to 3, leader changed from bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65) to ac08c897ed7e478bbd7852f67f11caff (127.12.109.67). New cstate: current_term: 3 leader_uuid: "ac08c897ed7e478bbd7852f67f11caff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac08c897ed7e478bbd7852f67f11caff" member_type: VOTER last_known_addr { host: "127.12.109.67" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:19.788964 25156 raft_consensus.cc:1275] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 3 FOLLOWER]: Refusing update from remote peer ac08c897ed7e478bbd7852f67f11caff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:19.790457 25394 consensus_queue.cc:1048] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:19.803383 25230 raft_consensus.cc:1275] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 3 FOLLOWER]: Refusing update from remote peer ac08c897ed7e478bbd7852f67f11caff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:19.804806 25394 consensus_queue.cc:1048] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:20.061503 25230 raft_consensus.cc:1275] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 3 FOLLOWER]: Refusing update from remote peer ac08c897ed7e478bbd7852f67f11caff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:20.063011 25394 consensus_queue.cc:1048] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3395c72252c4138b3d77bf5f0d7d630" member_type: VOTER last_known_addr { host: "127.12.109.66" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:20.074700 25156 raft_consensus.cc:1275] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 3 FOLLOWER]: Refusing update from remote peer ac08c897ed7e478bbd7852f67f11caff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:20.077044 25394 consensus_queue.cc:1048] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca68f82aedd42f5bf4c4fdfcb0c72db" member_type: VOTER last_known_addr { host: "127.12.109.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:20.429453 12725 tablet_server.cc:179] TabletServer@127.12.109.65:0 shutting down...
I20260709 13:09:20.463322 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:20.464299 12725 tablet_replica.cc:333] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.465299 12725 raft_consensus.cc:2243] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.465910 12725 raft_consensus.cc:2272] T 7f19f2e70f5e4b189c52a968eeb7c61b P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.468417 12725 tablet_replica.cc:333] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.469067 12725 raft_consensus.cc:2243] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.469662 12725 raft_consensus.cc:2272] T 97e2ed0f2196447f997303c0889538af P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.471899 12725 tablet_replica.cc:333] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.472535 12725 raft_consensus.cc:2243] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.473372 12725 raft_consensus.cc:2272] T 0ff976e9e6d84f4e9708af71a410e275 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.475481 12725 tablet_replica.cc:333] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.476078 12725 raft_consensus.cc:2243] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.476536 12725 raft_consensus.cc:2272] T 34d2357c56f641e48b3aba13247ec1d1 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.478873 12725 tablet_replica.cc:333] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.479528 12725 raft_consensus.cc:2243] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.480002 12725 raft_consensus.cc:2272] T 034d7d105c1b4cfbbe230cfd8011cec9 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.482445 12725 tablet_replica.cc:333] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.483023 12725 raft_consensus.cc:2243] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.483516 12725 raft_consensus.cc:2272] T f79486558b854673aa41d16a1494868e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.485925 12725 tablet_replica.cc:333] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.486763 12725 raft_consensus.cc:2243] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.487821 12725 raft_consensus.cc:2272] T 76ad5adfa4824e669ae7414ebd491691 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.490643 12725 tablet_replica.cc:333] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.491513 12725 raft_consensus.cc:2243] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.492636 12725 raft_consensus.cc:2272] T e64433f5d83d40a2be74255a152a160a P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.495467 12725 tablet_replica.cc:333] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.496073 12725 raft_consensus.cc:2243] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.497059 12725 raft_consensus.cc:2272] T 7dbe909446ac4c1b8e329828cc311a52 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.499199 12725 tablet_replica.cc:333] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.499876 12725 raft_consensus.cc:2243] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.500285 12725 raft_consensus.cc:2272] T 25090e4349ea43b5a76c9295ae73eaab P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.502300 12725 tablet_replica.cc:333] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.502805 12725 raft_consensus.cc:2243] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.503364 12725 raft_consensus.cc:2272] T 161bbc3aba054bcdabb265faed76e09e P bca68f82aedd42f5bf4c4fdfcb0c72db [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.506947 12725 tablet_replica.cc:333] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.507588 12725 raft_consensus.cc:2243] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.508105 12725 raft_consensus.cc:2272] T 58b3849e07f54c6ab1df11556d0c7212 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.510335 12725 tablet_replica.cc:333] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.510895 12725 raft_consensus.cc:2243] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.511359 12725 raft_consensus.cc:2272] T 304954defc094f77bd6356c5b30b2d56 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.513295 12725 tablet_replica.cc:333] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.513883 12725 raft_consensus.cc:2243] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.514845 12725 raft_consensus.cc:2272] T afc98daf1f6a48d7a59b85bf860351e4 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.517131 12725 tablet_replica.cc:333] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.517668 12725 raft_consensus.cc:2243] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.518169 12725 raft_consensus.cc:2272] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P bca68f82aedd42f5bf4c4fdfcb0c72db [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.520968 12725 tablet_replica.cc:333] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.521543 12725 raft_consensus.cc:2243] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.522018 12725 raft_consensus.cc:2272] T dd30bf4abddf45c38bd41a6dbfbed3ac P bca68f82aedd42f5bf4c4fdfcb0c72db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.524565 12725 tablet_replica.cc:333] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.525170 12725 raft_consensus.cc:2243] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.526131 12725 raft_consensus.cc:2272] T 8a417912724d4279b7b1d1e1ddfcec1c P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.528266 12725 tablet_replica.cc:333] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db: stopping tablet replica
I20260709 13:09:20.528781 12725 raft_consensus.cc:2243] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.529657 12725 raft_consensus.cc:2272] T 9aa95aeebb5246e49558118198cf4d01 P bca68f82aedd42f5bf4c4fdfcb0c72db [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:20.551728 25196 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 10 similar messages]
W20260709 13:09:20.555509 25196 consensus_peers.cc:597] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 -> Peer bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649): Couldn't send request to peer bca68f82aedd42f5bf4c4fdfcb0c72db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:20.575496 12725 tablet_server.cc:196] TabletServer@127.12.109.65:0 shutdown complete.
W20260709 13:09:20.602383 25196 consensus_peers.cc:597] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 -> Peer bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649): Couldn't send request to peer bca68f82aedd42f5bf4c4fdfcb0c72db. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:37649: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:20.615391 12725 tablet_server.cc:179] TabletServer@127.12.109.66:0 shutting down...
W20260709 13:09:20.624791 25270 consensus_peers.cc:597] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff -> Peer bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649): Couldn't send request to peer bca68f82aedd42f5bf4c4fdfcb0c72db. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:37649: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:20.653611 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:20.654628 12725 tablet_replica.cc:333] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.655370 12725 raft_consensus.cc:2243] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.656339 12725 raft_consensus.cc:2272] T 7f19f2e70f5e4b189c52a968eeb7c61b P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.658634 12725 tablet_replica.cc:333] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.659289 12725 raft_consensus.cc:2243] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.660147 12725 raft_consensus.cc:2272] T 25090e4349ea43b5a76c9295ae73eaab P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.662309 12725 tablet_replica.cc:333] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.662949 12725 raft_consensus.cc:2243] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.663414 12725 raft_consensus.cc:2272] T 7dbe909446ac4c1b8e329828cc311a52 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.665416 12725 tablet_replica.cc:333] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.666002 12725 raft_consensus.cc:2243] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.666875 12725 raft_consensus.cc:2272] T 97e2ed0f2196447f997303c0889538af P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.668956 12725 tablet_replica.cc:333] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.669703 12725 raft_consensus.cc:2243] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.670359 12725 raft_consensus.cc:2272] T 0ff976e9e6d84f4e9708af71a410e275 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.672559 12725 tablet_replica.cc:333] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.673172 12725 raft_consensus.cc:2243] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.673671 12725 raft_consensus.cc:2272] T 34d2357c56f641e48b3aba13247ec1d1 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.675690 12725 tablet_replica.cc:333] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.676527 12725 raft_consensus.cc:2243] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.677415 12725 raft_consensus.cc:2272] T f79486558b854673aa41d16a1494868e P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.679903 12725 tablet_replica.cc:333] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.680519 12725 raft_consensus.cc:2243] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.681355 12725 raft_consensus.cc:2272] T 034d7d105c1b4cfbbe230cfd8011cec9 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.683969 12725 tablet_replica.cc:333] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.684561 12725 raft_consensus.cc:2243] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.685093 12725 raft_consensus.cc:2272] T afc98daf1f6a48d7a59b85bf860351e4 P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.687580 12725 tablet_replica.cc:333] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.688167 12725 raft_consensus.cc:2243] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.689002 12725 raft_consensus.cc:2272] T 304954defc094f77bd6356c5b30b2d56 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.691071 12725 tablet_replica.cc:333] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.691639 12725 raft_consensus.cc:2243] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.692550 12725 raft_consensus.cc:2272] T 58b3849e07f54c6ab1df11556d0c7212 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.695026 12725 tablet_replica.cc:333] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.695716 12725 raft_consensus.cc:2243] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.696257 12725 raft_consensus.cc:2272] T 161bbc3aba054bcdabb265faed76e09e P c3395c72252c4138b3d77bf5f0d7d630 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.699774 12725 tablet_replica.cc:333] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
W20260709 13:09:20.699812 25270 consensus_peers.cc:597] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff -> Peer c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153): Couldn't send request to peer c3395c72252c4138b3d77bf5f0d7d630. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:20.700469 12725 raft_consensus.cc:2243] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.701018 12725 raft_consensus.cc:2272] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P c3395c72252c4138b3d77bf5f0d7d630 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.704411 12725 tablet_replica.cc:333] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.704993 12725 raft_consensus.cc:2243] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.705474 12725 raft_consensus.cc:2272] T 8a417912724d4279b7b1d1e1ddfcec1c P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.707563 12725 tablet_replica.cc:333] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.708127 12725 raft_consensus.cc:2243] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.708600 12725 raft_consensus.cc:2272] T 9aa95aeebb5246e49558118198cf4d01 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.710789 12725 tablet_replica.cc:333] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.711306 12725 raft_consensus.cc:2243] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.711750 12725 raft_consensus.cc:2272] T dd30bf4abddf45c38bd41a6dbfbed3ac P c3395c72252c4138b3d77bf5f0d7d630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.713896 12725 tablet_replica.cc:333] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.714480 12725 raft_consensus.cc:2243] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.714977 12725 raft_consensus.cc:2272] T 76ad5adfa4824e669ae7414ebd491691 P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.716957 12725 tablet_replica.cc:333] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630: stopping tablet replica
I20260709 13:09:20.717471 12725 raft_consensus.cc:2243] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.717962 12725 raft_consensus.cc:2272] T e64433f5d83d40a2be74255a152a160a P c3395c72252c4138b3d77bf5f0d7d630 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:20.736814 25270 consensus_peers.cc:597] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff -> Peer c3395c72252c4138b3d77bf5f0d7d630 (127.12.109.66:36153): Couldn't send request to peer c3395c72252c4138b3d77bf5f0d7d630. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:20.762151 12725 tablet_server.cc:196] TabletServer@127.12.109.66:0 shutdown complete.
W20260709 13:09:20.776670 25270 consensus_peers.cc:597] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff -> Peer bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649): Couldn't send request to peer bca68f82aedd42f5bf4c4fdfcb0c72db. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:37649: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:20.802409 12725 tablet_server.cc:179] TabletServer@127.12.109.67:0 shutting down...
W20260709 13:09:20.825071 25270 consensus_peers.cc:597] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff -> Peer bca68f82aedd42f5bf4c4fdfcb0c72db (127.12.109.65:37649): Couldn't send request to peer bca68f82aedd42f5bf4c4fdfcb0c72db. Status: Network error: Client connection negotiation failed: client connection to 127.12.109.65:37649: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:20.836845 12725 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:20.837713 12725 tablet_replica.cc:333] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.838487 12725 raft_consensus.cc:2243] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.839056 12725 raft_consensus.cc:2272] T 7f19f2e70f5e4b189c52a968eeb7c61b P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.841337 12725 tablet_replica.cc:333] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.842221 12725 raft_consensus.cc:2243] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.842701 12725 raft_consensus.cc:2272] T 97e2ed0f2196447f997303c0889538af P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.844769 12725 tablet_replica.cc:333] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.845357 12725 raft_consensus.cc:2243] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.845829 12725 raft_consensus.cc:2272] T 0ff976e9e6d84f4e9708af71a410e275 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.847815 12725 tablet_replica.cc:333] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.848522 12725 raft_consensus.cc:2243] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.849414 12725 raft_consensus.cc:2272] T 33e80f79ae7547a7a8e6bd6fd8d95dcb P ac08c897ed7e478bbd7852f67f11caff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.852250 12725 tablet_replica.cc:333] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.852898 12725 raft_consensus.cc:2243] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.853399 12725 raft_consensus.cc:2272] T 304954defc094f77bd6356c5b30b2d56 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.855516 12725 tablet_replica.cc:333] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.856148 12725 raft_consensus.cc:2243] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.856642 12725 raft_consensus.cc:2272] T afc98daf1f6a48d7a59b85bf860351e4 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.858978 12725 tablet_replica.cc:333] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.859614 12725 raft_consensus.cc:2243] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.860514 12725 raft_consensus.cc:2272] T 161bbc3aba054bcdabb265faed76e09e P ac08c897ed7e478bbd7852f67f11caff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.862833 12725 tablet_replica.cc:333] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.863451 12725 raft_consensus.cc:2243] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.863952 12725 raft_consensus.cc:2272] T 58b3849e07f54c6ab1df11556d0c7212 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.866330 12725 tablet_replica.cc:333] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.866938 12725 raft_consensus.cc:2243] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.867805 12725 raft_consensus.cc:2272] T 34d2357c56f641e48b3aba13247ec1d1 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.869870 12725 tablet_replica.cc:333] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.870579 12725 raft_consensus.cc:2243] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.871040 12725 raft_consensus.cc:2272] T 034d7d105c1b4cfbbe230cfd8011cec9 P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.873253 12725 tablet_replica.cc:333] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.873822 12725 raft_consensus.cc:2243] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.874311 12725 raft_consensus.cc:2272] T f79486558b854673aa41d16a1494868e P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.876500 12725 tablet_replica.cc:333] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.877072 12725 raft_consensus.cc:2243] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.877547 12725 raft_consensus.cc:2272] T 76ad5adfa4824e669ae7414ebd491691 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.879531 12725 tablet_replica.cc:333] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.880095 12725 raft_consensus.cc:2243] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.880548 12725 raft_consensus.cc:2272] T e64433f5d83d40a2be74255a152a160a P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.882754 12725 tablet_replica.cc:333] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.883288 12725 raft_consensus.cc:2243] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.883747 12725 raft_consensus.cc:2272] T 25090e4349ea43b5a76c9295ae73eaab P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.885869 12725 tablet_replica.cc:333] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.886544 12725 raft_consensus.cc:2243] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.886973 12725 raft_consensus.cc:2272] T 7dbe909446ac4c1b8e329828cc311a52 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.888993 12725 tablet_replica.cc:333] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.889521 12725 raft_consensus.cc:2243] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.889906 12725 raft_consensus.cc:2272] T 8a417912724d4279b7b1d1e1ddfcec1c P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.891850 12725 tablet_replica.cc:333] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.892366 12725 raft_consensus.cc:2243] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:20.892773 12725 raft_consensus.cc:2272] T 9aa95aeebb5246e49558118198cf4d01 P ac08c897ed7e478bbd7852f67f11caff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.894963 12725 tablet_replica.cc:333] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff: stopping tablet replica
I20260709 13:09:20.895432 12725 raft_consensus.cc:2243] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:20.896276 12725 raft_consensus.cc:2272] T dd30bf4abddf45c38bd41a6dbfbed3ac P ac08c897ed7e478bbd7852f67f11caff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:20.941154 12725 tablet_server.cc:196] TabletServer@127.12.109.67:0 shutdown complete.
I20260709 13:09:20.979413 12725 master.cc:562] Master@127.12.109.126:43651 shutting down...
I20260709 13:09:21.005599 12725 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:21.006297 12725 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:21.006649 12725 tablet_replica.cc:333] T 00000000000000000000000000000000 P f047b5858163488e82044de7758dcc34: stopping tablet replica
I20260709 13:09:21.028113 12725 master.cc:584] Master@127.12.109.126:43651 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (12332 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (25410 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (372069 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:09:21.090529 12725 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 411 messages since previous log ~15 seconds ago
I20260709 13:09:21.090751 12725 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 411 messages since previous log ~15 seconds ago
I20260709 13:09:21.090902 12725 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 410 messages since previous log ~15 seconds ago
I20260709 13:09:21.091063 12725 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:09:21.091218 12725 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:09:21.091388 12725 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 4 messages since previous log ~128 seconds ago
I20260709 13:09:21.091539 12725 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 2 messages since previous log ~152 seconds ago
I20260709 13:09:21.091687 12725 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 2 messages since previous log ~153 seconds ago
I20260709 13:09:21.091918 12725 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 6 messages since previous log ~0 seconds ago
I20260709 13:09:21.092222 12725 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 26 messages since previous log ~6 seconds ago
